2023-08-25T03:20:16.134 INFO:root:teuthology version: 0.0.1.dev185+g449a1bc 2023-08-25T03:20:16.136 DEBUG:teuthology.run:Teuthology command: teuthology --name rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379011 --owner scheduled_rishabh@teuthology --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota} -- /home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379011/orig.config.yaml 2023-08-25T03:20:16.201 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-25T03:20:16.292 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379011 branch: wip-rishabh-2023aug3-b5 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota} email: null first_in_suite: false job_id: '7379011' kernel: client: sha1: distro flavor: default kdb: true sha1: a6a69670761025e039dc5ae86619b0cf04c46dc2 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-2023aug3-b5 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: ba6b39e4f4f77fb95c747099b53db0467d435f56 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: ba6b39e4f4f77fb95c747099b53db0467d435f56 kclient: syntax: v1 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-2023aug3-b5 sha1: ba6b39e4f4f77fb95c747099b53db0467d435f56 owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: ba6b39e4f4f77fb95c747099b53db0467d435f56 sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-2023aug3-b5 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_ba6b39e4f4f77fb95c747099b53db0467d435f56/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: ba6b39e4f4f77fb95c747099b53db0467d435f56 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi002.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_quota teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 449a1bc2027504e7b3c3d7b30fa4178906581da7 timestamp: 2023-08-25_01:50:32 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2224 2023-08-25T03:20:16.292 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_ba6b39e4f4f77fb95c747099b53db0467d435f56/qa; will attempt to use it 2023-08-25T03:20:16.293 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_ba6b39e4f4f77fb95c747099b53db0467d435f56/qa/tasks 2023-08-25T03:20:16.294 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-08-25T03:20:16.296 INFO:teuthology.task.internal:Checking packages... 2023-08-25T03:20:16.336 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'ba6b39e4f4f77fb95c747099b53db0467d435f56' 2023-08-25T03:20:16.336 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-08-25T03:20:16.336 INFO:teuthology.packaging:ref: None 2023-08-25T03:20:16.337 INFO:teuthology.packaging:tag: None 2023-08-25T03:20:16.337 INFO:teuthology.packaging:branch: wip-rishabh-2023aug3-b5 2023-08-25T03:20:16.337 INFO:teuthology.packaging:sha1: ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T03:20:16.337 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-2023aug3-b5 2023-08-25T03:20:16.569 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-5703.gba6b39e4 2023-08-25T03:20:16.569 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-08-25T03:20:16.575 INFO:teuthology.task.internal:no buildpackages task found 2023-08-25T03:20:16.576 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-08-25T03:20:16.602 INFO:teuthology.task.internal:Saving configuration 2023-08-25T03:20:16.619 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-08-25T03:20:16.698 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-25T03:20:16.725 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi002.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379011', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-25 03:15:23.904500', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-08-25T03:20:16.752 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379011', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-25 03:15:23.906528', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-08-25T03:20:16.753 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-08-25T03:20:16.766 INFO:teuthology.task.internal:roles: ubuntu@smithi002.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-08-25T03:20:16.767 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-08-25T03:20:16.767 INFO:teuthology.run_tasks:Running task console_log... 2023-08-25T03:20:16.847 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa11e9f2f70>, signals=[15]) 2023-08-25T03:20:16.849 INFO:teuthology.run_tasks:Running task internal.connect... 2023-08-25T03:20:16.855 INFO:teuthology.task.internal:Opening connections... 2023-08-25T03:20:16.855 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi002.front.sepia.ceph.com 2023-08-25T03:20:16.857 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T03:20:16.930 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi031.front.sepia.ceph.com 2023-08-25T03:20:16.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T03:20:16.999 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-08-25T03:20:17.011 DEBUG:teuthology.orchestra.run.smithi002:> uname -m 2023-08-25T03:20:17.033 INFO:teuthology.orchestra.run.smithi002.stdout:x86_64 2023-08-25T03:20:17.034 DEBUG:teuthology.orchestra.run.smithi002:> cat /etc/os-release 2023-08-25T03:20:17.094 INFO:teuthology.orchestra.run.smithi002.stdout:NAME="Red Hat Enterprise Linux" 2023-08-25T03:20:17.095 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION="8.6 (Ootpa)" 2023-08-25T03:20:17.095 INFO:teuthology.orchestra.run.smithi002.stdout:ID="rhel" 2023-08-25T03:20:17.095 INFO:teuthology.orchestra.run.smithi002.stdout:ID_LIKE="fedora" 2023-08-25T03:20:17.095 INFO:teuthology.orchestra.run.smithi002.stdout:VERSION_ID="8.6" 2023-08-25T03:20:17.096 INFO:teuthology.orchestra.run.smithi002.stdout:PLATFORM_ID="platform:el8" 2023-08-25T03:20:17.096 INFO:teuthology.orchestra.run.smithi002.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-08-25T03:20:17.096 INFO:teuthology.orchestra.run.smithi002.stdout:ANSI_COLOR="0;31" 2023-08-25T03:20:17.097 INFO:teuthology.orchestra.run.smithi002.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-08-25T03:20:17.097 INFO:teuthology.orchestra.run.smithi002.stdout:HOME_URL="https://www.redhat.com/" 2023-08-25T03:20:17.097 INFO:teuthology.orchestra.run.smithi002.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-08-25T03:20:17.097 INFO:teuthology.orchestra.run.smithi002.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-25T03:20:17.097 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:20:17.097 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-25T03:20:17.097 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-08-25T03:20:17.098 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-08-25T03:20:17.098 INFO:teuthology.orchestra.run.smithi002.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-08-25T03:20:17.098 INFO:teuthology.lock.ops:Updating smithi002.front.sepia.ceph.com on lock server 2023-08-25T03:20:17.116 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2023-08-25T03:20:17.137 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2023-08-25T03:20:17.137 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2023-08-25T03:20:17.196 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="Red Hat Enterprise Linux" 2023-08-25T03:20:17.197 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="8.6 (Ootpa)" 2023-08-25T03:20:17.197 INFO:teuthology.orchestra.run.smithi031.stdout:ID="rhel" 2023-08-25T03:20:17.197 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="fedora" 2023-08-25T03:20:17.197 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="8.6" 2023-08-25T03:20:17.197 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el8" 2023-08-25T03:20:17.197 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-08-25T03:20:17.197 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2023-08-25T03:20:17.198 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-08-25T03:20:17.198 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://www.redhat.com/" 2023-08-25T03:20:17.198 INFO:teuthology.orchestra.run.smithi031.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-08-25T03:20:17.198 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-08-25T03:20:17.198 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:20:17.198 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-08-25T03:20:17.198 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-08-25T03:20:17.199 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-08-25T03:20:17.199 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-08-25T03:20:17.199 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2023-08-25T03:20:17.224 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-08-25T03:20:17.233 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-08-25T03:20:17.239 INFO:teuthology.task.internal:Checking for old test directory... 2023-08-25T03:20:17.239 DEBUG:teuthology.orchestra.run.smithi002:> test '!' -e /home/ubuntu/cephtest 2023-08-25T03:20:17.243 DEBUG:teuthology.orchestra.run.smithi031:> test '!' -e /home/ubuntu/cephtest 2023-08-25T03:20:17.265 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-08-25T03:20:17.273 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-08-25T03:20:17.274 DEBUG:teuthology.orchestra.run.smithi002:> test -z $(ls -A /var/lib/ceph) 2023-08-25T03:20:17.304 DEBUG:teuthology.orchestra.run.smithi031:> test -z $(ls -A /var/lib/ceph) 2023-08-25T03:20:17.333 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-08-25T03:20:17.390 INFO:teuthology.run_tasks:Running task kernel... 2023-08-25T03:20:17.416 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'a6a69670761025e039dc5ae86619b0cf04c46dc2'} 2023-08-25T03:20:17.417 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-08-25T03:20:17.417 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-08-25T03:20:17.418 DEBUG:teuthology.orchestra.run.smithi002:> test -f /run/.containerenv -o -f /.dockerenv 2023-08-25T03:20:17.418 DEBUG:teuthology.orchestra.run.smithi031:> test -f /run/.containerenv -o -f /.dockerenv 2023-08-25T03:20:17.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T03:20:17.438 DEBUG:teuthology.orchestra.run.smithi002:> uname -r 2023-08-25T03:20:17.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T03:20:17.440 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2023-08-25T03:20:17.500 INFO:teuthology.orchestra.run.smithi031.stdout:4.18.0-372.9.1.el8.x86_64 2023-08-25T03:20:17.501 INFO:teuthology.task.kernel:Running kernel on smithi031: 4.18.0-372.9.1.el8.x86_64 2023-08-25T03:20:17.501 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum install -y kernel 2023-08-25T03:20:17.502 INFO:teuthology.orchestra.run.smithi002.stdout:4.18.0-372.9.1.el8.x86_64 2023-08-25T03:20:17.503 INFO:teuthology.task.kernel:Running kernel on smithi002: 4.18.0-372.9.1.el8.x86_64 2023-08-25T03:20:17.503 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum install -y kernel 2023-08-25T03:20:18.679 INFO:teuthology.orchestra.run.smithi002.stdout:Updating Subscription Management repositories. 2023-08-25T03:20:18.680 INFO:teuthology.orchestra.run.smithi002.stdout:Unable to read consumer identity 2023-08-25T03:20:18.695 INFO:teuthology.orchestra.run.smithi031.stdout:Updating Subscription Management repositories. 2023-08-25T03:20:18.695 INFO:teuthology.orchestra.run.smithi031.stdout:Unable to read consumer identity 2023-08-25T03:20:20.174 INFO:teuthology.orchestra.run.smithi031.stdout:Copr repo for python3-asyncssh owned by ceph 20 kB/s | 3.5 kB 00:00 2023-08-25T03:20:20.178 INFO:teuthology.orchestra.run.smithi002.stdout:Copr repo for python3-asyncssh owned by ceph 14 kB/s | 3.5 kB 00:00 2023-08-25T03:20:20.925 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 22 MB/s | 16 MB 00:00 2023-08-25T03:20:21.246 INFO:teuthology.orchestra.run.smithi002.stdout:Extra Packages for Enterprise Linux 15 MB/s | 16 MB 00:01 2023-08-25T03:20:24.610 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 383 kB/s | 24 kB 00:00 2023-08-25T03:20:24.912 INFO:teuthology.orchestra.run.smithi002.stdout:lab-extras 465 kB/s | 24 kB 00:00 2023-08-25T03:20:25.403 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-08-25T03:20:25.453 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2023-08-25T03:20:25.454 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2023-08-25T03:20:25.454 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2023-08-25T03:20:25.615 INFO:teuthology.orchestra.run.smithi002.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-08-25T03:20:25.620 DEBUG:teuthology.orchestra.run.smithi031:> echo no | sudo yum reinstall kernel || true 2023-08-25T03:20:25.672 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2023-08-25T03:20:25.674 INFO:teuthology.orchestra.run.smithi002.stdout:Nothing to do. 2023-08-25T03:20:25.674 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2023-08-25T03:20:25.793 DEBUG:teuthology.orchestra.run.smithi002:> echo no | sudo yum reinstall kernel || true 2023-08-25T03:20:25.969 INFO:teuthology.orchestra.run.smithi031.stdout:Updating Subscription Management repositories. 2023-08-25T03:20:25.969 INFO:teuthology.orchestra.run.smithi031.stdout:Unable to read consumer identity 2023-08-25T03:20:26.186 INFO:teuthology.orchestra.run.smithi002.stdout:Updating Subscription Management repositories. 2023-08-25T03:20:26.187 INFO:teuthology.orchestra.run.smithi002.stdout:Unable to read consumer identity 2023-08-25T03:20:26.242 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:02 ago on Fri 25 Aug 2023 03:20:24 AM UTC. 2023-08-25T03:20:26.287 INFO:teuthology.orchestra.run.smithi031.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-08-25T03:20:26.345 INFO:teuthology.orchestra.run.smithi031.stderr:Error: No packages marked for reinstall. 2023-08-25T03:20:26.392 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum reinstall -y kernel || true 2023-08-25T03:20:26.459 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:00:02 ago on Fri 25 Aug 2023 03:20:24 AM UTC. 2023-08-25T03:20:26.503 INFO:teuthology.orchestra.run.smithi002.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-08-25T03:20:26.553 INFO:teuthology.orchestra.run.smithi002.stderr:Error: No packages marked for reinstall. 2023-08-25T03:20:26.600 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum reinstall -y kernel || true 2023-08-25T03:20:26.742 INFO:teuthology.orchestra.run.smithi031.stdout:Updating Subscription Management repositories. 2023-08-25T03:20:26.742 INFO:teuthology.orchestra.run.smithi031.stdout:Unable to read consumer identity 2023-08-25T03:20:26.949 INFO:teuthology.orchestra.run.smithi002.stdout:Updating Subscription Management repositories. 2023-08-25T03:20:26.949 INFO:teuthology.orchestra.run.smithi002.stdout:Unable to read consumer identity 2023-08-25T03:20:27.011 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:03 ago on Fri 25 Aug 2023 03:20:24 AM UTC. 2023-08-25T03:20:27.052 INFO:teuthology.orchestra.run.smithi031.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-08-25T03:20:27.084 INFO:teuthology.orchestra.run.smithi031.stderr:Error: No packages marked for reinstall. 2023-08-25T03:20:27.132 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2023-08-25T03:20:27.215 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-08-25T03:20:27.216 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-08-25T03:20:27.216 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-08-25T03:20:27.216 DEBUG:teuthology.parallel:result is None 2023-08-25T03:20:27.218 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:00:03 ago on Fri 25 Aug 2023 03:20:24 AM UTC. 2023-08-25T03:20:27.261 INFO:teuthology.orchestra.run.smithi002.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-08-25T03:20:27.286 INFO:teuthology.orchestra.run.smithi002.stderr:Error: No packages marked for reinstall. 2023-08-25T03:20:27.338 DEBUG:teuthology.orchestra.run.smithi002:> rpm -q kernel | sort -rV | head -n 1 2023-08-25T03:20:27.422 INFO:teuthology.orchestra.run.smithi002.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-08-25T03:20:27.422 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-08-25T03:20:27.423 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-08-25T03:20:27.423 DEBUG:teuthology.parallel:result is None 2023-08-25T03:20:27.423 INFO:teuthology.run_tasks:Running task internal.base... 2023-08-25T03:20:27.433 INFO:teuthology.task.internal:Creating test directory... 2023-08-25T03:20:27.433 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-08-25T03:20:27.436 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-08-25T03:20:27.457 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-08-25T03:20:27.463 INFO:teuthology.run_tasks:Running task internal.archive... 2023-08-25T03:20:27.578 INFO:teuthology.task.internal:Creating archive directory... 2023-08-25T03:20:27.579 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-08-25T03:20:27.581 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-08-25T03:20:27.606 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-08-25T03:20:27.612 INFO:teuthology.task.internal:Enabling coredump saving... 2023-08-25T03:20:27.613 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-08-25T03:20:27.639 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-08-25T03:20:27.676 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-25T03:20:27.686 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-25T03:20:27.691 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-25T03:20:27.698 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-08-25T03:20:27.699 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-08-25T03:20:27.706 INFO:teuthology.task.internal:Configuring sudo... 2023-08-25T03:20:27.706 DEBUG:teuthology.orchestra.run.smithi002:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-08-25T03:20:27.734 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-08-25T03:20:27.762 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-08-25T03:20:27.771 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-08-25T03:20:27.772 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-08-25T03:20:27.802 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-08-25T03:20:27.820 DEBUG:teuthology.orchestra.run.smithi002:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-25T03:20:27.897 DEBUG:teuthology.orchestra.run.smithi002:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-25T03:20:27.976 DEBUG:teuthology.orchestra.run.smithi002:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-25T03:20:28.015 DEBUG:teuthology.orchestra.run.smithi002:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-25T03:20:28.061 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:20:28.061 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-08-25T03:20:28.138 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-25T03:20:28.188 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-08-25T03:20:28.356 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-25T03:20:28.404 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-08-25T03:20:28.447 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:20:28.448 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-08-25T03:20:28.528 DEBUG:teuthology.orchestra.run.smithi002:> sudo service rsyslog restart 2023-08-25T03:20:28.530 DEBUG:teuthology.orchestra.run.smithi031:> sudo service rsyslog restart 2023-08-25T03:20:28.607 INFO:teuthology.orchestra.run.smithi002.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-25T03:20:28.629 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-25T03:20:28.980 INFO:teuthology.run_tasks:Running task internal.timer... 2023-08-25T03:20:28.988 INFO:teuthology.task.internal:Starting timer... 2023-08-25T03:20:28.988 INFO:teuthology.run_tasks:Running task pcp... 2023-08-25T03:20:29.101 INFO:teuthology.run_tasks:Running task selinux... 2023-08-25T03:20:29.119 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-08-25T03:20:29.120 DEBUG:teuthology.orchestra.run.smithi002:> sudo service auditd rotate 2023-08-25T03:20:29.198 INFO:teuthology.orchestra.run.smithi002.stdout:Rotating logs: 2023-08-25T03:20:29.199 DEBUG:teuthology.orchestra.run.smithi031:> sudo service auditd rotate 2023-08-25T03:20:29.254 INFO:teuthology.orchestra.run.smithi031.stdout:Rotating logs: 2023-08-25T03:20:29.256 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-08-25T03:20:29.256 DEBUG:teuthology.orchestra.run.smithi002:> /usr/sbin/getenforce 2023-08-25T03:20:29.290 INFO:teuthology.orchestra.run.smithi002.stdout:Permissive 2023-08-25T03:20:29.291 DEBUG:teuthology.orchestra.run.smithi031:> /usr/sbin/getenforce 2023-08-25T03:20:29.309 INFO:teuthology.orchestra.run.smithi031.stdout:Permissive 2023-08-25T03:20:29.310 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi002.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi031.front.sepia.ceph.com': 'permissive'} 2023-08-25T03:20:29.310 DEBUG:teuthology.orchestra.run.smithi002:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-25T03:20:29.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T03:20:29.360 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-25T03:20:29.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T03:20:29.389 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-08-25T03:20:29.389 DEBUG:teuthology.orchestra.run.smithi002:> sudo /usr/sbin/setenforce permissive 2023-08-25T03:20:29.429 DEBUG:teuthology.orchestra.run.smithi031:> sudo /usr/sbin/setenforce permissive 2023-08-25T03:20:29.457 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-08-25T03:20:29.468 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-08-25T03:20:29.476 INFO:teuthology.repo_utils:Fetching main from origin 2023-08-25T03:20:29.668 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-08-25T03:20:29.714 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-08-25T03:20:29.715 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi002.front.sepia.ceph.com,smithi031.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-08-25T03:30:26.281 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi002.front.sepia.ceph.com'), Remote(name='ubuntu@smithi031.front.sepia.ceph.com')] 2023-08-25T03:30:26.282 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi002.front.sepia.ceph.com' 2023-08-25T03:30:26.283 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi002.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T03:30:26.352 DEBUG:teuthology.orchestra.run.smithi002:> true 2023-08-25T03:30:26.427 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi002.front.sepia.ceph.com' 2023-08-25T03:30:26.427 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-08-25T03:30:26.428 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-08-25T03:30:26.498 DEBUG:teuthology.orchestra.run.smithi031:> true 2023-08-25T03:30:26.572 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2023-08-25T03:30:26.573 INFO:teuthology.run_tasks:Running task clock... 2023-08-25T03:30:26.585 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-08-25T03:30:26.586 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-08-25T03:30:26.586 DEBUG:teuthology.orchestra.run.smithi002:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-25T03:30:26.589 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-08-25T03:30:26.590 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-25T03:30:26.621 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-08-25T03:30:26.637 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-08-25T03:30:26.650 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-08-25T03:30:26.668 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-08-25T03:30:26.724 INFO:teuthology.orchestra.run.smithi002.stderr:sudo: ntpd: command not found 2023-08-25T03:30:26.736 INFO:teuthology.orchestra.run.smithi002.stdout:506 Cannot talk to daemon 2023-08-25T03:30:26.743 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: ntpd: command not found 2023-08-25T03:30:26.752 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-08-25T03:30:26.755 INFO:teuthology.orchestra.run.smithi031.stdout:506 Cannot talk to daemon 2023-08-25T03:30:26.766 INFO:teuthology.orchestra.run.smithi002.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-08-25T03:30:26.771 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-08-25T03:30:26.786 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-08-25T03:30:26.847 INFO:teuthology.orchestra.run.smithi002.stderr:bash: ntpq: command not found 2023-08-25T03:30:26.850 INFO:teuthology.orchestra.run.smithi002.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-25T03:30:26.850 INFO:teuthology.orchestra.run.smithi002.stdout:=============================================================================== 2023-08-25T03:30:26.850 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T03:30:26.850 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T03:30:26.850 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T03:30:26.851 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T03:30:26.865 INFO:teuthology.orchestra.run.smithi031.stderr:bash: ntpq: command not found 2023-08-25T03:30:26.869 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-25T03:30:26.869 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2023-08-25T03:30:26.869 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T03:30:26.869 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T03:30:26.869 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T03:30:26.869 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T03:30:26.870 INFO:teuthology.run_tasks:Running task install... 2023-08-25T03:30:26.879 DEBUG:teuthology.task.install:project ceph 2023-08-25T03:30:26.880 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56'}} 2023-08-25T03:30:26.880 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56'} 2023-08-25T03:30:26.880 INFO:teuthology.task.install:Using flavor: default 2023-08-25T03:30:26.885 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-08-25T03:30:26.886 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-08-25T03:30:26.887 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56', 'tag': None, 'wait_for_package': False} 2023-08-25T03:30:26.887 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T03:30:26.889 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56', 'tag': None, 'wait_for_package': False} 2023-08-25T03:30:26.889 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T03:30:27.024 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/ 2023-08-25T03:30:27.024 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5703.gba6b39e4 2023-08-25T03:30:27.098 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/ 2023-08-25T03:30:27.098 INFO:teuthology.task.install.rpm:Package version is 18.0.0-5703.gba6b39e4 2023-08-25T03:30:27.190 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-08-25T03:30:27.190 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:30:27.190 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-08-25T03:30:27.202 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023aug3-b5/ba6b39e4f4f77fb95c747099b53db0467d435f56/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-08-25T03:30:27.202 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:30:27.203 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-08-25T03:30:27.220 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-08-25T03:30:27.221 DEBUG:teuthology.orchestra.run.smithi031:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ba6b39e4f4f77fb95c747099b53db0467d435f56/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-08-25T03:30:27.232 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-08-25T03:30:27.233 DEBUG:teuthology.orchestra.run.smithi002:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/ba6b39e4f4f77fb95c747099b53db0467d435f56/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-08-25T03:30:27.291 DEBUG:teuthology.orchestra.run.smithi031:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-08-25T03:30:27.303 DEBUG:teuthology.orchestra.run.smithi002:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-08-25T03:30:27.371 DEBUG:teuthology.orchestra.run.smithi031:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-08-25T03:30:27.383 DEBUG:teuthology.orchestra.run.smithi002:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-08-25T03:30:27.440 INFO:teuthology.orchestra.run.smithi031.stdout:check_obsoletes = 1 2023-08-25T03:30:27.441 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum clean all 2023-08-25T03:30:27.450 INFO:teuthology.orchestra.run.smithi002.stdout:check_obsoletes = 1 2023-08-25T03:30:27.452 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum clean all 2023-08-25T03:30:27.835 INFO:teuthology.orchestra.run.smithi031.stdout:Updating Subscription Management repositories. 2023-08-25T03:30:27.844 INFO:teuthology.orchestra.run.smithi002.stdout:Updating Subscription Management repositories. 2023-08-25T03:30:28.697 INFO:teuthology.orchestra.run.smithi002.stdout:44 files removed 2023-08-25T03:30:28.716 INFO:teuthology.orchestra.run.smithi031.stdout:44 files removed 2023-08-25T03:30:28.752 DEBUG:teuthology.orchestra.run.smithi002:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-08-25T03:30:28.770 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-08-25T03:30:29.102 INFO:teuthology.orchestra.run.smithi002.stdout:Updating Subscription Management repositories. 2023-08-25T03:30:29.122 INFO:teuthology.orchestra.run.smithi031.stdout:Updating Subscription Management repositories. 2023-08-25T03:30:29.843 INFO:teuthology.orchestra.run.smithi002.stdout:ceph packages for x86_64 246 kB/s | 81 kB 00:00 2023-08-25T03:30:30.047 INFO:teuthology.orchestra.run.smithi031.stdout:ceph packages for x86_64 156 kB/s | 81 kB 00:00 2023-08-25T03:30:30.193 INFO:teuthology.orchestra.run.smithi002.stdout:ceph noarch packages 30 kB/s | 9.4 kB 00:00 2023-08-25T03:30:30.364 INFO:teuthology.orchestra.run.smithi031.stdout:ceph noarch packages 33 kB/s | 9.4 kB 00:00 2023-08-25T03:30:30.436 INFO:teuthology.orchestra.run.smithi002.stdout:ceph source packages 6.8 kB/s | 1.5 kB 00:00 2023-08-25T03:30:30.578 INFO:teuthology.orchestra.run.smithi002.stdout:Copr repo for python3-asyncssh owned by ceph 29 kB/s | 3.5 kB 00:00 2023-08-25T03:30:30.781 INFO:teuthology.orchestra.run.smithi031.stdout:ceph source packages 3.8 kB/s | 1.5 kB 00:00 2023-08-25T03:30:31.024 INFO:teuthology.orchestra.run.smithi031.stdout:Copr repo for python3-asyncssh owned by ceph 16 kB/s | 3.5 kB 00:00 2023-08-25T03:30:31.753 INFO:teuthology.orchestra.run.smithi002.stdout:Extra Packages for Enterprise Linux 14 MB/s | 16 MB 00:01 2023-08-25T03:30:32.479 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 11 MB/s | 16 MB 00:01 2023-08-25T03:30:35.445 INFO:teuthology.orchestra.run.smithi002.stdout:lab-extras 462 kB/s | 24 kB 00:00 2023-08-25T03:30:36.233 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 256 kB/s | 24 kB 00:00 2023-08-25T03:30:36.389 INFO:teuthology.orchestra.run.smithi002.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 61 MB/s | 53 MB 00:00 2023-08-25T03:30:37.320 INFO:teuthology.orchestra.run.smithi031.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 47 MB/s | 47 MB 00:01 2023-08-25T03:30:44.948 INFO:teuthology.orchestra.run.smithi002.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 59 MB/s | 47 MB 00:00 2023-08-25T03:30:45.679 INFO:teuthology.orchestra.run.smithi031.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 58 MB/s | 53 MB 00:00 2023-08-25T03:30:56.678 INFO:teuthology.orchestra.run.smithi002.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-25T03:30:56.679 INFO:teuthology.orchestra.run.smithi002.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-25T03:30:56.695 INFO:teuthology.orchestra.run.smithi002.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-08-25T03:30:56.696 INFO:teuthology.orchestra.run.smithi002.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-08-25T03:30:56.697 INFO:teuthology.orchestra.run.smithi002.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-08-25T03:30:56.698 INFO:teuthology.orchestra.run.smithi002.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-08-25T03:30:56.704 INFO:teuthology.orchestra.run.smithi002.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-08-25T03:30:56.705 INFO:teuthology.orchestra.run.smithi002.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-08-25T03:30:56.705 INFO:teuthology.orchestra.run.smithi002.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-08-25T03:30:56.847 INFO:teuthology.orchestra.run.smithi002.stdout:Dependencies resolved. 2023-08-25T03:30:56.861 INFO:teuthology.orchestra.run.smithi002.stdout:=========================================================================================================================== 2023-08-25T03:30:56.861 INFO:teuthology.orchestra.run.smithi002.stdout: Package Arch Version Repository Size 2023-08-25T03:30:56.861 INFO:teuthology.orchestra.run.smithi002.stdout:=========================================================================================================================== 2023-08-25T03:30:56.862 INFO:teuthology.orchestra.run.smithi002.stdout:Installing: 2023-08-25T03:30:56.862 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-08-25T03:30:56.862 INFO:teuthology.orchestra.run.smithi002.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-08-25T03:30:56.862 INFO:teuthology.orchestra.run.smithi002.stdout: ceph x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 6.4 k 2023-08-25T03:30:56.862 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-base x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 5.4 M 2023-08-25T03:30:56.863 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-fuse x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 862 k 2023-08-25T03:30:56.863 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 167 k 2023-08-25T03:30:56.863 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 1.6 M 2023-08-25T03:30:56.864 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 132 k 2023-08-25T03:30:56.864 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 1.7 M 2023-08-25T03:30:56.864 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 7.4 M 2023-08-25T03:30:56.864 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-rook noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 47 k 2023-08-25T03:30:56.865 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-radosgw x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 14 M 2023-08-25T03:30:56.865 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-test x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 48 M 2023-08-25T03:30:56.865 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-volume noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 261 k 2023-08-25T03:30:56.865 INFO:teuthology.orchestra.run.smithi002.stdout: cephadm noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 171 k 2023-08-25T03:30:56.865 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-mirror x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 260 k 2023-08-25T03:30:56.865 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-top noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 18 k 2023-08-25T03:30:56.865 INFO:teuthology.orchestra.run.smithi002.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-08-25T03:30:56.866 INFO:teuthology.orchestra.run.smithi002.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-08-25T03:30:56.866 INFO:teuthology.orchestra.run.smithi002.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-08-25T03:30:56.866 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs-devel x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 31 k 2023-08-25T03:30:56.866 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs2 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 784 k 2023-08-25T03:30:56.866 INFO:teuthology.orchestra.run.smithi002.stdout: librados-devel x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 139 k 2023-08-25T03:30:56.866 INFO:teuthology.orchestra.run.smithi002.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-08-25T03:30:56.866 INFO:teuthology.orchestra.run.smithi002.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-08-25T03:30:56.867 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-08-25T03:30:56.867 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-08-25T03:30:56.867 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-08-25T03:30:56.867 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-08-25T03:30:56.867 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cephfs x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 197 k 2023-08-25T03:30:56.867 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rados x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 371 k 2023-08-25T03:30:56.867 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rbd x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 364 k 2023-08-25T03:30:56.868 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rgw x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 107 k 2023-08-25T03:30:56.868 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-fuse x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 94 k 2023-08-25T03:30:56.868 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-mirror x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 3.5 M 2023-08-25T03:30:56.868 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-nbd x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 179 k 2023-08-25T03:30:56.868 INFO:teuthology.orchestra.run.smithi002.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-08-25T03:30:56.868 INFO:teuthology.orchestra.run.smithi002.stdout:Upgrading: 2023-08-25T03:30:56.869 INFO:teuthology.orchestra.run.smithi002.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-08-25T03:30:56.870 INFO:teuthology.orchestra.run.smithi002.stdout: librados2 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 3.7 M 2023-08-25T03:30:56.870 INFO:teuthology.orchestra.run.smithi002.stdout: librbd1 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 3.6 M 2023-08-25T03:30:56.870 INFO:teuthology.orchestra.run.smithi002.stdout:Installing dependencies: 2023-08-25T03:30:56.870 INFO:teuthology.orchestra.run.smithi002.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-08-25T03:30:56.870 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-common x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 22 M 2023-08-25T03:30:56.871 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 23 k 2023-08-25T03:30:56.871 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mds x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 2.3 M 2023-08-25T03:30:56.871 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 242 k 2023-08-25T03:30:56.871 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mon x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 4.6 M 2023-08-25T03:30:56.871 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-osd x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 18 M 2023-08-25T03:30:56.871 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 15 k 2023-08-25T03:30:56.872 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-selinux x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 24 k 2023-08-25T03:30:56.872 INFO:teuthology.orchestra.run.smithi002.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-08-25T03:30:56.872 INFO:teuthology.orchestra.run.smithi002.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-08-25T03:30:56.872 INFO:teuthology.orchestra.run.smithi002.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-08-25T03:30:56.872 INFO:teuthology.orchestra.run.smithi002.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-08-25T03:30:56.872 INFO:teuthology.orchestra.run.smithi002.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-08-25T03:30:56.872 INFO:teuthology.orchestra.run.smithi002.stdout: libcephsqlite x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 139 k 2023-08-25T03:30:56.873 INFO:teuthology.orchestra.run.smithi002.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-08-25T03:30:56.873 INFO:teuthology.orchestra.run.smithi002.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-08-25T03:30:56.873 INFO:teuthology.orchestra.run.smithi002.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-08-25T03:30:56.873 INFO:teuthology.orchestra.run.smithi002.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-08-25T03:30:56.873 INFO:teuthology.orchestra.run.smithi002.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-08-25T03:30:56.873 INFO:teuthology.orchestra.run.smithi002.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-08-25T03:30:56.873 INFO:teuthology.orchestra.run.smithi002.stdout: libradosstriper1 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 473 k 2023-08-25T03:30:56.874 INFO:teuthology.orchestra.run.smithi002.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-08-25T03:30:56.874 INFO:teuthology.orchestra.run.smithi002.stdout: librgw2 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 6.8 M 2023-08-25T03:30:56.874 INFO:teuthology.orchestra.run.smithi002.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-08-25T03:30:56.874 INFO:teuthology.orchestra.run.smithi002.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-08-25T03:30:56.874 INFO:teuthology.orchestra.run.smithi002.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-08-25T03:30:56.874 INFO:teuthology.orchestra.run.smithi002.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-08-25T03:30:56.874 INFO:teuthology.orchestra.run.smithi002.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-08-25T03:30:56.875 INFO:teuthology.orchestra.run.smithi002.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-08-25T03:30:56.875 INFO:teuthology.orchestra.run.smithi002.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-08-25T03:30:56.875 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-08-25T03:30:56.875 INFO:teuthology.orchestra.run.smithi002.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-08-25T03:30:56.876 INFO:teuthology.orchestra.run.smithi002.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-08-25T03:30:56.876 INFO:teuthology.orchestra.run.smithi002.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-08-25T03:30:56.876 INFO:teuthology.orchestra.run.smithi002.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-08-25T03:30:56.876 INFO:teuthology.orchestra.run.smithi002.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-08-25T03:30:56.877 INFO:teuthology.orchestra.run.smithi002.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-08-25T03:30:56.877 INFO:teuthology.orchestra.run.smithi002.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-08-25T03:30:56.877 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-08-25T03:30:56.877 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-argparse x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 45 k 2023-08-25T03:30:56.877 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-common x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 126 k 2023-08-25T03:30:56.878 INFO:teuthology.orchestra.run.smithi002.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-08-25T03:30:56.878 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-08-25T03:30:56.878 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-08-25T03:30:56.878 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-08-25T03:30:56.878 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-08-25T03:30:56.879 INFO:teuthology.orchestra.run.smithi002.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-08-25T03:30:56.879 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-08-25T03:30:56.879 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-08-25T03:30:56.880 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-08-25T03:30:56.880 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-08-25T03:30:56.880 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-08-25T03:30:56.880 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-25T03:30:56.881 INFO:teuthology.orchestra.run.smithi002.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-08-25T03:30:56.881 INFO:teuthology.orchestra.run.smithi002.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-08-25T03:30:56.881 INFO:teuthology.orchestra.run.smithi002.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-08-25T03:30:56.881 INFO:teuthology.orchestra.run.smithi002.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-08-25T03:30:56.882 INFO:teuthology.orchestra.run.smithi002.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-08-25T03:30:56.882 INFO:teuthology.orchestra.run.smithi002.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-08-25T03:30:56.882 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-08-25T03:30:56.882 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-08-25T03:30:56.882 INFO:teuthology.orchestra.run.smithi002.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-08-25T03:30:56.883 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-08-25T03:30:56.883 INFO:teuthology.orchestra.run.smithi002.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-08-25T03:30:56.883 INFO:teuthology.orchestra.run.smithi002.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-08-25T03:30:56.883 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-08-25T03:30:56.883 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-08-25T03:30:56.883 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-08-25T03:30:56.884 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-08-25T03:30:56.884 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-08-25T03:30:56.884 INFO:teuthology.orchestra.run.smithi002.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-08-25T03:30:56.884 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-25T03:30:56.884 INFO:teuthology.orchestra.run.smithi002.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-08-25T03:30:56.884 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-08-25T03:30:56.885 INFO:teuthology.orchestra.run.smithi002.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-08-25T03:30:56.885 INFO:teuthology.orchestra.run.smithi002.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-08-25T03:30:56.885 INFO:teuthology.orchestra.run.smithi002.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-08-25T03:30:56.885 INFO:teuthology.orchestra.run.smithi002.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-08-25T03:30:56.885 INFO:teuthology.orchestra.run.smithi002.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-08-25T03:30:56.885 INFO:teuthology.orchestra.run.smithi002.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-08-25T03:30:56.885 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-08-25T03:30:56.886 INFO:teuthology.orchestra.run.smithi002.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-08-25T03:30:56.886 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-08-25T03:30:56.886 INFO:teuthology.orchestra.run.smithi002.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-08-25T03:30:56.886 INFO:teuthology.orchestra.run.smithi002.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-08-25T03:30:56.886 INFO:teuthology.orchestra.run.smithi002.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-08-25T03:30:56.886 INFO:teuthology.orchestra.run.smithi002.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-08-25T03:30:56.886 INFO:teuthology.orchestra.run.smithi002.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-08-25T03:30:56.887 INFO:teuthology.orchestra.run.smithi002.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-08-25T03:30:56.887 INFO:teuthology.orchestra.run.smithi002.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-08-25T03:30:56.887 INFO:teuthology.orchestra.run.smithi002.stdout:Enabling module streams: 2023-08-25T03:30:56.887 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql 10 2023-08-25T03:30:56.887 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:30:56.887 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction Summary 2023-08-25T03:30:56.887 INFO:teuthology.orchestra.run.smithi002.stdout:=========================================================================================================================== 2023-08-25T03:30:56.888 INFO:teuthology.orchestra.run.smithi002.stdout:Install 128 Packages 2023-08-25T03:30:56.888 INFO:teuthology.orchestra.run.smithi002.stdout:Upgrade 3 Packages 2023-08-25T03:30:56.888 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:30:56.889 INFO:teuthology.orchestra.run.smithi002.stdout:Total download size: 206 M 2023-08-25T03:30:56.889 INFO:teuthology.orchestra.run.smithi002.stdout:Downloading Packages: 2023-08-25T03:30:57.222 INFO:teuthology.orchestra.run.smithi002.stdout:(1/131): ceph-18.0.0-5703.gba6b39e4.el8.x86_64. 21 kB/s | 6.4 kB 00:00 2023-08-25T03:30:57.653 INFO:teuthology.orchestra.run.smithi031.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-25T03:30:57.654 INFO:teuthology.orchestra.run.smithi031.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-08-25T03:30:57.669 INFO:teuthology.orchestra.run.smithi031.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-08-25T03:30:57.670 INFO:teuthology.orchestra.run.smithi031.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-08-25T03:30:57.672 INFO:teuthology.orchestra.run.smithi031.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-08-25T03:30:57.673 INFO:teuthology.orchestra.run.smithi031.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-08-25T03:30:57.675 INFO:teuthology.orchestra.run.smithi031.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-08-25T03:30:57.678 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-08-25T03:30:57.680 INFO:teuthology.orchestra.run.smithi031.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-08-25T03:30:57.773 INFO:teuthology.orchestra.run.smithi002.stdout:(2/131): ceph-fuse-18.0.0-5703.gba6b39e4.el8.x8 1.5 MB/s | 862 kB 00:00 2023-08-25T03:30:57.823 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2023-08-25T03:30:57.836 INFO:teuthology.orchestra.run.smithi031.stdout:=========================================================================================================================== 2023-08-25T03:30:57.837 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2023-08-25T03:30:57.837 INFO:teuthology.orchestra.run.smithi031.stdout:=========================================================================================================================== 2023-08-25T03:30:57.837 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2023-08-25T03:30:57.837 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-08-25T03:30:57.838 INFO:teuthology.orchestra.run.smithi031.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-08-25T03:30:57.838 INFO:teuthology.orchestra.run.smithi031.stdout: ceph x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 6.4 k 2023-08-25T03:30:57.838 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 5.4 M 2023-08-25T03:30:57.838 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 862 k 2023-08-25T03:30:57.838 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 167 k 2023-08-25T03:30:57.838 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 1.6 M 2023-08-25T03:30:57.839 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 132 k 2023-08-25T03:30:57.839 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 1.7 M 2023-08-25T03:30:57.839 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 7.4 M 2023-08-25T03:30:57.839 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 47 k 2023-08-25T03:30:57.839 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 14 M 2023-08-25T03:30:57.839 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 48 M 2023-08-25T03:30:57.839 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 261 k 2023-08-25T03:30:57.840 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 171 k 2023-08-25T03:30:57.840 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 260 k 2023-08-25T03:30:57.840 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 18 k 2023-08-25T03:30:57.840 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-08-25T03:30:57.841 INFO:teuthology.orchestra.run.smithi031.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-08-25T03:30:57.841 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-08-25T03:30:57.841 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 31 k 2023-08-25T03:30:57.841 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 784 k 2023-08-25T03:30:57.842 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 139 k 2023-08-25T03:30:57.842 INFO:teuthology.orchestra.run.smithi031.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-08-25T03:30:57.842 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-08-25T03:30:57.843 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-08-25T03:30:57.843 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-08-25T03:30:57.843 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-08-25T03:30:57.843 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-08-25T03:30:57.843 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 197 k 2023-08-25T03:30:57.843 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 371 k 2023-08-25T03:30:57.844 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 364 k 2023-08-25T03:30:57.844 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 107 k 2023-08-25T03:30:57.844 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 94 k 2023-08-25T03:30:57.844 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 3.5 M 2023-08-25T03:30:57.844 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 179 k 2023-08-25T03:30:57.844 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-08-25T03:30:57.845 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrading: 2023-08-25T03:30:57.845 INFO:teuthology.orchestra.run.smithi031.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-08-25T03:30:57.845 INFO:teuthology.orchestra.run.smithi031.stdout: librados2 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 3.7 M 2023-08-25T03:30:57.845 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 3.6 M 2023-08-25T03:30:57.845 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2023-08-25T03:30:57.845 INFO:teuthology.orchestra.run.smithi031.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-08-25T03:30:57.845 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 22 M 2023-08-25T03:30:57.846 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 23 k 2023-08-25T03:30:57.846 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 2.3 M 2023-08-25T03:30:57.846 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 242 k 2023-08-25T03:30:57.846 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 4.6 M 2023-08-25T03:30:57.846 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 18 M 2023-08-25T03:30:57.846 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts noarch 2:18.0.0-5703.gba6b39e4.el8 ceph-noarch 15 k 2023-08-25T03:30:57.846 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 24 k 2023-08-25T03:30:57.847 INFO:teuthology.orchestra.run.smithi031.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-08-25T03:30:57.847 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-08-25T03:30:57.847 INFO:teuthology.orchestra.run.smithi031.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-08-25T03:30:57.847 INFO:teuthology.orchestra.run.smithi031.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-08-25T03:30:57.847 INFO:teuthology.orchestra.run.smithi031.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-08-25T03:30:57.847 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 139 k 2023-08-25T03:30:57.847 INFO:teuthology.orchestra.run.smithi031.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-08-25T03:30:57.848 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-08-25T03:30:57.848 INFO:teuthology.orchestra.run.smithi031.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-08-25T03:30:57.848 INFO:teuthology.orchestra.run.smithi031.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-08-25T03:30:57.848 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-08-25T03:30:57.848 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-08-25T03:30:57.848 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 473 k 2023-08-25T03:30:57.849 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-08-25T03:30:57.849 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2 x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 6.8 M 2023-08-25T03:30:57.849 INFO:teuthology.orchestra.run.smithi031.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-08-25T03:30:57.849 INFO:teuthology.orchestra.run.smithi031.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-08-25T03:30:57.849 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-08-25T03:30:57.849 INFO:teuthology.orchestra.run.smithi031.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-08-25T03:30:57.849 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-08-25T03:30:57.850 INFO:teuthology.orchestra.run.smithi031.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-08-25T03:30:57.850 INFO:teuthology.orchestra.run.smithi031.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-08-25T03:30:57.850 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-08-25T03:30:57.850 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-08-25T03:30:57.850 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-08-25T03:30:57.850 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-08-25T03:30:57.850 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-08-25T03:30:57.851 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-08-25T03:30:57.851 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-08-25T03:30:57.851 INFO:teuthology.orchestra.run.smithi031.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-08-25T03:30:57.851 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-08-25T03:30:57.851 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 45 k 2023-08-25T03:30:57.851 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common x86_64 2:18.0.0-5703.gba6b39e4.el8 ceph 126 k 2023-08-25T03:30:57.851 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-08-25T03:30:57.852 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-08-25T03:30:57.852 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-08-25T03:30:57.852 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-08-25T03:30:57.852 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-08-25T03:30:57.852 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-08-25T03:30:57.852 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-08-25T03:30:57.853 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-08-25T03:30:57.853 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-08-25T03:30:57.853 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-08-25T03:30:57.853 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-08-25T03:30:57.853 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-25T03:30:57.854 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-08-25T03:30:57.854 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-08-25T03:30:57.854 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-08-25T03:30:57.854 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-08-25T03:30:57.854 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-08-25T03:30:57.854 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-08-25T03:30:57.855 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-08-25T03:30:57.855 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-08-25T03:30:57.855 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-08-25T03:30:57.855 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-08-25T03:30:57.855 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-08-25T03:30:57.855 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-08-25T03:30:57.856 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-08-25T03:30:57.856 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-08-25T03:30:57.856 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-08-25T03:30:57.856 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-08-25T03:30:57.856 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-08-25T03:30:57.856 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-08-25T03:30:57.856 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-08-25T03:30:57.857 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-08-25T03:30:57.857 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-08-25T03:30:57.857 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-08-25T03:30:57.857 INFO:teuthology.orchestra.run.smithi031.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-08-25T03:30:57.857 INFO:teuthology.orchestra.run.smithi031.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-08-25T03:30:57.857 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-08-25T03:30:57.857 INFO:teuthology.orchestra.run.smithi031.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-08-25T03:30:57.858 INFO:teuthology.orchestra.run.smithi031.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-08-25T03:30:57.858 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-08-25T03:30:57.858 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-08-25T03:30:57.858 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-08-25T03:30:57.858 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-08-25T03:30:57.858 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-08-25T03:30:57.858 INFO:teuthology.orchestra.run.smithi031.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-08-25T03:30:57.858 INFO:teuthology.orchestra.run.smithi031.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-08-25T03:30:57.859 INFO:teuthology.orchestra.run.smithi031.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-08-25T03:30:57.859 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-08-25T03:30:57.859 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-08-25T03:30:57.859 INFO:teuthology.orchestra.run.smithi031.stdout:Enabling module streams: 2023-08-25T03:30:57.859 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql 10 2023-08-25T03:30:57.859 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:30:57.860 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2023-08-25T03:30:57.860 INFO:teuthology.orchestra.run.smithi031.stdout:=========================================================================================================================== 2023-08-25T03:30:57.861 INFO:teuthology.orchestra.run.smithi031.stdout:Install 128 Packages 2023-08-25T03:30:57.861 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrade 3 Packages 2023-08-25T03:30:57.861 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:30:57.862 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 206 M 2023-08-25T03:30:57.863 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2023-08-25T03:30:57.907 INFO:teuthology.orchestra.run.smithi002.stdout:(3/131): ceph-immutable-object-cache-18.0.0-570 1.2 MB/s | 167 kB 00:00 2023-08-25T03:30:58.204 INFO:teuthology.orchestra.run.smithi031.stdout:(1/131): ceph-18.0.0-5703.gba6b39e4.el8.x86_64. 20 kB/s | 6.4 kB 00:00 2023-08-25T03:30:58.999 INFO:teuthology.orchestra.run.smithi002.stdout:(4/131): ceph-mds-18.0.0-5703.gba6b39e4.el8.x86 2.1 MB/s | 2.3 MB 00:01 2023-08-25T03:30:59.411 INFO:teuthology.orchestra.run.smithi031.stdout:(2/131): ceph-fuse-18.0.0-5703.gba6b39e4.el8.x8 714 kB/s | 862 kB 00:01 2023-08-25T03:30:59.524 INFO:teuthology.orchestra.run.smithi002.stdout:(5/131): ceph-mgr-18.0.0-5703.gba6b39e4.el8.x86 3.0 MB/s | 1.6 MB 00:00 2023-08-25T03:30:59.604 INFO:teuthology.orchestra.run.smithi031.stdout:(3/131): ceph-immutable-object-cache-18.0.0-570 866 kB/s | 167 kB 00:00 2023-08-25T03:31:01.632 INFO:teuthology.orchestra.run.smithi002.stdout:(6/131): ceph-mon-18.0.0-5703.gba6b39e4.el8.x86 2.2 MB/s | 4.6 MB 00:02 2023-08-25T03:31:01.671 INFO:teuthology.orchestra.run.smithi031.stdout:(4/131): ceph-base-18.0.0-5703.gba6b39e4.el8.x8 1.4 MB/s | 5.4 MB 00:03 2023-08-25T03:31:01.846 INFO:teuthology.orchestra.run.smithi031.stdout:(5/131): ceph-mds-18.0.0-5703.gba6b39e4.el8.x86 1.0 MB/s | 2.3 MB 00:02 2023-08-25T03:31:02.130 INFO:teuthology.orchestra.run.smithi031.stdout:(6/131): ceph-mgr-18.0.0-5703.gba6b39e4.el8.x86 3.4 MB/s | 1.6 MB 00:00 2023-08-25T03:31:02.133 INFO:teuthology.orchestra.run.smithi002.stdout:(7/131): ceph-base-18.0.0-5703.gba6b39e4.el8.x8 1.0 MB/s | 5.4 MB 00:05 2023-08-25T03:31:02.825 INFO:teuthology.orchestra.run.smithi002.stdout:(8/131): ceph-common-18.0.0-5703.gba6b39e4.el8. 3.8 MB/s | 22 MB 00:05 2023-08-25T03:31:02.950 INFO:teuthology.orchestra.run.smithi002.stdout:(9/131): ceph-selinux-18.0.0-5703.gba6b39e4.el8 195 kB/s | 24 kB 00:00 2023-08-25T03:31:05.921 INFO:teuthology.orchestra.run.smithi031.stdout:(7/131): ceph-osd-18.0.0-5703.gba6b39e4.el8.x86 4.7 MB/s | 18 MB 00:03 2023-08-25T03:31:06.213 INFO:teuthology.orchestra.run.smithi031.stdout:(8/131): ceph-mon-18.0.0-5703.gba6b39e4.el8.x86 1.1 MB/s | 4.6 MB 00:04 2023-08-25T03:31:06.335 INFO:teuthology.orchestra.run.smithi031.stdout:(9/131): ceph-selinux-18.0.0-5703.gba6b39e4.el8 199 kB/s | 24 kB 00:00 2023-08-25T03:31:08.997 INFO:teuthology.orchestra.run.smithi031.stdout:(10/131): ceph-radosgw-18.0.0-5703.gba6b39e4.el 4.4 MB/s | 14 MB 00:03 2023-08-25T03:31:09.114 INFO:teuthology.orchestra.run.smithi031.stdout:(11/131): cephfs-mirror-18.0.0-5703.gba6b39e4.e 2.2 MB/s | 260 kB 00:00 2023-08-25T03:31:09.214 INFO:teuthology.orchestra.run.smithi031.stdout:(12/131): libcephfs-devel-18.0.0-5703.gba6b39e4 310 kB/s | 31 kB 00:00 2023-08-25T03:31:09.506 INFO:teuthology.orchestra.run.smithi031.stdout:(13/131): libcephfs2-18.0.0-5703.gba6b39e4.el8. 2.6 MB/s | 784 kB 00:00 2023-08-25T03:31:09.606 INFO:teuthology.orchestra.run.smithi031.stdout:(14/131): libcephsqlite-18.0.0-5703.gba6b39e4.e 1.4 MB/s | 139 kB 00:00 2023-08-25T03:31:09.845 INFO:teuthology.orchestra.run.smithi031.stdout:(15/131): ceph-common-18.0.0-5703.gba6b39e4.el8 1.9 MB/s | 22 MB 00:11 2023-08-25T03:31:09.873 INFO:teuthology.orchestra.run.smithi031.stdout:(16/131): librados-devel-18.0.0-5703.gba6b39e4. 521 kB/s | 139 kB 00:00 2023-08-25T03:31:10.157 INFO:teuthology.orchestra.run.smithi031.stdout:(17/131): libradosstriper1-18.0.0-5703.gba6b39e 1.5 MB/s | 473 kB 00:00 2023-08-25T03:31:10.232 INFO:teuthology.orchestra.run.smithi031.stdout:(18/131): python3-ceph-argparse-18.0.0-5703.gba 596 kB/s | 45 kB 00:00 2023-08-25T03:31:10.332 INFO:teuthology.orchestra.run.smithi031.stdout:(19/131): python3-ceph-common-18.0.0-5703.gba6b 1.2 MB/s | 126 kB 00:00 2023-08-25T03:31:10.491 INFO:teuthology.orchestra.run.smithi031.stdout:(20/131): python3-cephfs-18.0.0-5703.gba6b39e4. 1.2 MB/s | 197 kB 00:00 2023-08-25T03:31:10.733 INFO:teuthology.orchestra.run.smithi002.stdout:(10/131): ceph-radosgw-18.0.0-5703.gba6b39e4.el 1.6 MB/s | 14 MB 00:08 2023-08-25T03:31:10.738 INFO:teuthology.orchestra.run.smithi031.stdout:(21/131): python3-rados-18.0.0-5703.gba6b39e4.e 1.5 MB/s | 371 kB 00:00 2023-08-25T03:31:10.833 INFO:teuthology.orchestra.run.smithi002.stdout:(11/131): cephfs-mirror-18.0.0-5703.gba6b39e4.e 2.6 MB/s | 260 kB 00:00 2023-08-25T03:31:10.933 INFO:teuthology.orchestra.run.smithi002.stdout:(12/131): libcephfs-devel-18.0.0-5703.gba6b39e4 311 kB/s | 31 kB 00:00 2023-08-25T03:31:10.958 INFO:teuthology.orchestra.run.smithi031.stdout:(22/131): python3-rbd-18.0.0-5703.gba6b39e4.el8 1.6 MB/s | 364 kB 00:00 2023-08-25T03:31:11.067 INFO:teuthology.orchestra.run.smithi031.stdout:(23/131): python3-rgw-18.0.0-5703.gba6b39e4.el8 986 kB/s | 107 kB 00:00 2023-08-25T03:31:11.151 INFO:teuthology.orchestra.run.smithi031.stdout:(24/131): rbd-fuse-18.0.0-5703.gba6b39e4.el8.x8 1.1 MB/s | 94 kB 00:00 2023-08-25T03:31:11.296 INFO:teuthology.orchestra.run.smithi002.stdout:(13/131): libcephfs2-18.0.0-5703.gba6b39e4.el8. 2.7 MB/s | 784 kB 00:00 2023-08-25T03:31:11.317 INFO:teuthology.orchestra.run.smithi002.stdout:(14/131): libcephsqlite-18.0.0-5703.gba6b39e4.e 1.4 MB/s | 139 kB 00:00 2023-08-25T03:31:11.426 INFO:teuthology.orchestra.run.smithi002.stdout:(15/131): librados-devel-18.0.0-5703.gba6b39e4. 1.3 MB/s | 139 kB 00:00 2023-08-25T03:31:11.618 INFO:teuthology.orchestra.run.smithi002.stdout:(16/131): libradosstriper1-18.0.0-5703.gba6b39e 2.4 MB/s | 473 kB 00:00 2023-08-25T03:31:11.642 INFO:teuthology.orchestra.run.smithi031.stdout:(25/131): librgw2-18.0.0-5703.gba6b39e4.el8.x86 3.8 MB/s | 6.8 MB 00:01 2023-08-25T03:31:11.768 INFO:teuthology.orchestra.run.smithi031.stdout:(26/131): rbd-nbd-18.0.0-5703.gba6b39e4.el8.x86 1.4 MB/s | 179 kB 00:00 2023-08-25T03:31:11.876 INFO:teuthology.orchestra.run.smithi031.stdout:(27/131): ceph-grafana-dashboards-18.0.0-5703.g 213 kB/s | 23 kB 00:00 2023-08-25T03:31:11.993 INFO:teuthology.orchestra.run.smithi031.stdout:(28/131): ceph-mgr-cephadm-18.0.0-5703.gba6b39e 1.1 MB/s | 132 kB 00:00 2023-08-25T03:31:12.460 INFO:teuthology.orchestra.run.smithi031.stdout:(29/131): ceph-mgr-dashboard-18.0.0-5703.gba6b3 3.8 MB/s | 1.7 MB 00:00 2023-08-25T03:31:12.860 INFO:teuthology.orchestra.run.smithi031.stdout:(30/131): rbd-mirror-18.0.0-5703.gba6b39e4.el8. 2.1 MB/s | 3.5 MB 00:01 2023-08-25T03:31:13.027 INFO:teuthology.orchestra.run.smithi031.stdout:(31/131): ceph-mgr-modules-core-18.0.0-5703.gba 1.4 MB/s | 242 kB 00:00 2023-08-25T03:31:13.111 INFO:teuthology.orchestra.run.smithi031.stdout:(32/131): ceph-mgr-rook-18.0.0-5703.gba6b39e4.e 569 kB/s | 47 kB 00:00 2023-08-25T03:31:13.186 INFO:teuthology.orchestra.run.smithi031.stdout:(33/131): ceph-prometheus-alerts-18.0.0-5703.gb 194 kB/s | 15 kB 00:00 2023-08-25T03:31:13.201 INFO:teuthology.orchestra.run.smithi002.stdout:(17/131): ceph-osd-18.0.0-5703.gba6b39e4.el8.x8 1.5 MB/s | 18 MB 00:11 2023-08-25T03:31:13.310 INFO:teuthology.orchestra.run.smithi002.stdout:(18/131): python3-ceph-argparse-18.0.0-5703.gba 414 kB/s | 45 kB 00:00 2023-08-25T03:31:13.367 INFO:teuthology.orchestra.run.smithi031.stdout:(34/131): ceph-volume-18.0.0-5703.gba6b39e4.el8 1.4 MB/s | 261 kB 00:00 2023-08-25T03:31:13.510 INFO:teuthology.orchestra.run.smithi002.stdout:(19/131): python3-ceph-common-18.0.0-5703.gba6b 628 kB/s | 126 kB 00:00 2023-08-25T03:31:13.529 INFO:teuthology.orchestra.run.smithi031.stdout:(35/131): cephadm-18.0.0-5703.gba6b39e4.el8.noa 1.0 MB/s | 171 kB 00:00 2023-08-25T03:31:13.653 INFO:teuthology.orchestra.run.smithi031.stdout:(36/131): cephfs-top-18.0.0-5703.gba6b39e4.el8. 144 kB/s | 18 kB 00:00 2023-08-25T03:31:13.702 INFO:teuthology.orchestra.run.smithi002.stdout:(20/131): python3-cephfs-18.0.0-5703.gba6b39e4. 1.0 MB/s | 197 kB 00:00 2023-08-25T03:31:13.746 INFO:teuthology.orchestra.run.smithi031.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 3.2 MB/s | 306 kB 00:00 2023-08-25T03:31:13.788 INFO:teuthology.orchestra.run.smithi031.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 1.4 MB/s | 59 kB 00:00 2023-08-25T03:31:13.822 INFO:teuthology.orchestra.run.smithi031.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 2.2 MB/s | 75 kB 00:00 2023-08-25T03:31:13.872 INFO:teuthology.orchestra.run.smithi031.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 9.3 MB/s | 476 kB 00:00 2023-08-25T03:31:13.886 INFO:teuthology.orchestra.run.smithi002.stdout:(21/131): librgw2-18.0.0-5703.gba6b39e4.el8.x86 3.0 MB/s | 6.8 MB 00:02 2023-08-25T03:31:13.905 INFO:teuthology.orchestra.run.smithi031.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 1.3 MB/s | 44 kB 00:00 2023-08-25T03:31:13.964 INFO:teuthology.orchestra.run.smithi031.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 3.1 MB/s | 185 kB 00:00 2023-08-25T03:31:13.986 INFO:teuthology.orchestra.run.smithi002.stdout:(22/131): python3-rados-18.0.0-5703.gba6b39e4.e 1.3 MB/s | 371 kB 00:00 2023-08-25T03:31:13.998 INFO:teuthology.orchestra.run.smithi031.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 998 kB/s | 33 kB 00:00 2023-08-25T03:31:14.053 INFO:teuthology.orchestra.run.smithi002.stdout:(23/131): python3-rbd-18.0.0-5703.gba6b39e4.el8 2.1 MB/s | 364 kB 00:00 2023-08-25T03:31:14.078 INFO:teuthology.orchestra.run.smithi002.stdout:(24/131): python3-rgw-18.0.0-5703.gba6b39e4.el8 1.1 MB/s | 107 kB 00:00 2023-08-25T03:31:14.106 INFO:teuthology.orchestra.run.smithi031.stdout:(44/131): ceph-mgr-diskprediction-local-18.0.0- 4.5 MB/s | 7.4 MB 00:01 2023-08-25T03:31:14.123 INFO:teuthology.orchestra.run.smithi031.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 130 kB/s | 16 kB 00:00 2023-08-25T03:31:14.128 INFO:teuthology.orchestra.run.smithi002.stdout:(25/131): rbd-fuse-18.0.0-5703.gba6b39e4.el8.x8 1.2 MB/s | 94 kB 00:00 2023-08-25T03:31:14.165 INFO:teuthology.orchestra.run.smithi031.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 9.0 MB/s | 384 kB 00:00 2023-08-25T03:31:14.190 INFO:teuthology.orchestra.run.smithi031.stdout:(47/131): python3-cheroot-8.5.2-1.el8.noarch.rp 2.0 MB/s | 173 kB 00:00 2023-08-25T03:31:14.229 INFO:teuthology.orchestra.run.smithi002.stdout:(26/131): rbd-nbd-18.0.0-5703.gba6b39e4.el8.x86 1.7 MB/s | 179 kB 00:00 2023-08-25T03:31:14.242 INFO:teuthology.orchestra.run.smithi031.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 1.0 MB/s | 81 kB 00:00 2023-08-25T03:31:14.266 INFO:teuthology.orchestra.run.smithi031.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 148 kB/s | 11 kB 00:00 2023-08-25T03:31:14.283 INFO:teuthology.orchestra.run.smithi031.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 434 kB/s | 18 kB 00:00 2023-08-25T03:31:14.308 INFO:teuthology.orchestra.run.smithi031.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 1.9 MB/s | 49 kB 00:00 2023-08-25T03:31:14.320 INFO:teuthology.orchestra.run.smithi002.stdout:(27/131): ceph-grafana-dashboards-18.0.0-5703.g 252 kB/s | 23 kB 00:00 2023-08-25T03:31:14.342 INFO:teuthology.orchestra.run.smithi031.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 15 MB/s | 1.2 MB 00:00 2023-08-25T03:31:14.359 INFO:teuthology.orchestra.run.smithi031.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 1.1 MB/s | 59 kB 00:00 2023-08-25T03:31:14.376 INFO:teuthology.orchestra.run.smithi031.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 1.8 MB/s | 60 kB 00:00 2023-08-25T03:31:14.409 INFO:teuthology.orchestra.run.smithi031.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 5.5 MB/s | 283 kB 00:00 2023-08-25T03:31:14.426 INFO:teuthology.orchestra.run.smithi031.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 311 kB/s | 16 kB 00:00 2023-08-25T03:31:14.443 INFO:teuthology.orchestra.run.smithi031.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 1.0 MB/s | 34 kB 00:00 2023-08-25T03:31:14.469 INFO:teuthology.orchestra.run.smithi031.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 4.6 MB/s | 196 kB 00:00 2023-08-25T03:31:14.471 INFO:teuthology.orchestra.run.smithi002.stdout:(28/131): ceph-mgr-cephadm-18.0.0-5703.gba6b39e 880 kB/s | 132 kB 00:00 2023-08-25T03:31:14.494 INFO:teuthology.orchestra.run.smithi031.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 1.3 MB/s | 64 kB 00:00 2023-08-25T03:31:14.553 INFO:teuthology.orchestra.run.smithi031.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 222 kB/s | 19 kB 00:00 2023-08-25T03:31:14.586 INFO:teuthology.orchestra.run.smithi031.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 261 kB/s | 24 kB 00:00 2023-08-25T03:31:14.611 INFO:teuthology.orchestra.run.smithi031.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 507 kB/s | 29 kB 00:00 2023-08-25T03:31:14.637 INFO:teuthology.orchestra.run.smithi031.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 546 kB/s | 27 kB 00:00 2023-08-25T03:31:14.662 INFO:teuthology.orchestra.run.smithi031.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 5.0 MB/s | 255 kB 00:00 2023-08-25T03:31:14.687 INFO:teuthology.orchestra.run.smithi031.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 4.9 MB/s | 251 kB 00:00 2023-08-25T03:31:14.712 INFO:teuthology.orchestra.run.smithi031.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-08-25T03:31:14.738 INFO:teuthology.orchestra.run.smithi031.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-08-25T03:31:14.755 INFO:teuthology.orchestra.run.smithi031.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 551 kB/s | 23 kB 00:00 2023-08-25T03:31:14.780 INFO:teuthology.orchestra.run.smithi031.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-08-25T03:31:14.856 INFO:teuthology.orchestra.run.smithi031.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 15 MB/s | 1.7 MB 00:00 2023-08-25T03:31:14.922 INFO:teuthology.orchestra.run.smithi031.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.9 MB/s | 110 kB 00:00 2023-08-25T03:31:14.956 INFO:teuthology.orchestra.run.smithi031.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-08-25T03:31:14.998 INFO:teuthology.orchestra.run.smithi031.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-08-25T03:31:15.040 INFO:teuthology.orchestra.run.smithi031.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-08-25T03:31:15.046 INFO:teuthology.orchestra.run.smithi002.stdout:(29/131): ceph-mgr-dashboard-18.0.0-5703.gba6b3 3.0 MB/s | 1.7 MB 00:00 2023-08-25T03:31:15.073 INFO:teuthology.orchestra.run.smithi031.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 1.1 MB/s | 39 kB 00:00 2023-08-25T03:31:15.150 INFO:teuthology.orchestra.run.smithi031.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 8.8 MB/s | 688 kB 00:00 2023-08-25T03:31:15.208 INFO:teuthology.orchestra.run.smithi031.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 9.3 MB/s | 538 kB 00:00 2023-08-25T03:31:15.241 INFO:teuthology.orchestra.run.smithi031.stdout:(78/131): python3-mako-1.0.6-13.el8.noarch.rpm 4.6 MB/s | 157 kB 00:00 2023-08-25T03:31:15.399 INFO:teuthology.orchestra.run.smithi031.stdout:(79/131): python3-scipy-1.0.0-21.module+el8.5.0 22 MB/s | 14 MB 00:00 2023-08-25T03:31:15.453 INFO:teuthology.orchestra.run.smithi031.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 883 kB/s | 187 kB 00:00 2023-08-25T03:31:15.492 INFO:teuthology.orchestra.run.smithi031.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 7.6 MB/s | 713 kB 00:00 2023-08-25T03:31:15.533 INFO:teuthology.orchestra.run.smithi031.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 638 kB/s | 27 kB 00:00 2023-08-25T03:31:15.576 INFO:teuthology.orchestra.run.smithi031.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-08-25T03:31:15.659 INFO:teuthology.orchestra.run.smithi031.stdout:(84/131): python3-babel-2.5.1-7.el8.noarch.rpm 23 MB/s | 4.8 MB 00:00 2023-08-25T03:31:15.684 INFO:teuthology.orchestra.run.smithi031.stdout:(85/131): python3-prettytable-0.7.2-14.el8.noar 405 kB/s | 44 kB 00:00 2023-08-25T03:31:15.760 INFO:teuthology.orchestra.run.smithi031.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 6.0 MB/s | 457 kB 00:00 2023-08-25T03:31:15.836 INFO:teuthology.orchestra.run.smithi031.stdout:(87/131): python3-numpy-1.14.3-10.el8.x86_64.rp 21 MB/s | 3.7 MB 00:00 2023-08-25T03:31:15.935 INFO:teuthology.orchestra.run.smithi031.stdout:(88/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.3 MB/s | 225 kB 00:00 2023-08-25T03:31:16.054 INFO:teuthology.orchestra.run.smithi002.stdout:(30/131): rbd-mirror-18.0.0-5703.gba6b39e4.el8. 1.8 MB/s | 3.5 MB 00:01 2023-08-25T03:31:16.313 INFO:teuthology.orchestra.run.smithi031.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 135 kB/s | 64 kB 00:00 2023-08-25T03:31:16.336 INFO:teuthology.orchestra.run.smithi031.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 315 kB/s | 126 kB 00:00 2023-08-25T03:31:16.411 INFO:teuthology.orchestra.run.smithi031.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 200 kB/s | 20 kB 00:00 2023-08-25T03:31:16.436 INFO:teuthology.orchestra.run.smithi031.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.0 MB/s | 103 kB 00:00 2023-08-25T03:31:16.478 INFO:teuthology.orchestra.run.smithi031.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-08-25T03:31:16.520 INFO:teuthology.orchestra.run.smithi031.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.7 MB/s | 198 kB 00:00 2023-08-25T03:31:16.658 INFO:teuthology.orchestra.run.smithi031.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 22 MB/s | 4.9 MB 00:00 2023-08-25T03:31:16.704 INFO:teuthology.orchestra.run.smithi031.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.3 MB/s | 202 kB 00:00 2023-08-25T03:31:16.788 INFO:teuthology.orchestra.run.smithi031.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 18 MB/s | 4.7 MB 00:00 2023-08-25T03:31:16.822 INFO:teuthology.orchestra.run.smithi031.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.0 MB/s | 354 kB 00:00 2023-08-25T03:31:16.881 INFO:teuthology.orchestra.run.smithi031.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 14 MB/s | 808 kB 00:00 2023-08-25T03:31:17.014 INFO:teuthology.orchestra.run.smithi031.stdout:(100/131): postgresql-server-10.21-2.module+el8 22 MB/s | 5.1 MB 00:00 2023-08-25T03:31:17.047 INFO:teuthology.orchestra.run.smithi031.stdout:(101/131): postgresql-10.21-2.module+el8.6.0+15 9.0 MB/s | 1.5 MB 00:00 2023-08-25T03:31:17.073 INFO:teuthology.orchestra.run.smithi031.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 800 kB/s | 47 kB 00:00 2023-08-25T03:31:17.098 INFO:teuthology.orchestra.run.smithi031.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2023-08-25T03:31:17.115 INFO:teuthology.orchestra.run.smithi031.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-08-25T03:31:17.140 INFO:teuthology.orchestra.run.smithi031.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2023-08-25T03:31:17.157 INFO:teuthology.orchestra.run.smithi031.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 1.1 MB/s | 48 kB 00:00 2023-08-25T03:31:17.182 INFO:teuthology.orchestra.run.smithi031.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2023-08-25T03:31:17.199 INFO:teuthology.orchestra.run.smithi031.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 1.0 MB/s | 43 kB 00:00 2023-08-25T03:31:17.248 INFO:teuthology.orchestra.run.smithi031.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 1.6 MB/s | 109 kB 00:00 2023-08-25T03:31:17.283 INFO:teuthology.orchestra.run.smithi031.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 2.7 MB/s | 229 kB 00:00 2023-08-25T03:31:17.316 INFO:teuthology.orchestra.run.smithi031.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 8.1 MB/s | 560 kB 00:00 2023-08-25T03:31:17.334 INFO:teuthology.orchestra.run.smithi031.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 372 kB/s | 19 kB 00:00 2023-08-25T03:31:17.359 INFO:teuthology.orchestra.run.smithi031.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.6 MB/s | 155 kB 00:00 2023-08-25T03:31:17.392 INFO:teuthology.orchestra.run.smithi031.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 9.4 MB/s | 559 kB 00:00 2023-08-25T03:31:17.418 INFO:teuthology.orchestra.run.smithi031.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 306 kB/s | 18 kB 00:00 2023-08-25T03:31:17.451 INFO:teuthology.orchestra.run.smithi031.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-08-25T03:31:17.476 INFO:teuthology.orchestra.run.smithi031.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-08-25T03:31:17.506 INFO:teuthology.orchestra.run.smithi002.stdout:(31/131): ceph-mgr-modules-core-18.0.0-5703.gba 167 kB/s | 242 kB 00:01 2023-08-25T03:31:17.543 INFO:teuthology.orchestra.run.smithi031.stdout:(118/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.1 MB/s | 72 kB 00:00 2023-08-25T03:31:17.602 INFO:teuthology.orchestra.run.smithi031.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 15 MB/s | 2.3 MB 00:00 2023-08-25T03:31:17.627 INFO:teuthology.orchestra.run.smithi031.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 992 kB/s | 83 kB 00:00 2023-08-25T03:31:17.653 INFO:teuthology.orchestra.run.smithi031.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-08-25T03:31:17.678 INFO:teuthology.orchestra.run.smithi031.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-08-25T03:31:17.703 INFO:teuthology.orchestra.run.smithi031.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-08-25T03:31:17.737 INFO:teuthology.orchestra.run.smithi031.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 657 kB/s | 39 kB 00:00 2023-08-25T03:31:17.779 INFO:teuthology.orchestra.run.smithi031.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 8.4 MB/s | 644 kB 00:00 2023-08-25T03:31:17.804 INFO:teuthology.orchestra.run.smithi031.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2023-08-25T03:31:17.854 INFO:teuthology.orchestra.run.smithi031.stdout:(127/131): NetworkManager-initscripts-updown-1. 1.8 MB/s | 138 kB 00:00 2023-08-25T03:31:17.922 INFO:teuthology.orchestra.run.smithi002.stdout:(32/131): ceph-mgr-rook-18.0.0-5703.gba6b39e4.e 114 kB/s | 47 kB 00:00 2023-08-25T03:31:18.030 INFO:teuthology.orchestra.run.smithi002.stdout:(33/131): ceph-prometheus-alerts-18.0.0-5703.gb 134 kB/s | 15 kB 00:00 2023-08-25T03:31:18.297 INFO:teuthology.orchestra.run.smithi002.stdout:(34/131): ceph-mgr-diskprediction-local-18.0.0- 2.3 MB/s | 7.4 MB 00:03 2023-08-25T03:31:18.323 INFO:teuthology.orchestra.run.smithi002.stdout:(35/131): ceph-volume-18.0.0-5703.gba6b39e4.el8 892 kB/s | 261 kB 00:00 2023-08-25T03:31:18.415 INFO:teuthology.orchestra.run.smithi002.stdout:(36/131): cephadm-18.0.0-5703.gba6b39e4.el8.noa 1.4 MB/s | 171 kB 00:00 2023-08-25T03:31:18.432 INFO:teuthology.orchestra.run.smithi002.stdout:(37/131): cephfs-top-18.0.0-5703.gba6b39e4.el8. 165 kB/s | 18 kB 00:00 2023-08-25T03:31:18.698 INFO:teuthology.orchestra.run.smithi002.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 221 kB/s | 59 kB 00:00 2023-08-25T03:31:18.807 INFO:teuthology.orchestra.run.smithi002.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 693 kB/s | 75 kB 00:00 2023-08-25T03:31:18.832 INFO:teuthology.orchestra.run.smithi002.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 734 kB/s | 306 kB 00:00 2023-08-25T03:31:18.900 INFO:teuthology.orchestra.run.smithi002.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 654 kB/s | 44 kB 00:00 2023-08-25T03:31:18.991 INFO:teuthology.orchestra.run.smithi002.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 2.0 MB/s | 185 kB 00:00 2023-08-25T03:31:19.016 INFO:teuthology.orchestra.run.smithi002.stdout:(43/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.2 MB/s | 476 kB 00:00 2023-08-25T03:31:19.059 INFO:teuthology.orchestra.run.smithi002.stdout:(44/131): python3-cachetools-3.1.1-4.el8.noarch 493 kB/s | 33 kB 00:00 2023-08-25T03:31:19.084 INFO:teuthology.orchestra.run.smithi002.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 243 kB/s | 16 kB 00:00 2023-08-25T03:31:19.151 INFO:teuthology.orchestra.run.smithi002.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.8 MB/s | 173 kB 00:00 2023-08-25T03:31:19.184 INFO:teuthology.orchestra.run.smithi002.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.7 MB/s | 384 kB 00:00 2023-08-25T03:31:19.235 INFO:teuthology.orchestra.run.smithi002.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 975 kB/s | 81 kB 00:00 2023-08-25T03:31:19.260 INFO:teuthology.orchestra.run.smithi002.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 148 kB/s | 11 kB 00:00 2023-08-25T03:31:19.310 INFO:teuthology.orchestra.run.smithi002.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 233 kB/s | 18 kB 00:00 2023-08-25T03:31:19.394 INFO:teuthology.orchestra.run.smithi002.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 585 kB/s | 49 kB 00:00 2023-08-25T03:31:19.427 INFO:teuthology.orchestra.run.smithi002.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 7.0 MB/s | 1.2 MB 00:00 2023-08-25T03:31:19.469 INFO:teuthology.orchestra.run.smithi002.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 782 kB/s | 59 kB 00:00 2023-08-25T03:31:19.503 INFO:teuthology.orchestra.run.smithi002.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 800 kB/s | 60 kB 00:00 2023-08-25T03:31:19.570 INFO:teuthology.orchestra.run.smithi002.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.8 MB/s | 283 kB 00:00 2023-08-25T03:31:19.595 INFO:teuthology.orchestra.run.smithi002.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 170 kB/s | 16 kB 00:00 2023-08-25T03:31:19.646 INFO:teuthology.orchestra.run.smithi002.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 449 kB/s | 34 kB 00:00 2023-08-25T03:31:19.662 INFO:teuthology.orchestra.run.smithi002.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 2.9 MB/s | 196 kB 00:00 2023-08-25T03:31:19.721 INFO:teuthology.orchestra.run.smithi002.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 854 kB/s | 64 kB 00:00 2023-08-25T03:31:19.747 INFO:teuthology.orchestra.run.smithi002.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 223 kB/s | 19 kB 00:00 2023-08-25T03:31:19.754 INFO:teuthology.orchestra.run.smithi031.stdout:(128/131): librbd1-18.0.0-5703.gba6b39e4.el8.x8 1.9 MB/s | 3.6 MB 00:01 2023-08-25T03:31:19.797 INFO:teuthology.orchestra.run.smithi002.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 318 kB/s | 24 kB 00:00 2023-08-25T03:31:19.813 INFO:teuthology.orchestra.run.smithi031.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-08-25T03:31:19.822 INFO:teuthology.orchestra.run.smithi002.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 388 kB/s | 29 kB 00:00 2023-08-25T03:31:19.872 INFO:teuthology.orchestra.run.smithi002.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 366 kB/s | 27 kB 00:00 2023-08-25T03:31:19.906 INFO:teuthology.orchestra.run.smithi002.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 3.0 MB/s | 255 kB 00:00 2023-08-25T03:31:19.973 INFO:teuthology.orchestra.run.smithi002.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.4 MB/s | 251 kB 00:00 2023-08-25T03:31:19.998 INFO:teuthology.orchestra.run.smithi002.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 665 kB/s | 61 kB 00:00 2023-08-25T03:31:20.057 INFO:teuthology.orchestra.run.smithi002.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 1.0 MB/s | 85 kB 00:00 2023-08-25T03:31:20.082 INFO:teuthology.orchestra.run.smithi002.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 275 kB/s | 23 kB 00:00 2023-08-25T03:31:20.099 INFO:teuthology.orchestra.run.smithi002.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 4.1 MB/s | 70 kB 00:00 2023-08-25T03:31:20.158 INFO:teuthology.orchestra.run.smithi002.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 804 kB/s | 47 kB 00:00 2023-08-25T03:31:20.200 INFO:teuthology.orchestra.run.smithi002.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 4.7 MB/s | 200 kB 00:00 2023-08-25T03:31:20.233 INFO:teuthology.orchestra.run.smithi002.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.3 MB/s | 43 kB 00:00 2023-08-25T03:31:20.275 INFO:teuthology.orchestra.run.smithi002.stdout:(73/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2023-08-25T03:31:20.326 INFO:teuthology.orchestra.run.smithi002.stdout:(74/131): thrift-0.13.0-2.el8.x86_64.rpm 6.5 MB/s | 1.7 MB 00:00 2023-08-25T03:31:20.351 INFO:teuthology.orchestra.run.smithi002.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 639 kB/s | 48 kB 00:00 2023-08-25T03:31:20.401 INFO:teuthology.orchestra.run.smithi002.stdout:(76/131): python3-requests-oauthlib-1.0.0-1.el8 1.0 MB/s | 43 kB 00:00 2023-08-25T03:31:20.427 INFO:teuthology.orchestra.run.smithi002.stdout:(77/131): libsepol-devel-2.9-3.el8.x86_64.rpm 867 kB/s | 87 kB 00:00 2023-08-25T03:31:20.452 INFO:teuthology.orchestra.run.smithi002.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-08-25T03:31:20.477 INFO:teuthology.orchestra.run.smithi002.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-08-25T03:31:20.511 INFO:teuthology.orchestra.run.smithi002.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-08-25T03:31:20.528 INFO:teuthology.orchestra.run.smithi002.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 373 kB/s | 19 kB 00:00 2023-08-25T03:31:20.553 INFO:teuthology.orchestra.run.smithi002.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.6 MB/s | 155 kB 00:00 2023-08-25T03:31:20.586 INFO:teuthology.orchestra.run.smithi002.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 9.3 MB/s | 559 kB 00:00 2023-08-25T03:31:20.603 INFO:teuthology.orchestra.run.smithi002.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 357 kB/s | 18 kB 00:00 2023-08-25T03:31:20.637 INFO:teuthology.orchestra.run.smithi002.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 12 MB/s | 605 kB 00:00 2023-08-25T03:31:20.662 INFO:teuthology.orchestra.run.smithi002.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-08-25T03:31:20.696 INFO:teuthology.orchestra.run.smithi002.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 2.1 MB/s | 72 kB 00:00 2023-08-25T03:31:20.738 INFO:teuthology.orchestra.run.smithi002.stdout:(88/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-08-25T03:31:20.780 INFO:teuthology.orchestra.run.smithi002.stdout:(89/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 16 MB/s | 2.3 MB 00:00 2023-08-25T03:31:20.805 INFO:teuthology.orchestra.run.smithi002.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 895 kB/s | 60 kB 00:00 2023-08-25T03:31:20.830 INFO:teuthology.orchestra.run.smithi002.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-08-25T03:31:20.856 INFO:teuthology.orchestra.run.smithi002.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-08-25T03:31:20.881 INFO:teuthology.orchestra.run.smithi002.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 763 kB/s | 39 kB 00:00 2023-08-25T03:31:20.915 INFO:teuthology.orchestra.run.smithi002.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 11 MB/s | 644 kB 00:00 2023-08-25T03:31:20.940 INFO:teuthology.orchestra.run.smithi002.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.9 MB/s | 171 kB 00:00 2023-08-25T03:31:20.965 INFO:teuthology.orchestra.run.smithi002.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-08-25T03:31:21.007 INFO:teuthology.orchestra.run.smithi002.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-08-25T03:31:21.049 INFO:teuthology.orchestra.run.smithi002.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-08-25T03:31:21.091 INFO:teuthology.orchestra.run.smithi002.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-08-25T03:31:21.133 INFO:teuthology.orchestra.run.smithi002.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 7.6 MB/s | 323 kB 00:00 2023-08-25T03:31:21.167 INFO:teuthology.orchestra.run.smithi002.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 1.1 MB/s | 39 kB 00:00 2023-08-25T03:31:21.217 INFO:teuthology.orchestra.run.smithi002.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-08-25T03:31:21.268 INFO:teuthology.orchestra.run.smithi002.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 11 MB/s | 538 kB 00:00 2023-08-25T03:31:21.309 INFO:teuthology.orchestra.run.smithi002.stdout:(104/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-08-25T03:31:21.343 INFO:teuthology.orchestra.run.smithi002.stdout:(105/131): oniguruma-6.8.2-2.el8.x86_64.rpm 5.5 MB/s | 187 kB 00:00 2023-08-25T03:31:21.485 INFO:teuthology.orchestra.run.smithi002.stdout:(106/131): python3-scipy-1.0.0-21.module+el8.5. 25 MB/s | 14 MB 00:00 2023-08-25T03:31:21.518 INFO:teuthology.orchestra.run.smithi002.stdout:(107/131): automake-1.16.1-7.el8.noarch.rpm 4.0 MB/s | 713 kB 00:00 2023-08-25T03:31:21.552 INFO:teuthology.orchestra.run.smithi002.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 794 kB/s | 27 kB 00:00 2023-08-25T03:31:21.602 INFO:teuthology.orchestra.run.smithi002.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 6.3 MB/s | 320 kB 00:00 2023-08-25T03:31:21.686 INFO:teuthology.orchestra.run.smithi002.stdout:(110/131): python3-babel-2.5.1-7.el8.noarch.rpm 24 MB/s | 4.8 MB 00:00 2023-08-25T03:31:21.711 INFO:teuthology.orchestra.run.smithi002.stdout:(111/131): python3-prettytable-0.7.2-14.el8.noa 406 kB/s | 44 kB 00:00 2023-08-25T03:31:21.761 INFO:teuthology.orchestra.run.smithi002.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 8.9 MB/s | 457 kB 00:00 2023-08-25T03:31:21.837 INFO:teuthology.orchestra.run.smithi002.stdout:(113/131): python3-numpy-1.14.3-10.el8.x86_64.r 25 MB/s | 3.7 MB 00:00 2023-08-25T03:31:21.862 INFO:teuthology.orchestra.run.smithi002.stdout:(114/131): python3-numpy-f2py-1.14.3-10.el8.x86 2.2 MB/s | 225 kB 00:00 2023-08-25T03:31:21.888 INFO:teuthology.orchestra.run.smithi002.stdout:(115/131): uuid-1.6.2-43.el8.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2023-08-25T03:31:21.921 INFO:teuthology.orchestra.run.smithi002.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 2.1 MB/s | 126 kB 00:00 2023-08-25T03:31:21.946 INFO:teuthology.orchestra.run.smithi002.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 341 kB/s | 20 kB 00:00 2023-08-25T03:31:21.971 INFO:teuthology.orchestra.run.smithi002.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-08-25T03:31:22.013 INFO:teuthology.orchestra.run.smithi002.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-08-25T03:31:22.055 INFO:teuthology.orchestra.run.smithi002.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-08-25T03:31:22.156 INFO:teuthology.orchestra.run.smithi002.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 26 MB/s | 4.9 MB 00:00 2023-08-25T03:31:22.197 INFO:teuthology.orchestra.run.smithi002.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-08-25T03:31:22.240 INFO:teuthology.orchestra.run.smithi002.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.2 MB/s | 354 kB 00:00 2023-08-25T03:31:22.307 INFO:teuthology.orchestra.run.smithi002.stdout:(124/131): openblas-0.3.15-3.el8.x86_64.rpm 19 MB/s | 4.7 MB 00:00 2023-08-25T03:31:22.357 INFO:teuthology.orchestra.run.smithi002.stdout:(125/131): postgresql-contrib-10.21-2.module+el 16 MB/s | 808 kB 00:00 2023-08-25T03:31:22.449 INFO:teuthology.orchestra.run.smithi002.stdout:(126/131): postgresql-server-10.21-2.module+el8 24 MB/s | 5.1 MB 00:00 2023-08-25T03:31:22.491 INFO:teuthology.orchestra.run.smithi002.stdout:(127/131): postgresql-10.21-2.module+el8.6.0+15 11 MB/s | 1.5 MB 00:00 2023-08-25T03:31:23.096 INFO:teuthology.orchestra.run.smithi031.stdout:(130/131): ceph-test-18.0.0-5703.gba6b39e4.el8. 2.9 MB/s | 48 MB 00:16 2023-08-25T03:31:24.516 INFO:teuthology.orchestra.run.smithi002.stdout:(128/131): librbd1-18.0.0-5703.gba6b39e4.el8.x8 1.8 MB/s | 3.6 MB 00:02 2023-08-25T03:31:24.566 INFO:teuthology.orchestra.run.smithi002.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-08-25T03:31:25.196 INFO:teuthology.orchestra.run.smithi031.stdout:(131/131): librados2-18.0.0-5703.gba6b39e4.el8. 511 kB/s | 3.7 MB 00:07 2023-08-25T03:31:25.199 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2023-08-25T03:31:25.199 INFO:teuthology.orchestra.run.smithi031.stdout:Total 7.5 MB/s | 206 MB 00:27 2023-08-25T03:31:25.216 INFO:teuthology.orchestra.run.smithi002.stdout:(130/131): librados2-18.0.0-5703.gba6b39e4.el8. 1.3 MB/s | 3.7 MB 00:02 2023-08-25T03:31:26.048 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2023-08-25T03:31:26.180 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2023-08-25T03:31:26.181 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2023-08-25T03:31:27.380 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2023-08-25T03:31:27.384 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2023-08-25T03:31:28.475 INFO:teuthology.orchestra.run.smithi002.stdout:(131/131): ceph-test-18.0.0-5703.gba6b39e4.el8. 1.9 MB/s | 48 MB 00:25 2023-08-25T03:31:28.478 INFO:teuthology.orchestra.run.smithi002.stdout:-------------------------------------------------------------------------------- 2023-08-25T03:31:28.478 INFO:teuthology.orchestra.run.smithi002.stdout:Total 6.5 MB/s | 206 MB 00:31 2023-08-25T03:31:29.339 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction check 2023-08-25T03:31:29.472 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction check succeeded. 2023-08-25T03:31:29.473 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction test 2023-08-25T03:31:30.664 INFO:teuthology.orchestra.run.smithi002.stdout:Transaction test succeeded. 2023-08-25T03:31:30.668 INFO:teuthology.orchestra.run.smithi002.stdout:Running transaction 2023-08-25T03:31:32.790 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2023-08-25T03:31:32.987 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-08-25T03:31:33.187 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-08-25T03:31:33.373 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-08-25T03:31:33.564 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-08-25T03:31:33.832 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-08-25T03:31:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-08-25T03:31:34.483 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-08-25T03:31:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 6/134 2023-08-25T03:31:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 6/134 2023-08-25T03:31:35.547 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 7/134 2023-08-25T03:31:35.548 INFO:teuthology.orchestra.run.smithi002.stdout: Preparing : 1/1 2023-08-25T03:31:35.684 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 7/134 2023-08-25T03:31:35.736 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-08-25T03:31:35.889 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 8/134 2023-08-25T03:31:35.955 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-08-25T03:31:35.987 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 8/134 2023-08-25T03:31:36.146 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-08-25T03:31:36.216 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rados-2:18.0.0-5703.gba6b39e4.el8.x86_64 9/134 2023-08-25T03:31:36.269 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-08-25T03:31:36.485 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-08-25T03:31:36.496 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-08-25T03:31:36.710 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-08-25T03:31:36.923 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-08-25T03:31:37.127 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-08-25T03:31:37.129 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 5/134 2023-08-25T03:31:37.405 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-08-25T03:31:37.549 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 5/134 2023-08-25T03:31:37.613 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 14/134 2023-08-25T03:31:37.725 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 14/134 2023-08-25T03:31:37.746 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 6/134 2023-08-25T03:31:37.879 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 6/134 2023-08-25T03:31:37.922 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86 15/134 2023-08-25T03:31:38.012 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86 15/134 2023-08-25T03:31:38.071 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 7/134 2023-08-25T03:31:38.138 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-08-25T03:31:38.180 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 7/134 2023-08-25T03:31:38.311 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-08-25T03:31:38.376 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rados-2:18.0.0-5703.gba6b39e4.el8.x86_64 8/134 2023-08-25T03:31:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-08-25T03:31:38.564 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-08-25T03:31:38.969 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-08-25T03:31:39.117 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-08-25T03:31:39.193 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-08-25T03:31:39.271 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-08-25T03:31:39.644 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-08-25T03:31:39.930 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-08-25T03:31:39.999 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-08-25T03:31:40.121 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-08-25T03:31:40.162 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 14/134 2023-08-25T03:31:40.272 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 14/134 2023-08-25T03:31:40.446 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86 15/134 2023-08-25T03:31:40.607 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86 15/134 2023-08-25T03:31:40.785 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-08-25T03:31:40.879 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-08-25T03:31:40.987 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-08-25T03:31:41.177 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-08-25T03:31:41.278 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-08-25T03:31:41.328 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-08-25T03:31:41.530 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-08-25T03:31:41.538 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-08-25T03:31:41.680 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-08-25T03:31:41.767 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-08-25T03:31:41.928 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-08-25T03:31:42.167 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-08-25T03:31:42.335 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-08-25T03:31:42.403 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-08-25T03:31:42.520 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-08-25T03:31:42.594 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-common-2:18.0.0-5703.gba6b39e4.el8. 26/134 2023-08-25T03:31:42.725 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el 27/134 2023-08-25T03:31:43.155 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-08-25T03:31:43.345 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-08-25T03:31:43.719 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cephfs-2:18.0.0-5703.gba6b39e4.el8.x86_6 28/134 2023-08-25T03:31:44.085 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-08-25T03:31:44.179 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-08-25T03:31:44.335 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-08-25T03:31:44.477 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-08-25T03:31:44.531 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-08-25T03:31:44.741 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-08-25T03:31:44.885 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-08-25T03:31:44.970 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-08-25T03:31:45.044 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ceph-common-2:18.0.0-5703.gba6b39e4.el8. 26/134 2023-08-25T03:31:45.165 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el 27/134 2023-08-25T03:31:45.424 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cephfs-2:18.0.0-5703.gba6b39e4.el8.x86_6 28/134 2023-08-25T03:31:45.595 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-08-25T03:31:46.844 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-08-25T03:31:47.342 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-08-25T03:31:47.478 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-08-25T03:31:47.545 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-08-25T03:31:47.783 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-08-25T03:31:47.985 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 36/134 2023-08-25T03:31:48.176 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librados-devel-2:18.0.0-5703.gba6b39e4.el8.x86_6 37/134 2023-08-25T03:31:48.375 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-08-25T03:31:48.565 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-08-25T03:31:48.623 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-08-25T03:31:48.831 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-08-25T03:31:48.975 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-08-25T03:31:49.002 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-08-25T03:31:49.214 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-08-25T03:31:49.251 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-08-25T03:31:49.445 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-08-25T03:31:49.461 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-08-25T03:31:49.659 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-08-25T03:31:49.849 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-08-25T03:31:49.857 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-08-25T03:31:50.051 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 36/134 2023-08-25T03:31:50.149 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-08-25T03:31:50.226 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librados-devel-2:18.0.0-5703.gba6b39e4.el8.x86_6 37/134 2023-08-25T03:31:50.365 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-08-25T03:31:50.395 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-08-25T03:31:50.564 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-08-25T03:31:50.584 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-08-25T03:31:50.769 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-08-25T03:31:50.811 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-08-25T03:31:50.845 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-08-25T03:31:51.087 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-08-25T03:31:51.154 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-08-25T03:31:51.316 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-08-25T03:31:51.395 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-08-25T03:31:51.477 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-08-25T03:31:51.549 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-08-25T03:31:51.613 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-08-25T03:31:51.754 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-08-25T03:31:51.863 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-08-25T03:31:51.924 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-08-25T03:31:52.037 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-08-25T03:31:52.094 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-08-25T03:31:52.233 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-08-25T03:31:52.280 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-08-25T03:31:52.505 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-08-25T03:31:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-08-25T03:31:52.682 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-08-25T03:31:52.893 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-08-25T03:31:52.921 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-08-25T03:31:53.146 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-08-25T03:31:53.170 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-08-25T03:31:53.418 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-08-25T03:31:53.611 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-08-25T03:31:53.660 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-08-25T03:31:53.756 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-08-25T03:31:53.914 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-08-25T03:31:53.943 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-08-25T03:31:54.017 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-08-25T03:31:54.106 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-08-25T03:31:54.286 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-08-25T03:31:54.363 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-08-25T03:31:54.532 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-08-25T03:31:54.677 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-08-25T03:31:54.711 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-08-25T03:31:54.770 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-08-25T03:31:54.978 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-08-25T03:31:55.056 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-08-25T03:31:55.163 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-08-25T03:31:55.303 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-08-25T03:31:55.345 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-08-25T03:31:55.518 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-08-25T03:31:55.526 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-08-25T03:31:55.754 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-08-25T03:31:55.822 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-08-25T03:31:55.957 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-08-25T03:31:56.068 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-08-25T03:31:56.123 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-08-25T03:31:56.215 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-08-25T03:31:56.398 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-08-25T03:31:56.453 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-08-25T03:31:56.615 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-08-25T03:31:56.717 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-08-25T03:31:56.809 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-08-25T03:31:56.969 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-08-25T03:31:57.016 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-08-25T03:31:57.186 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-08-25T03:31:57.197 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-08-25T03:31:57.350 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-08-25T03:31:57.360 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-08-25T03:31:57.549 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-08-25T03:31:57.586 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-08-25T03:31:57.807 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-08-25T03:31:57.848 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-08-25T03:31:58.075 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-08-25T03:31:58.084 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-08-25T03:31:58.240 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-08-25T03:31:58.313 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-08-25T03:31:58.389 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-08-25T03:31:58.526 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-08-25T03:31:58.590 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-08-25T03:31:58.719 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-08-25T03:31:58.791 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-08-25T03:31:58.963 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-08-25T03:31:58.999 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-08-25T03:31:59.163 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-08-25T03:31:59.708 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-08-25T03:31:59.953 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-08-25T03:32:00.147 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-08-25T03:32:00.165 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-08-25T03:32:00.374 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-08-25T03:32:00.416 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-08-25T03:32:00.649 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-08-25T03:32:00.686 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-08-25T03:32:00.893 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-08-25T03:32:00.989 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-08-25T03:32:01.060 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-08-25T03:32:01.249 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-08-25T03:32:01.279 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-08-25T03:32:01.434 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-08-25T03:32:02.322 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-08-25T03:32:02.432 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-08-25T03:32:02.534 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 92/134 2023-08-25T03:32:02.594 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 92/134 2023-08-25T03:32:02.683 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-08-25T03:32:02.804 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 93/134 2023-08-25T03:32:02.937 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-08-25T03:32:03.274 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-08-25T03:32:03.528 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-08-25T03:32:03.726 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-08-25T03:32:04.588 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-08-25T03:32:04.784 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 92/134 2023-08-25T03:32:04.877 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 92/134 2023-08-25T03:32:05.067 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : python3-rgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 93/134 2023-08-25T03:32:06.810 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 94/134 2023-08-25T03:32:07.025 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 94/134 2023-08-25T03:32:07.655 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 94/134 2023-08-25T03:32:07.946 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 95/134 2023-08-25T03:32:08.070 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 95/134 2023-08-25T03:32:08.972 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 94/134 2023-08-25T03:32:09.065 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 96/134 2023-08-25T03:32:09.200 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 94/134 2023-08-25T03:32:09.828 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 94/134 2023-08-25T03:32:10.824 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 95/134 2023-08-25T03:32:10.917 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 95/134 2023-08-25T03:32:11.082 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 96/134 2023-08-25T03:32:28.738 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 96/134 2023-08-25T03:32:28.738 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /sys 2023-08-25T03:32:28.739 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /proc 2023-08-25T03:32:28.739 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /dev 2023-08-25T03:32:28.739 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /run 2023-08-25T03:32:28.739 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /mnt 2023-08-25T03:32:28.739 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /var/tmp 2023-08-25T03:32:28.739 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /home 2023-08-25T03:32:28.739 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /tmp 2023-08-25T03:32:28.740 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /dev 2023-08-25T03:32:28.740 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:32:29.105 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 97/134 2023-08-25T03:32:30.747 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 96/134 2023-08-25T03:32:30.747 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /sys 2023-08-25T03:32:30.747 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /proc 2023-08-25T03:32:30.747 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /dev 2023-08-25T03:32:30.748 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /run 2023-08-25T03:32:30.748 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /mnt 2023-08-25T03:32:30.748 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /var/tmp 2023-08-25T03:32:30.748 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /home 2023-08-25T03:32:30.748 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /tmp 2023-08-25T03:32:30.749 INFO:teuthology.orchestra.run.smithi002.stdout:skipping the directory /dev 2023-08-25T03:32:30.749 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:32:30.787 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 97/134 2023-08-25T03:32:30.992 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 98/134 2023-08-25T03:32:31.113 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 97/134 2023-08-25T03:32:31.224 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 98/134 2023-08-25T03:32:31.355 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 99/134 2023-08-25T03:32:31.836 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 99/134 2023-08-25T03:32:32.080 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 100/134 2023-08-25T03:32:32.097 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 100/134 2023-08-25T03:32:32.794 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 97/134 2023-08-25T03:32:32.977 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 98/134 2023-08-25T03:32:33.210 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 98/134 2023-08-25T03:32:33.371 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 99/134 2023-08-25T03:32:33.854 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 99/134 2023-08-25T03:32:33.902 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 101/134 2023-08-25T03:32:34.063 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 100/134 2023-08-25T03:32:34.080 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 100/134 2023-08-25T03:32:34.095 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 101/134 2023-08-25T03:32:34.219 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.e 102/134 2023-08-25T03:32:34.439 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4. 103/134 2023-08-25T03:32:34.687 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noa 104/134 2023-08-25T03:32:34.914 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noa 104/134 2023-08-25T03:32:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.n 105/134 2023-08-25T03:32:35.848 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 101/134 2023-08-25T03:32:36.029 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 101/134 2023-08-25T03:32:36.169 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.e 102/134 2023-08-25T03:32:36.213 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.n 105/134 2023-08-25T03:32:36.345 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 106/134 2023-08-25T03:32:36.369 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4. 103/134 2023-08-25T03:32:36.441 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 106/134 2023-08-25T03:32:36.570 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noa 104/134 2023-08-25T03:32:36.796 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el 107/134 2023-08-25T03:32:36.797 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noa 104/134 2023-08-25T03:32:36.967 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 108/134 2023-08-25T03:32:37.003 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 108/134 2023-08-25T03:32:37.143 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.n 105/134 2023-08-25T03:32:37.203 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 109/134 2023-08-25T03:32:37.215 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 109/134 2023-08-25T03:32:38.165 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.n 105/134 2023-08-25T03:32:38.336 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 106/134 2023-08-25T03:32:38.435 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 106/134 2023-08-25T03:32:38.618 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-2:18.0.0-5703.gba6b39e4.el8.x86_64 110/134 2023-08-25T03:32:38.807 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el 107/134 2023-08-25T03:32:38.843 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 111/134 2023-08-25T03:32:38.883 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 111/134 2023-08-25T03:32:38.993 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 108/134 2023-08-25T03:32:39.020 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 112/134 2023-08-25T03:32:39.029 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 108/134 2023-08-25T03:32:39.063 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 112/134 2023-08-25T03:32:39.203 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 109/134 2023-08-25T03:32:39.204 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 113/134 2023-08-25T03:32:39.215 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 109/134 2023-08-25T03:32:39.578 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 113/134 2023-08-25T03:32:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 114/134 2023-08-25T03:32:40.594 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-2:18.0.0-5703.gba6b39e4.el8.x86_64 110/134 2023-08-25T03:32:40.807 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 111/134 2023-08-25T03:32:40.845 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 111/134 2023-08-25T03:32:41.020 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 112/134 2023-08-25T03:32:41.063 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 112/134 2023-08-25T03:32:41.237 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 113/134 2023-08-25T03:32:41.611 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 113/134 2023-08-25T03:32:41.787 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 114/134 2023-08-25T03:32:44.635 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 114/134 2023-08-25T03:32:44.926 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-test-2:18.0.0-5703.gba6b39e4.el8.x86_64 115/134 2023-08-25T03:32:45.102 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-08-25T03:32:45.306 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-08-25T03:32:46.248 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-08-25T03:32:46.573 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-08-25T03:32:46.763 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 119/134 2023-08-25T03:32:46.789 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 114/134 2023-08-25T03:32:46.964 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-08-25T03:32:47.123 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs-devel-2:18.0.0-5703.gba6b39e4.el8.x86_ 121/134 2023-08-25T03:32:47.303 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-25T03:32:47.709 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-test-2:18.0.0-5703.gba6b39e4.el8.x86_64 115/134 2023-08-25T03:32:48.031 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-08-25T03:32:48.216 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : ceph-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 117/134 2023-08-25T03:32:48.468 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-08-25T03:32:48.643 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-08-25T03:32:48.850 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-08-25T03:32:49.081 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-08-25T03:32:49.248 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libcephfs-devel-2:18.0.0-5703.gba6b39e4.el8.x86_ 121/134 2023-08-25T03:32:49.394 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-25T03:32:49.680 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-25T03:32:49.935 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-25T03:32:49.953 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-25T03:32:50.149 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephfs-top-2:18.0.0-5703.gba6b39e4.el8.noarch 124/134 2023-08-25T03:32:50.410 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 125/134 2023-08-25T03:32:50.725 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-nbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 126/134 2023-08-25T03:32:51.045 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-08-25T03:32:51.184 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-25T03:32:51.193 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-25T03:32:51.414 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-08-25T03:32:51.610 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-08-25T03:32:51.697 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-25T03:32:51.701 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-08-25T03:32:51.935 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-25T03:32:51.951 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-08-25T03:32:51.953 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-08-25T03:32:51.964 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-08-25T03:32:51.965 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-25T03:32:52.127 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : cephfs-top-2:18.0.0-5703.gba6b39e4.el8.noarch 124/134 2023-08-25T03:32:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-25T03:32:52.138 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-25T03:32:52.394 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 125/134 2023-08-25T03:32:52.408 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-25T03:32:52.600 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-25T03:32:52.678 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : rbd-nbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 126/134 2023-08-25T03:32:53.008 INFO:teuthology.orchestra.run.smithi002.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-08-25T03:32:53.132 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-08-25T03:32:53.132 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-2:18.0.0-5703.gba6b39e4.el8.x86_64 1/134 2023-08-25T03:32:53.132 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 2/134 2023-08-25T03:32:53.132 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 3/134 2023-08-25T03:32:53.133 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 4/134 2023-08-25T03:32:53.133 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 5/134 2023-08-25T03:32:53.133 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 6/134 2023-08-25T03:32:53.133 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 7/134 2023-08-25T03:32:53.133 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 8/134 2023-08-25T03:32:53.133 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 9/134 2023-08-25T03:32:53.133 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 10/134 2023-08-25T03:32:53.134 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 11/134 2023-08-25T03:32:53.134 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-test-2:18.0.0-5703.gba6b39e4.el8.x86_64 12/134 2023-08-25T03:32:53.135 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 13/134 2023-08-25T03:32:53.135 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs-devel-2:18.0.0-5703.gba6b39e4.el8.x86_ 14/134 2023-08-25T03:32:53.136 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 15/134 2023-08-25T03:32:53.136 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 16/134 2023-08-25T03:32:53.137 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados-devel-2:18.0.0-5703.gba6b39e4.el8.x86_6 17/134 2023-08-25T03:32:53.137 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86 18/134 2023-08-25T03:32:53.137 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 19/134 2023-08-25T03:32:53.138 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el 20/134 2023-08-25T03:32:53.139 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-common-2:18.0.0-5703.gba6b39e4.el8. 21/134 2023-08-25T03:32:53.139 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cephfs-2:18.0.0-5703.gba6b39e4.el8.x86_6 22/134 2023-08-25T03:32:53.139 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rados-2:18.0.0-5703.gba6b39e4.el8.x86_64 23/134 2023-08-25T03:32:53.139 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 24/134 2023-08-25T03:32:53.139 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 25/134 2023-08-25T03:32:53.139 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 26/134 2023-08-25T03:32:53.139 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 27/134 2023-08-25T03:32:53.140 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-nbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 28/134 2023-08-25T03:32:53.140 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4. 29/134 2023-08-25T03:32:53.140 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noa 30/134 2023-08-25T03:32:53.147 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.n 31/134 2023-08-25T03:32:53.147 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 32/134 2023-08-25T03:32:53.148 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el 33/134 2023-08-25T03:32:53.148 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 34/134 2023-08-25T03:32:53.148 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.e 35/134 2023-08-25T03:32:53.148 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 36/134 2023-08-25T03:32:53.148 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 37/134 2023-08-25T03:32:53.148 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephfs-top-2:18.0.0-5703.gba6b39e4.el8.noarch 38/134 2023-08-25T03:32:53.149 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-08-25T03:32:53.149 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-08-25T03:32:53.149 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-08-25T03:32:53.149 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-08-25T03:32:53.149 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-08-25T03:32:53.149 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-08-25T03:32:53.149 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-08-25T03:32:53.150 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-08-25T03:32:53.150 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-08-25T03:32:53.150 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-08-25T03:32:53.150 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-08-25T03:32:53.150 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-08-25T03:32:53.150 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-08-25T03:32:53.150 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-08-25T03:32:53.150 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-08-25T03:32:53.151 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-08-25T03:32:53.151 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-08-25T03:32:53.151 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-08-25T03:32:53.151 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-08-25T03:32:53.151 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-08-25T03:32:53.151 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-08-25T03:32:53.151 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-08-25T03:32:53.151 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-08-25T03:32:53.152 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-08-25T03:32:53.152 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-08-25T03:32:53.152 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-08-25T03:32:53.152 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-08-25T03:32:53.152 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-08-25T03:32:53.152 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-08-25T03:32:53.152 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-08-25T03:32:53.153 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-08-25T03:32:53.153 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-08-25T03:32:53.153 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-08-25T03:32:53.153 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-08-25T03:32:53.153 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-08-25T03:32:53.153 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-08-25T03:32:53.153 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-08-25T03:32:53.153 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-08-25T03:32:53.154 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-08-25T03:32:53.154 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-08-25T03:32:53.154 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-08-25T03:32:53.154 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-08-25T03:32:53.154 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-08-25T03:32:53.154 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-08-25T03:32:53.154 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-08-25T03:32:53.155 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-08-25T03:32:53.155 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-08-25T03:32:53.155 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-08-25T03:32:53.155 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-08-25T03:32:53.155 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-08-25T03:32:53.156 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-08-25T03:32:53.156 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-08-25T03:32:53.156 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-08-25T03:32:53.156 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-08-25T03:32:53.156 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-08-25T03:32:53.156 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-08-25T03:32:53.157 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-08-25T03:32:53.157 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-08-25T03:32:53.157 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-08-25T03:32:53.157 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-08-25T03:32:53.157 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-08-25T03:32:53.158 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-08-25T03:32:53.158 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-08-25T03:32:53.158 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-08-25T03:32:53.158 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-08-25T03:32:53.158 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-08-25T03:32:53.158 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-08-25T03:32:53.158 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-08-25T03:32:53.159 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-08-25T03:32:53.159 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-08-25T03:32:53.159 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-08-25T03:32:53.159 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-08-25T03:32:53.159 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-08-25T03:32:53.159 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-08-25T03:32:53.159 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-08-25T03:32:53.160 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-08-25T03:32:53.160 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-08-25T03:32:53.160 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-08-25T03:32:53.160 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-08-25T03:32:53.160 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-08-25T03:32:53.160 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-08-25T03:32:53.160 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-08-25T03:32:53.161 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-08-25T03:32:53.161 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-08-25T03:32:53.161 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-08-25T03:32:53.161 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-08-25T03:32:53.161 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-08-25T03:32:53.161 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-08-25T03:32:53.161 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-08-25T03:32:53.161 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-08-25T03:32:53.162 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 129/134 2023-08-25T03:32:53.162 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-08-25T03:32:53.162 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 131/134 2023-08-25T03:32:53.162 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-25T03:32:53.162 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-08-25T03:32:53.217 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-08-25T03:32:53.303 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-08-25T03:32:53.526 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-08-25T03:32:53.539 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-08-25T03:32:53.692 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-08-25T03:32:53.701 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-08-25T03:32:53.828 INFO:teuthology.orchestra.run.smithi002.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-08-25T03:32:53.829 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-25T03:32:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-25T03:32:54.046 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-25T03:32:54.283 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-08-25T03:32:54.467 INFO:teuthology.orchestra.run.smithi002.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-25T03:32:55.005 INFO:teuthology.orchestra.run.smithi002.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-08-25T03:32:55.006 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-2:18.0.0-5703.gba6b39e4.el8.x86_64 1/134 2023-08-25T03:32:55.006 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 2/134 2023-08-25T03:32:55.006 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 3/134 2023-08-25T03:32:55.006 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 4/134 2023-08-25T03:32:55.006 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-5703.gba6b3 5/134 2023-08-25T03:32:55.007 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 6/134 2023-08-25T03:32:55.007 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 7/134 2023-08-25T03:32:55.007 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 8/134 2023-08-25T03:32:55.007 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 9/134 2023-08-25T03:32:55.007 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 10/134 2023-08-25T03:32:55.007 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 11/134 2023-08-25T03:32:55.008 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-test-2:18.0.0-5703.gba6b39e4.el8.x86_64 12/134 2023-08-25T03:32:55.008 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 13/134 2023-08-25T03:32:55.008 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephfs-devel-2:18.0.0-5703.gba6b39e4.el8.x86_ 14/134 2023-08-25T03:32:55.008 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 15/134 2023-08-25T03:32:55.008 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 16/134 2023-08-25T03:32:55.009 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados-devel-2:18.0.0-5703.gba6b39e4.el8.x86_6 17/134 2023-08-25T03:32:55.009 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86 18/134 2023-08-25T03:32:55.009 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 19/134 2023-08-25T03:32:55.009 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el 20/134 2023-08-25T03:32:55.009 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-ceph-common-2:18.0.0-5703.gba6b39e4.el8. 21/134 2023-08-25T03:32:55.009 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cephfs-2:18.0.0-5703.gba6b39e4.el8.x86_6 22/134 2023-08-25T03:32:55.009 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rados-2:18.0.0-5703.gba6b39e4.el8.x86_64 23/134 2023-08-25T03:32:55.010 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 24/134 2023-08-25T03:32:55.010 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 25/134 2023-08-25T03:32:55.010 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 26/134 2023-08-25T03:32:55.010 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 27/134 2023-08-25T03:32:55.010 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : rbd-nbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 28/134 2023-08-25T03:32:55.010 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4. 29/134 2023-08-25T03:32:55.011 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noa 30/134 2023-08-25T03:32:55.011 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.n 31/134 2023-08-25T03:32:55.011 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6 32/134 2023-08-25T03:32:55.011 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el 33/134 2023-08-25T03:32:55.011 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 34/134 2023-08-25T03:32:55.011 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.e 35/134 2023-08-25T03:32:55.011 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 36/134 2023-08-25T03:32:55.011 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 37/134 2023-08-25T03:32:55.012 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : cephfs-top-2:18.0.0-5703.gba6b39e4.el8.noarch 38/134 2023-08-25T03:32:55.012 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-08-25T03:32:55.012 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-08-25T03:32:55.012 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-08-25T03:32:55.012 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-08-25T03:32:55.012 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-08-25T03:32:55.013 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-08-25T03:32:55.013 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-08-25T03:32:55.013 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-08-25T03:32:55.013 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-08-25T03:32:55.013 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-08-25T03:32:55.013 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-08-25T03:32:55.013 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-08-25T03:32:55.014 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-08-25T03:32:55.014 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-08-25T03:32:55.014 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-08-25T03:32:55.014 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-08-25T03:32:55.014 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-08-25T03:32:55.014 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-08-25T03:32:55.015 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-08-25T03:32:55.015 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-08-25T03:32:55.015 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-08-25T03:32:55.015 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-08-25T03:32:55.015 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-08-25T03:32:55.015 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-08-25T03:32:55.015 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-08-25T03:32:55.015 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-08-25T03:32:55.016 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-08-25T03:32:55.016 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-08-25T03:32:55.016 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-08-25T03:32:55.016 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-08-25T03:32:55.016 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-08-25T03:32:55.016 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-08-25T03:32:55.016 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-08-25T03:32:55.017 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-08-25T03:32:55.017 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-08-25T03:32:55.017 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-08-25T03:32:55.017 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-08-25T03:32:55.017 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-08-25T03:32:55.017 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-08-25T03:32:55.017 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-08-25T03:32:55.017 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-08-25T03:32:55.018 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-08-25T03:32:55.018 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-08-25T03:32:55.018 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-08-25T03:32:55.018 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-08-25T03:32:55.018 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-08-25T03:32:55.018 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-08-25T03:32:55.018 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-08-25T03:32:55.018 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-08-25T03:32:55.019 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-08-25T03:32:55.019 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-08-25T03:32:55.019 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-08-25T03:32:55.019 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-08-25T03:32:55.019 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-08-25T03:32:55.019 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-08-25T03:32:55.019 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-08-25T03:32:55.019 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-08-25T03:32:55.020 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-08-25T03:32:55.020 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-08-25T03:32:55.020 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-08-25T03:32:55.020 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-08-25T03:32:55.020 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-08-25T03:32:55.021 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-08-25T03:32:55.021 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-08-25T03:32:55.021 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-08-25T03:32:55.021 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-08-25T03:32:55.021 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-08-25T03:32:55.021 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-08-25T03:32:55.021 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-08-25T03:32:55.022 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-08-25T03:32:55.022 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-08-25T03:32:55.022 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-08-25T03:32:55.022 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-08-25T03:32:55.022 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-08-25T03:32:55.022 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-08-25T03:32:55.022 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-08-25T03:32:55.023 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-08-25T03:32:55.023 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-08-25T03:32:55.023 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-08-25T03:32:55.023 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-08-25T03:32:55.023 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-08-25T03:32:55.023 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-08-25T03:32:55.023 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-08-25T03:32:55.024 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-08-25T03:32:55.024 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-08-25T03:32:55.024 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-08-25T03:32:55.024 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-08-25T03:32:55.024 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-08-25T03:32:55.024 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-08-25T03:32:55.025 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-08-25T03:32:55.025 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 129/134 2023-08-25T03:32:55.025 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-08-25T03:32:55.025 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 131/134 2023-08-25T03:32:55.025 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-08-25T03:32:55.025 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-08-25T03:32:55.938 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-25T03:32:55.938 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:10 ago on Fri 25 Aug 2023 03:30:45 AM UTC. 2023-08-25T03:32:57.554 INFO:teuthology.orchestra.run.smithi002.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-08-25T03:32:57.555 INFO:teuthology.orchestra.run.smithi002.stdout:Last metadata expiration check: 0:02:13 ago on Fri 25 Aug 2023 03:30:44 AM UTC. 2023-08-25T03:32:58.614 INFO:teuthology.orchestra.run.smithi031.stdout:Installed products updated. 2023-08-25T03:32:58.853 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:32:58.854 INFO:teuthology.orchestra.run.smithi031.stdout:Upgraded: 2023-08-25T03:32:58.854 INFO:teuthology.orchestra.run.smithi031.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-08-25T03:32:58.854 INFO:teuthology.orchestra.run.smithi031.stdout: librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.854 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.854 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2023-08-25T03:32:58.854 INFO:teuthology.orchestra.run.smithi031.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-08-25T03:32:58.854 INFO:teuthology.orchestra.run.smithi031.stdout: automake-1.16.1-7.el8.noarch 2023-08-25T03:32:58.855 INFO:teuthology.orchestra.run.smithi031.stdout: bison-3.0.4-10.el8.x86_64 2023-08-25T03:32:58.855 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.855 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.855 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.855 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.855 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T03:32:58.855 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.855 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.856 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.856 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T03:32:58.856 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T03:32:58.857 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T03:32:58.857 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T03:32:58.857 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T03:32:58.857 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.858 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.858 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T03:32:58.858 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.858 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.858 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.858 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T03:32:58.859 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T03:32:58.859 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.859 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-top-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T03:32:58.860 INFO:teuthology.orchestra.run.smithi031.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-08-25T03:32:58.860 INFO:teuthology.orchestra.run.smithi031.stdout: flex-2.6.1-9.el8.x86_64 2023-08-25T03:32:58.860 INFO:teuthology.orchestra.run.smithi031.stdout: fuse-2.9.7-15.el8.x86_64 2023-08-25T03:32:58.860 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-08-25T03:32:58.861 INFO:teuthology.orchestra.run.smithi031.stdout: jq-1.6-3.el8.x86_64 2023-08-25T03:32:58.861 INFO:teuthology.orchestra.run.smithi031.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-08-25T03:32:58.861 INFO:teuthology.orchestra.run.smithi031.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-08-25T03:32:58.861 INFO:teuthology.orchestra.run.smithi031.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-08-25T03:32:58.862 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.862 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.862 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.862 INFO:teuthology.orchestra.run.smithi031.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-08-25T03:32:58.863 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-08-25T03:32:58.863 INFO:teuthology.orchestra.run.smithi031.stdout: liboath-2.6.2-3.el8.x86_64 2023-08-25T03:32:58.863 INFO:teuthology.orchestra.run.smithi031.stdout: libpq-13.5-1.el8.x86_64 2023-08-25T03:32:58.863 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-08-25T03:32:58.864 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-08-25T03:32:58.864 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.864 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.864 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-08-25T03:32:58.864 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.864 INFO:teuthology.orchestra.run.smithi031.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-08-25T03:32:58.864 INFO:teuthology.orchestra.run.smithi031.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-08-25T03:32:58.865 INFO:teuthology.orchestra.run.smithi031.stdout: libtool-2.4.6-25.el8.x86_64 2023-08-25T03:32:58.865 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind-1.3.1-3.el8.x86_64 2023-08-25T03:32:58.865 INFO:teuthology.orchestra.run.smithi031.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-08-25T03:32:58.865 INFO:teuthology.orchestra.run.smithi031.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-08-25T03:32:58.865 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-08-25T03:32:58.865 INFO:teuthology.orchestra.run.smithi031.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-08-25T03:32:58.865 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-0.3.15-3.el8.x86_64 2023-08-25T03:32:58.866 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-08-25T03:32:58.866 INFO:teuthology.orchestra.run.smithi031.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-08-25T03:32:58.866 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-08-25T03:32:58.867 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-08-25T03:32:58.867 INFO:teuthology.orchestra.run.smithi031.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-08-25T03:32:58.867 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-25T03:32:58.867 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-25T03:32:58.867 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-25T03:32:58.868 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-08-25T03:32:58.868 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel-2.5.1-7.el8.noarch 2023-08-25T03:32:58.868 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-08-25T03:32:58.868 INFO:teuthology.orchestra.run.smithi031.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-08-25T03:32:58.868 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-08-25T03:32:58.868 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.868 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.869 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.869 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-08-25T03:32:58.869 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-08-25T03:32:58.869 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-08-25T03:32:58.869 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-08-25T03:32:58.869 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-08-25T03:32:58.869 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-08-25T03:32:58.869 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-6.2-6.el8.noarch 2023-08-25T03:32:58.870 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-08-25T03:32:58.870 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-08-25T03:32:58.870 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-08-25T03:32:58.870 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-08-25T03:32:58.870 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-08-25T03:32:58.870 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-08-25T03:32:58.870 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-08-25T03:32:58.871 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako-1.0.6-13.el8.noarch 2023-08-25T03:32:58.871 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-08-25T03:32:58.871 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-08-25T03:32:58.871 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-08-25T03:32:58.871 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-08-25T03:32:58.871 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-08-25T03:32:58.872 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-08-25T03:32:58.872 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-08-25T03:32:58.872 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend-2.6-1.el8.noarch 2023-08-25T03:32:58.872 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-08-25T03:32:58.872 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-08-25T03:32:58.872 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-08-25T03:32:58.873 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-08-25T03:32:58.873 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser-2.14-14.el8.noarch 2023-08-25T03:32:58.873 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz-2017.2-9.el8.noarch 2023-08-25T03:32:58.873 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.873 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.873 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-08-25T03:32:58.873 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-08-25T03:32:58.874 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.874 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes-2.4.1-12.el8.noarch 2023-08-25T03:32:58.874 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa-4.9-2.el8.noarch 2023-08-25T03:32:58.874 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-08-25T03:32:58.874 INFO:teuthology.orchestra.run.smithi031.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-08-25T03:32:58.874 INFO:teuthology.orchestra.run.smithi031.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-08-25T03:32:58.874 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-08-25T03:32:58.874 INFO:teuthology.orchestra.run.smithi031.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-08-25T03:32:58.875 INFO:teuthology.orchestra.run.smithi031.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-08-25T03:32:58.875 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-08-25T03:32:58.875 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-08-25T03:32:58.875 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-08-25T03:32:58.876 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-08-25T03:32:58.876 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-08-25T03:32:58.876 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.876 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.876 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:32:58.876 INFO:teuthology.orchestra.run.smithi031.stdout: socat-1.7.4.1-1.el8.x86_64 2023-08-25T03:32:58.877 INFO:teuthology.orchestra.run.smithi031.stdout: thrift-0.13.0-2.el8.x86_64 2023-08-25T03:32:58.877 INFO:teuthology.orchestra.run.smithi031.stdout: uuid-1.6.2-43.el8.x86_64 2023-08-25T03:32:58.877 INFO:teuthology.orchestra.run.smithi031.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-08-25T03:32:58.877 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-08-25T03:32:58.877 INFO:teuthology.orchestra.run.smithi031.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-08-25T03:32:58.878 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:32:58.878 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2023-08-25T03:32:59.171 DEBUG:teuthology.parallel:result is None 2023-08-25T03:33:00.504 INFO:teuthology.orchestra.run.smithi002.stdout:Installed products updated. 2023-08-25T03:33:00.752 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:00.753 INFO:teuthology.orchestra.run.smithi002.stdout:Upgraded: 2023-08-25T03:33:00.753 INFO:teuthology.orchestra.run.smithi002.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-08-25T03:33:00.753 INFO:teuthology.orchestra.run.smithi002.stdout: librados2-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.753 INFO:teuthology.orchestra.run.smithi002.stdout: librbd1-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.753 INFO:teuthology.orchestra.run.smithi002.stdout:Installed: 2023-08-25T03:33:00.754 INFO:teuthology.orchestra.run.smithi002.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-08-25T03:33:00.754 INFO:teuthology.orchestra.run.smithi002.stdout: automake-1.16.1-7.el8.noarch 2023-08-25T03:33:00.754 INFO:teuthology.orchestra.run.smithi002.stdout: bison-3.0.4-10.el8.x86_64 2023-08-25T03:33:00.754 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.754 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-base-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.754 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.754 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.755 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-grafana-dashboards-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T03:33:00.755 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-immutable-object-cache-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.755 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mds-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.755 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.755 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T03:33:00.755 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-dashboard-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T03:33:00.755 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-diskprediction-local-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T03:33:00.755 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-modules-core-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T03:33:00.756 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mgr-rook-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T03:33:00.756 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-mon-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.756 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-osd-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.756 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-prometheus-alerts-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T03:33:00.756 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-radosgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.756 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-selinux-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.756 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-test-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.757 INFO:teuthology.orchestra.run.smithi002.stdout: ceph-volume-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T03:33:00.757 INFO:teuthology.orchestra.run.smithi002.stdout: cephadm-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T03:33:00.757 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.757 INFO:teuthology.orchestra.run.smithi002.stdout: cephfs-top-2:18.0.0-5703.gba6b39e4.el8.noarch 2023-08-25T03:33:00.757 INFO:teuthology.orchestra.run.smithi002.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-08-25T03:33:00.757 INFO:teuthology.orchestra.run.smithi002.stdout: flex-2.6.1-9.el8.x86_64 2023-08-25T03:33:00.757 INFO:teuthology.orchestra.run.smithi002.stdout: fuse-2.9.7-15.el8.x86_64 2023-08-25T03:33:00.757 INFO:teuthology.orchestra.run.smithi002.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-08-25T03:33:00.758 INFO:teuthology.orchestra.run.smithi002.stdout: jq-1.6-3.el8.x86_64 2023-08-25T03:33:00.758 INFO:teuthology.orchestra.run.smithi002.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-08-25T03:33:00.758 INFO:teuthology.orchestra.run.smithi002.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-08-25T03:33:00.758 INFO:teuthology.orchestra.run.smithi002.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-08-25T03:33:00.758 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs-devel-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.758 INFO:teuthology.orchestra.run.smithi002.stdout: libcephfs2-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.758 INFO:teuthology.orchestra.run.smithi002.stdout: libcephsqlite-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.759 INFO:teuthology.orchestra.run.smithi002.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-08-25T03:33:00.759 INFO:teuthology.orchestra.run.smithi002.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-08-25T03:33:00.759 INFO:teuthology.orchestra.run.smithi002.stdout: liboath-2.6.2-3.el8.x86_64 2023-08-25T03:33:00.759 INFO:teuthology.orchestra.run.smithi002.stdout: libpq-13.5-1.el8.x86_64 2023-08-25T03:33:00.759 INFO:teuthology.orchestra.run.smithi002.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-08-25T03:33:00.759 INFO:teuthology.orchestra.run.smithi002.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-08-25T03:33:00.759 INFO:teuthology.orchestra.run.smithi002.stdout: librados-devel-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.759 INFO:teuthology.orchestra.run.smithi002.stdout: libradosstriper1-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.760 INFO:teuthology.orchestra.run.smithi002.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-08-25T03:33:00.760 INFO:teuthology.orchestra.run.smithi002.stdout: librgw2-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.760 INFO:teuthology.orchestra.run.smithi002.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-08-25T03:33:00.760 INFO:teuthology.orchestra.run.smithi002.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-08-25T03:33:00.760 INFO:teuthology.orchestra.run.smithi002.stdout: libtool-2.4.6-25.el8.x86_64 2023-08-25T03:33:00.760 INFO:teuthology.orchestra.run.smithi002.stdout: libunwind-1.3.1-3.el8.x86_64 2023-08-25T03:33:00.760 INFO:teuthology.orchestra.run.smithi002.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-08-25T03:33:00.760 INFO:teuthology.orchestra.run.smithi002.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-08-25T03:33:00.761 INFO:teuthology.orchestra.run.smithi002.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-08-25T03:33:00.761 INFO:teuthology.orchestra.run.smithi002.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-08-25T03:33:00.761 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-0.3.15-3.el8.x86_64 2023-08-25T03:33:00.761 INFO:teuthology.orchestra.run.smithi002.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-08-25T03:33:00.761 INFO:teuthology.orchestra.run.smithi002.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-08-25T03:33:00.761 INFO:teuthology.orchestra.run.smithi002.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-08-25T03:33:00.761 INFO:teuthology.orchestra.run.smithi002.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-08-25T03:33:00.762 INFO:teuthology.orchestra.run.smithi002.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-08-25T03:33:00.762 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-25T03:33:00.762 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-25T03:33:00.762 INFO:teuthology.orchestra.run.smithi002.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-08-25T03:33:00.762 INFO:teuthology.orchestra.run.smithi002.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-08-25T03:33:00.762 INFO:teuthology.orchestra.run.smithi002.stdout: python3-babel-2.5.1-7.el8.noarch 2023-08-25T03:33:00.762 INFO:teuthology.orchestra.run.smithi002.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-08-25T03:33:00.762 INFO:teuthology.orchestra.run.smithi002.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-08-25T03:33:00.763 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-08-25T03:33:00.763 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-argparse-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.763 INFO:teuthology.orchestra.run.smithi002.stdout: python3-ceph-common-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.763 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cephfs-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.763 INFO:teuthology.orchestra.run.smithi002.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-08-25T03:33:00.763 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-08-25T03:33:00.763 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-08-25T03:33:00.764 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-08-25T03:33:00.764 INFO:teuthology.orchestra.run.smithi002.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-08-25T03:33:00.764 INFO:teuthology.orchestra.run.smithi002.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-08-25T03:33:00.764 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-6.2-6.el8.noarch 2023-08-25T03:33:00.764 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-08-25T03:33:00.764 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-08-25T03:33:00.764 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-08-25T03:33:00.764 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-08-25T03:33:00.765 INFO:teuthology.orchestra.run.smithi002.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-08-25T03:33:00.765 INFO:teuthology.orchestra.run.smithi002.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-08-25T03:33:00.765 INFO:teuthology.orchestra.run.smithi002.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-08-25T03:33:00.765 INFO:teuthology.orchestra.run.smithi002.stdout: python3-mako-1.0.6-13.el8.noarch 2023-08-25T03:33:00.765 INFO:teuthology.orchestra.run.smithi002.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-08-25T03:33:00.765 INFO:teuthology.orchestra.run.smithi002.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-08-25T03:33:00.765 INFO:teuthology.orchestra.run.smithi002.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-08-25T03:33:00.765 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-08-25T03:33:00.766 INFO:teuthology.orchestra.run.smithi002.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-08-25T03:33:00.766 INFO:teuthology.orchestra.run.smithi002.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-08-25T03:33:00.766 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-08-25T03:33:00.766 INFO:teuthology.orchestra.run.smithi002.stdout: python3-portend-2.6-1.el8.noarch 2023-08-25T03:33:00.766 INFO:teuthology.orchestra.run.smithi002.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-08-25T03:33:00.766 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-08-25T03:33:00.766 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-08-25T03:33:00.767 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-08-25T03:33:00.767 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pycparser-2.14-14.el8.noarch 2023-08-25T03:33:00.767 INFO:teuthology.orchestra.run.smithi002.stdout: python3-pytz-2017.2-9.el8.noarch 2023-08-25T03:33:00.767 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rados-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.767 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.767 INFO:teuthology.orchestra.run.smithi002.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-08-25T03:33:00.767 INFO:teuthology.orchestra.run.smithi002.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-08-25T03:33:00.767 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rgw-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.768 INFO:teuthology.orchestra.run.smithi002.stdout: python3-routes-2.4.1-12.el8.noarch 2023-08-25T03:33:00.768 INFO:teuthology.orchestra.run.smithi002.stdout: python3-rsa-4.9-2.el8.noarch 2023-08-25T03:33:00.768 INFO:teuthology.orchestra.run.smithi002.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-08-25T03:33:00.768 INFO:teuthology.orchestra.run.smithi002.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-08-25T03:33:00.768 INFO:teuthology.orchestra.run.smithi002.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-08-25T03:33:00.768 INFO:teuthology.orchestra.run.smithi002.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-08-25T03:33:00.768 INFO:teuthology.orchestra.run.smithi002.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-08-25T03:33:00.769 INFO:teuthology.orchestra.run.smithi002.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-08-25T03:33:00.769 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-08-25T03:33:00.769 INFO:teuthology.orchestra.run.smithi002.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-08-25T03:33:00.769 INFO:teuthology.orchestra.run.smithi002.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-08-25T03:33:00.769 INFO:teuthology.orchestra.run.smithi002.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-08-25T03:33:00.769 INFO:teuthology.orchestra.run.smithi002.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-08-25T03:33:00.769 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-fuse-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.770 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-mirror-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.770 INFO:teuthology.orchestra.run.smithi002.stdout: rbd-nbd-2:18.0.0-5703.gba6b39e4.el8.x86_64 2023-08-25T03:33:00.770 INFO:teuthology.orchestra.run.smithi002.stdout: socat-1.7.4.1-1.el8.x86_64 2023-08-25T03:33:00.770 INFO:teuthology.orchestra.run.smithi002.stdout: thrift-0.13.0-2.el8.x86_64 2023-08-25T03:33:00.770 INFO:teuthology.orchestra.run.smithi002.stdout: uuid-1.6.2-43.el8.x86_64 2023-08-25T03:33:00.770 INFO:teuthology.orchestra.run.smithi002.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-08-25T03:33:00.770 INFO:teuthology.orchestra.run.smithi002.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-08-25T03:33:00.771 INFO:teuthology.orchestra.run.smithi002.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-08-25T03:33:00.771 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:00.771 INFO:teuthology.orchestra.run.smithi002.stdout:Complete! 2023-08-25T03:33:00.961 DEBUG:teuthology.parallel:result is None 2023-08-25T03:33:00.962 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T03:33:01.108 DEBUG:teuthology.orchestra.run.smithi002:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-08-25T03:33:01.133 INFO:teuthology.orchestra.run.smithi002.stdout:18.0.0-5703.gba6b39e4.el8 2023-08-25T03:33:01.134 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5703.gba6b39e4.el8 2023-08-25T03:33:01.134 INFO:teuthology.task.install:The correct ceph version 18.0.0-5703.gba6b39e4 is installed. 2023-08-25T03:33:01.135 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=ba6b39e4f4f77fb95c747099b53db0467d435f56 2023-08-25T03:33:01.361 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-08-25T03:33:01.390 INFO:teuthology.orchestra.run.smithi031.stdout:18.0.0-5703.gba6b39e4.el8 2023-08-25T03:33:01.390 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-5703.gba6b39e4.el8 2023-08-25T03:33:01.391 INFO:teuthology.task.install:The correct ceph version 18.0.0-5703.gba6b39e4 is installed. 2023-08-25T03:33:01.391 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-08-25T03:33:01.392 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:33:01.392 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-08-25T03:33:01.436 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:33:01.436 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-08-25T03:33:01.485 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-08-25T03:33:01.486 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:33:01.486 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/daemon-helper 2023-08-25T03:33:01.514 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-08-25T03:33:01.582 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:33:01.582 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/daemon-helper 2023-08-25T03:33:01.613 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-08-25T03:33:01.683 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-08-25T03:33:01.684 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:33:01.684 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/adjust-ulimits 2023-08-25T03:33:01.712 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-08-25T03:33:01.779 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:33:01.780 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/adjust-ulimits 2023-08-25T03:33:01.809 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-08-25T03:33:01.879 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-08-25T03:33:01.880 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:33:01.880 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/usr/bin/stdin-killer 2023-08-25T03:33:01.909 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-08-25T03:33:01.976 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:33:01.976 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/stdin-killer 2023-08-25T03:33:02.006 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-08-25T03:33:02.075 INFO:teuthology.run_tasks:Running task ceph... 2023-08-25T03:33:02.165 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-08-25T03:33:02.166 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 777 /var/log/ceph 2023-08-25T03:33:02.168 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 777 /var/log/ceph 2023-08-25T03:33:02.197 INFO:tasks.ceph:Disabling ceph logrotate... 2023-08-25T03:33:02.198 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /etc/logrotate.d/ceph 2023-08-25T03:33:02.237 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/logrotate.d/ceph 2023-08-25T03:33:02.266 INFO:tasks.ceph:Creating extra log directories... 2023-08-25T03:33:02.267 DEBUG:teuthology.orchestra.run.smithi002:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-08-25T03:33:02.305 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-08-25T03:33:02.342 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-08-25T03:33:02.343 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-08-25T03:33:02.343 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-08-25T03:33:02.344 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:33:02.344 DEBUG:teuthology.orchestra.run.smithi002:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-08-25T03:33:02.388 DEBUG:teuthology.orchestra.run.smithi002:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-08-25T03:33:02.518 DEBUG:teuthology.orchestra.run.smithi002:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-08-25T03:33:02.551 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:33:02.551 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-08-25T03:33:02.571 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-08-25T03:33:02.709 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-08-25T03:33:02.744 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-08-25T03:33:02.744 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-08-25T03:33:02.745 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379011', 'branch': 'wip-rishabh-2023aug3-b5', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota}', 'email': None, 'first_in_suite': False, 'job_id': '7379011', 'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'a6a69670761025e039dc5ae86619b0cf04c46dc2'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-2023aug3-b5'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56'}}, 'kclient': {'syntax': 'v1'}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-2023aug3-b5', 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 80, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-rishabh-2023aug3-b5', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_ba6b39e4f4f77fb95c747099b53db0467d435f56/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi002.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac=', 'smithi031.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': 'a6a69670761025e039dc5ae86619b0cf04c46dc2'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'ba6b39e4f4f77fb95c747099b53db0467d435f56'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_quota']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '449a1bc2027504e7b3c3d7b30fa4178906581da7', 'timestamp': '2023-08-25_01:50:32', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2224'} 2023-08-25T03:33:02.746 DEBUG:teuthology.orchestra.run.smithi002:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-08-25T03:33:02.748 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-08-25T03:33:02.797 DEBUG:teuthology.orchestra.run.smithi002:> sudo install -d -m0777 -- /var/run/ceph 2023-08-25T03:33:02.805 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/run/ceph 2023-08-25T03:33:02.866 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:33:02.867 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/scratch_devs of=/dev/stdout 2023-08-25T03:33:02.888 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-08-25T03:33:02.889 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_1 2023-08-25T03:33:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-08-25T03:33:02.946 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T03:33:02.946 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 6h/6d Inode: 387693 Links: 1 2023-08-25T03:33:02.946 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T03:33:02.947 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T03:33:02.947 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2023-08-25 03:32:54.683215413 +0000 2023-08-25T03:33:02.947 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2023-08-25 03:29:54.103074600 +0000 2023-08-25T03:33:02.947 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2023-08-25 03:29:54.103074600 +0000 2023-08-25T03:33:02.947 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: - 2023-08-25T03:33:02.947 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-08-25T03:33:03.012 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2023-08-25T03:33:03.013 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2023-08-25T03:33:03.013 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000105941 s, 4.8 MB/s 2023-08-25T03:33:03.014 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-08-25T03:33:03.071 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_2 2023-08-25T03:33:03.129 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-08-25T03:33:03.129 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T03:33:03.129 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 6h/6d Inode: 390680 Links: 1 2023-08-25T03:33:03.129 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T03:33:03.129 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T03:33:03.130 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2023-08-25 03:32:54.683215413 +0000 2023-08-25T03:33:03.130 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2023-08-25 03:29:54.648061757 +0000 2023-08-25T03:33:03.130 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2023-08-25 03:29:54.648061757 +0000 2023-08-25T03:33:03.130 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: - 2023-08-25T03:33:03.130 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-08-25T03:33:03.194 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2023-08-25T03:33:03.194 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2023-08-25T03:33:03.194 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000104378 s, 4.9 MB/s 2023-08-25T03:33:03.195 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-08-25T03:33:03.253 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_3 2023-08-25T03:33:03.311 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-08-25T03:33:03.311 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T03:33:03.312 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 6h/6d Inode: 387924 Links: 1 2023-08-25T03:33:03.312 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T03:33:03.312 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T03:33:03.312 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2023-08-25 03:32:54.683215413 +0000 2023-08-25T03:33:03.312 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2023-08-25 03:29:55.168049503 +0000 2023-08-25T03:33:03.313 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2023-08-25 03:29:55.168049503 +0000 2023-08-25T03:33:03.313 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: - 2023-08-25T03:33:03.313 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-08-25T03:33:03.377 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2023-08-25T03:33:03.377 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2023-08-25T03:33:03.377 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000109479 s, 4.7 MB/s 2023-08-25T03:33:03.379 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-08-25T03:33:03.437 DEBUG:teuthology.orchestra.run.smithi002:> stat /dev/vg_nvme/lv_4 2023-08-25T03:33:03.494 INFO:teuthology.orchestra.run.smithi002.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-08-25T03:33:03.494 INFO:teuthology.orchestra.run.smithi002.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T03:33:03.494 INFO:teuthology.orchestra.run.smithi002.stdout:Device: 6h/6d Inode: 384627 Links: 1 2023-08-25T03:33:03.494 INFO:teuthology.orchestra.run.smithi002.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T03:33:03.494 INFO:teuthology.orchestra.run.smithi002.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T03:33:03.495 INFO:teuthology.orchestra.run.smithi002.stdout:Access: 2023-08-25 03:32:54.683215413 +0000 2023-08-25T03:33:03.495 INFO:teuthology.orchestra.run.smithi002.stdout:Modify: 2023-08-25 03:29:55.690037202 +0000 2023-08-25T03:33:03.495 INFO:teuthology.orchestra.run.smithi002.stdout:Change: 2023-08-25 03:29:55.690037202 +0000 2023-08-25T03:33:03.495 INFO:teuthology.orchestra.run.smithi002.stdout: Birth: - 2023-08-25T03:33:03.495 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-08-25T03:33:03.560 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records in 2023-08-25T03:33:03.561 INFO:teuthology.orchestra.run.smithi002.stderr:1+0 records out 2023-08-25T03:33:03.561 INFO:teuthology.orchestra.run.smithi002.stderr:512 bytes copied, 0.000107669 s, 4.8 MB/s 2023-08-25T03:33:03.561 DEBUG:teuthology.orchestra.run.smithi002:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-08-25T03:33:03.618 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-25T03:33:03.619 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:33:03.619 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/scratch_devs of=/dev/stdout 2023-08-25T03:33:03.640 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-08-25T03:33:03.641 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_1 2023-08-25T03:33:03.701 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-08-25T03:33:03.702 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T03:33:03.702 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 6h/6d Inode: 383767 Links: 1 2023-08-25T03:33:03.702 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T03:33:03.703 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T03:33:03.703 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2023-08-25 03:32:52.821238315 +0000 2023-08-25T03:33:03.703 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2023-08-25 03:29:51.700595812 +0000 2023-08-25T03:33:03.703 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2023-08-25 03:29:51.700595812 +0000 2023-08-25T03:33:03.704 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: - 2023-08-25T03:33:03.704 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-08-25T03:33:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-08-25T03:33:03.772 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-08-25T03:33:03.773 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000264869 s, 1.9 MB/s 2023-08-25T03:33:03.773 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-08-25T03:33:03.834 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_2 2023-08-25T03:33:03.893 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-08-25T03:33:03.893 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T03:33:03.893 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 6h/6d Inode: 388487 Links: 1 2023-08-25T03:33:03.894 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T03:33:03.894 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T03:33:03.894 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2023-08-25 03:32:52.821238315 +0000 2023-08-25T03:33:03.894 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2023-08-25 03:29:52.232587509 +0000 2023-08-25T03:33:03.894 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2023-08-25 03:29:52.232587509 +0000 2023-08-25T03:33:03.894 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: - 2023-08-25T03:33:03.894 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-08-25T03:33:03.961 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-08-25T03:33:03.961 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-08-25T03:33:03.961 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000210894 s, 2.4 MB/s 2023-08-25T03:33:03.962 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-08-25T03:33:04.021 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_3 2023-08-25T03:33:04.081 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-08-25T03:33:04.081 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T03:33:04.081 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 6h/6d Inode: 391274 Links: 1 2023-08-25T03:33:04.082 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T03:33:04.082 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T03:33:04.082 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2023-08-25 03:32:52.821238315 +0000 2023-08-25T03:33:04.082 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2023-08-25 03:29:52.778578988 +0000 2023-08-25T03:33:04.082 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2023-08-25 03:29:52.778578988 +0000 2023-08-25T03:33:04.082 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: - 2023-08-25T03:33:04.083 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-08-25T03:33:04.149 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-08-25T03:33:04.149 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-08-25T03:33:04.149 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000268192 s, 1.9 MB/s 2023-08-25T03:33:04.150 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-08-25T03:33:04.209 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_4 2023-08-25T03:33:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-08-25T03:33:04.267 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-08-25T03:33:04.267 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 6h/6d Inode: 392266 Links: 1 2023-08-25T03:33:04.267 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-08-25T03:33:04.268 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2023-08-25T03:33:04.268 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2023-08-25 03:32:52.821238315 +0000 2023-08-25T03:33:04.268 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2023-08-25 03:29:53.316570592 +0000 2023-08-25T03:33:04.268 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2023-08-25 03:29:53.316570592 +0000 2023-08-25T03:33:04.268 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: - 2023-08-25T03:33:04.269 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-08-25T03:33:04.333 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2023-08-25T03:33:04.334 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2023-08-25T03:33:04.334 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000305155 s, 1.7 MB/s 2023-08-25T03:33:04.335 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-08-25T03:33:04.393 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-25T03:33:04.394 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi002.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-08-25T03:33:04.394 INFO:tasks.ceph:Generating config... 2023-08-25T03:33:04.395 INFO:tasks.ceph:[client] client mount timeout = 600 2023-08-25T03:33:04.396 INFO:tasks.ceph:[client] debug client = 20 2023-08-25T03:33:04.396 INFO:tasks.ceph:[client] debug ms = 1 2023-08-25T03:33:04.396 INFO:tasks.ceph:[client] pid file = 2023-08-25T03:33:04.396 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-08-25T03:33:04.397 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-08-25T03:33:04.397 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-08-25T03:33:04.397 INFO:tasks.ceph:[mds] debug mds = 20 2023-08-25T03:33:04.397 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-08-25T03:33:04.398 INFO:tasks.ceph:[mds] debug ms = 1 2023-08-25T03:33:04.398 INFO:tasks.ceph:[mds] mds debug frag = True 2023-08-25T03:33:04.398 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-08-25T03:33:04.398 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-08-25T03:33:04.398 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-08-25T03:33:04.399 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-08-25T03:33:04.399 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-08-25T03:33:04.399 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-08-25T03:33:04.399 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-08-25T03:33:04.399 INFO:tasks.ceph:[mgr] debug ms = 1 2023-08-25T03:33:04.400 INFO:tasks.ceph:[mon] debug mon = 20 2023-08-25T03:33:04.400 INFO:tasks.ceph:[mon] debug ms = 1 2023-08-25T03:33:04.400 INFO:tasks.ceph:[mon] debug paxos = 20 2023-08-25T03:33:04.400 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-08-25T03:33:04.401 INFO:tasks.ceph:[osd] bdev async discard = True 2023-08-25T03:33:04.401 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-08-25T03:33:04.401 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-08-25T03:33:04.401 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-08-25T03:33:04.401 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-08-25T03:33:04.402 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-08-25T03:33:04.402 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-08-25T03:33:04.402 INFO:tasks.ceph:[osd] debug ms = 1 2023-08-25T03:33:04.402 INFO:tasks.ceph:[osd] debug osd = 20 2023-08-25T03:33:04.402 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-08-25T03:33:04.402 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-08-25T03:33:04.403 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-08-25T03:33:04.403 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-08-25T03:33:04.403 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-08-25T03:33:04.403 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-08-25T03:33:04.403 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-08-25T03:33:04.404 INFO:tasks.ceph:Setting up mon.a... 2023-08-25T03:33:04.404 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-08-25T03:33:04.480 INFO:teuthology.orchestra.run.smithi002.stdout:creating /etc/ceph/ceph.keyring 2023-08-25T03:33:04.483 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-08-25T03:33:04.534 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-08-25T03:33:04.563 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.2'), ('mon.b', '172.21.15.31'), ('mon.c', '[v2:172.21.15.31:3301,v1:172.21.15.31:6790]')] 2023-08-25T03:33:04.564 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.2,172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-08-25T03:33:04.565 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:33:04.565 DEBUG:teuthology.orchestra.run.smithi002:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-08-25T03:33:04.622 DEBUG:teuthology.orchestra.run.smithi002:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.2 --add b 172.21.15.31 --addv c '[v2:172.21.15.31:3301,v1:172.21.15.31:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-08-25T03:33:04.713 INFO:teuthology.orchestra.run.smithi002.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-08-25T03:33:04.714 INFO:teuthology.orchestra.run.smithi002.stdout:monmaptool: generated fsid dc54136d-3683-431b-8a68-994ec618505c 2023-08-25T03:33:04.714 INFO:teuthology.orchestra.run.smithi002.stdout:setting min_mon_release = pacific 2023-08-25T03:33:04.714 INFO:teuthology.orchestra.run.smithi002.stdout:epoch 0 2023-08-25T03:33:04.714 INFO:teuthology.orchestra.run.smithi002.stdout:fsid dc54136d-3683-431b-8a68-994ec618505c 2023-08-25T03:33:04.714 INFO:teuthology.orchestra.run.smithi002.stdout:last_changed 2023-08-25T03:33:04.713461+0000 2023-08-25T03:33:04.714 INFO:teuthology.orchestra.run.smithi002.stdout:created 2023-08-25T03:33:04.713461+0000 2023-08-25T03:33:04.714 INFO:teuthology.orchestra.run.smithi002.stdout:min_mon_release 16 (pacific) 2023-08-25T03:33:04.715 INFO:teuthology.orchestra.run.smithi002.stdout:election_strategy: 1 2023-08-25T03:33:04.715 INFO:teuthology.orchestra.run.smithi002.stdout:0: [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] mon.a 2023-08-25T03:33:04.715 INFO:teuthology.orchestra.run.smithi002.stdout:1: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.b 2023-08-25T03:33:04.715 INFO:teuthology.orchestra.run.smithi002.stdout:2: [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] mon.c 2023-08-25T03:33:04.715 INFO:teuthology.orchestra.run.smithi002.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-08-25T03:33:04.716 DEBUG:teuthology.orchestra.run.smithi002:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-08-25T03:33:04.733 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID dc54136d-3683-431b-8a68-994ec618505c... 2023-08-25T03:33:04.735 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-08-25T03:33:04.776 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-08-25T03:33:04.821 INFO:teuthology.orchestra.run.smithi002.stdout:[global] 2023-08-25T03:33:04.822 INFO:teuthology.orchestra.run.smithi002.stdout: chdir = "" 2023-08-25T03:33:04.822 INFO:teuthology.orchestra.run.smithi002.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-08-25T03:33:04.823 INFO:teuthology.orchestra.run.smithi002.stdout: auth supported = cephx 2023-08-25T03:33:04.823 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.823 INFO:teuthology.orchestra.run.smithi002.stdout: filestore xattr use omap = true 2023-08-25T03:33:04.823 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.823 INFO:teuthology.orchestra.run.smithi002.stdout: mon clock drift allowed = 1.000 2023-08-25T03:33:04.824 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.824 INFO:teuthology.orchestra.run.smithi002.stdout: osd crush chooseleaf type = 0 2023-08-25T03:33:04.824 INFO:teuthology.orchestra.run.smithi002.stdout: auth debug = true 2023-08-25T03:33:04.824 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.824 INFO:teuthology.orchestra.run.smithi002.stdout: ms die on old message = true 2023-08-25T03:33:04.824 INFO:teuthology.orchestra.run.smithi002.stdout: ms die on bug = true 2023-08-25T03:33:04.824 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.825 INFO:teuthology.orchestra.run.smithi002.stdout: mon max pg per osd = 10000 # >= luminous 2023-08-25T03:33:04.825 INFO:teuthology.orchestra.run.smithi002.stdout: mon pg warn max object skew = 0 2023-08-25T03:33:04.825 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.825 INFO:teuthology.orchestra.run.smithi002.stdout: # disable pg_autoscaler by default for new pools 2023-08-25T03:33:04.825 INFO:teuthology.orchestra.run.smithi002.stdout: osd_pool_default_pg_autoscale_mode = off 2023-08-25T03:33:04.825 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.825 INFO:teuthology.orchestra.run.smithi002.stdout: osd pool default size = 2 2023-08-25T03:33:04.826 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.826 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd allow primary affinity = true 2023-08-25T03:33:04.826 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd allow pg remap = true 2023-08-25T03:33:04.826 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on legacy crush tunables = false 2023-08-25T03:33:04.826 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on crush straw calc version zero = false 2023-08-25T03:33:04.827 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on no sortbitwise = false 2023-08-25T03:33:04.827 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on osd down out interval zero = false 2023-08-25T03:33:04.827 INFO:teuthology.orchestra.run.smithi002.stdout: mon warn on too few osds = false 2023-08-25T03:33:04.828 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-08-25T03:33:04.829 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_pool_no_redundancy = false 2023-08-25T03:33:04.829 INFO:teuthology.orchestra.run.smithi002.stdout: mon_allow_pool_size_one = true 2023-08-25T03:33:04.829 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.829 INFO:teuthology.orchestra.run.smithi002.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-08-25T03:33:04.829 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.830 INFO:teuthology.orchestra.run.smithi002.stdout: osd default data pool replay window = 5 2023-08-25T03:33:04.830 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.830 INFO:teuthology.orchestra.run.smithi002.stdout: mon allow pool delete = true 2023-08-25T03:33:04.830 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.830 INFO:teuthology.orchestra.run.smithi002.stdout: mon cluster log file level = debug 2023-08-25T03:33:04.831 INFO:teuthology.orchestra.run.smithi002.stdout: debug asserts on shutdown = true 2023-08-25T03:33:04.831 INFO:teuthology.orchestra.run.smithi002.stdout: mon health detail to clog = false 2023-08-25T03:33:04.832 INFO:teuthology.orchestra.run.smithi002.stdout: mon host = "172.21.15.2,172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790]" 2023-08-25T03:33:04.832 INFO:teuthology.orchestra.run.smithi002.stdout: ms die on skipped message = False 2023-08-25T03:33:04.832 INFO:teuthology.orchestra.run.smithi002.stdout: fsid = dc54136d-3683-431b-8a68-994ec618505c 2023-08-25T03:33:04.832 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.832 INFO:teuthology.orchestra.run.smithi002.stdout:[osd] 2023-08-25T03:33:04.832 INFO:teuthology.orchestra.run.smithi002.stdout: osd journal size = 100 2023-08-25T03:33:04.833 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.833 INFO:teuthology.orchestra.run.smithi002.stdout: osd scrub load threshold = 5.0 2023-08-25T03:33:04.833 INFO:teuthology.orchestra.run.smithi002.stdout: osd scrub max interval = 600 2023-08-25T03:33:04.833 INFO:teuthology.orchestra.run.smithi002.stdout: osd mclock profile = high_recovery_ops 2023-08-25T03:33:04.833 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.833 INFO:teuthology.orchestra.run.smithi002.stdout: osd recover clone overlap = true 2023-08-25T03:33:04.833 INFO:teuthology.orchestra.run.smithi002.stdout: osd recovery max chunk = 1048576 2023-08-25T03:33:04.834 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.834 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug shutdown = true 2023-08-25T03:33:04.834 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug op order = true 2023-08-25T03:33:04.834 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug verify stray on activate = true 2023-08-25T03:33:04.834 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.834 INFO:teuthology.orchestra.run.smithi002.stdout: osd open classes on start = true 2023-08-25T03:33:04.834 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug pg log writeout = true 2023-08-25T03:33:04.834 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.835 INFO:teuthology.orchestra.run.smithi002.stdout: osd deep scrub update digest min age = 30 2023-08-25T03:33:04.835 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.835 INFO:teuthology.orchestra.run.smithi002.stdout: osd map max advance = 10 2023-08-25T03:33:04.835 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.835 INFO:teuthology.orchestra.run.smithi002.stdout: journal zero on create = true 2023-08-25T03:33:04.835 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.835 INFO:teuthology.orchestra.run.smithi002.stdout: filestore ondisk finisher threads = 3 2023-08-25T03:33:04.835 INFO:teuthology.orchestra.run.smithi002.stdout: filestore apply finisher threads = 3 2023-08-25T03:33:04.836 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.836 INFO:teuthology.orchestra.run.smithi002.stdout: bdev debug aio = true 2023-08-25T03:33:04.836 INFO:teuthology.orchestra.run.smithi002.stdout: osd debug misdirected ops = true 2023-08-25T03:33:04.836 INFO:teuthology.orchestra.run.smithi002.stdout: bdev async discard = True 2023-08-25T03:33:04.836 INFO:teuthology.orchestra.run.smithi002.stdout: bdev enable discard = True 2023-08-25T03:33:04.836 INFO:teuthology.orchestra.run.smithi002.stdout: bluestore allocator = bitmap 2023-08-25T03:33:04.836 INFO:teuthology.orchestra.run.smithi002.stdout: bluestore block size = 96636764160 2023-08-25T03:33:04.837 INFO:teuthology.orchestra.run.smithi002.stdout: bluestore fsck on mount = True 2023-08-25T03:33:04.837 INFO:teuthology.orchestra.run.smithi002.stdout: debug bluefs = 1/20 2023-08-25T03:33:04.837 INFO:teuthology.orchestra.run.smithi002.stdout: debug bluestore = 1/20 2023-08-25T03:33:04.837 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2023-08-25T03:33:04.837 INFO:teuthology.orchestra.run.smithi002.stdout: debug osd = 20 2023-08-25T03:33:04.837 INFO:teuthology.orchestra.run.smithi002.stdout: debug rocksdb = 4/10 2023-08-25T03:33:04.837 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd backfillfull_ratio = 0.85 2023-08-25T03:33:04.837 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd full ratio = 0.9 2023-08-25T03:33:04.838 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd nearfull ratio = 0.8 2023-08-25T03:33:04.838 INFO:teuthology.orchestra.run.smithi002.stdout: osd failsafe full ratio = 0.95 2023-08-25T03:33:04.838 INFO:teuthology.orchestra.run.smithi002.stdout: osd objectstore = bluestore 2023-08-25T03:33:04.838 INFO:teuthology.orchestra.run.smithi002.stdout: osd op complaint time = 180 2023-08-25T03:33:04.838 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.838 INFO:teuthology.orchestra.run.smithi002.stdout:[mgr] 2023-08-25T03:33:04.838 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2023-08-25T03:33:04.839 INFO:teuthology.orchestra.run.smithi002.stdout: debug mgr = 20 2023-08-25T03:33:04.839 INFO:teuthology.orchestra.run.smithi002.stdout: debug mon = 20 2023-08-25T03:33:04.839 INFO:teuthology.orchestra.run.smithi002.stdout: debug auth = 20 2023-08-25T03:33:04.839 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min pgs per osd = 4 2023-08-25T03:33:04.839 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min bytes per osd = 10 2023-08-25T03:33:04.839 INFO:teuthology.orchestra.run.smithi002.stdout: mgr/telemetry/nag = false 2023-08-25T03:33:04.839 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.839 INFO:teuthology.orchestra.run.smithi002.stdout:[mon] 2023-08-25T03:33:04.840 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2023-08-25T03:33:04.840 INFO:teuthology.orchestra.run.smithi002.stdout: debug mon = 20 2023-08-25T03:33:04.840 INFO:teuthology.orchestra.run.smithi002.stdout: debug paxos = 20 2023-08-25T03:33:04.840 INFO:teuthology.orchestra.run.smithi002.stdout: debug auth = 20 2023-08-25T03:33:04.840 INFO:teuthology.orchestra.run.smithi002.stdout: mon data avail warn = 5 2023-08-25T03:33:04.840 INFO:teuthology.orchestra.run.smithi002.stdout: mon mgr mkfs grace = 240 2023-08-25T03:33:04.840 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min pgs per osd = 4 2023-08-25T03:33:04.840 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd reporter subtree level = osd 2023-08-25T03:33:04.841 INFO:teuthology.orchestra.run.smithi002.stdout: mon osd prime pg temp = true 2023-08-25T03:33:04.841 INFO:teuthology.orchestra.run.smithi002.stdout: mon reweight min bytes per osd = 10 2023-08-25T03:33:04.841 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.841 INFO:teuthology.orchestra.run.smithi002.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-08-25T03:33:04.841 INFO:teuthology.orchestra.run.smithi002.stdout: auth mon ticket ttl = 660 # 11m 2023-08-25T03:33:04.841 INFO:teuthology.orchestra.run.smithi002.stdout: auth service ticket ttl = 240 # 4m 2023-08-25T03:33:04.841 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.842 INFO:teuthology.orchestra.run.smithi002.stdout: # don't complain about insecure global_id in the test suite 2023-08-25T03:33:04.842 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-08-25T03:33:04.842 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-08-25T03:33:04.842 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.842 INFO:teuthology.orchestra.run.smithi002.stdout: # 1m isn't quite enough 2023-08-25T03:33:04.842 INFO:teuthology.orchestra.run.smithi002.stdout: mon_down_mkfs_grace = 2m 2023-08-25T03:33:04.842 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.842 INFO:teuthology.orchestra.run.smithi002.stdout: mon_warn_on_filestore_osds = false 2023-08-25T03:33:04.843 INFO:teuthology.orchestra.run.smithi002.stdout: mon op complaint time = 120 2023-08-25T03:33:04.843 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:04.843 INFO:teuthology.orchestra.run.smithi002.stdout:[client] 2023-08-25T03:33:04.843 INFO:teuthology.orchestra.run.smithi002.stdout: rgw cache enabled = true 2023-08-25T03:33:04.843 INFO:teuthology.orchestra.run.smithi002.stdout: rgw enable ops log = true 2023-08-25T03:33:04.843 INFO:teuthology.orchestra.run.smithi002.stdout: rgw enable usage log = true 2023-08-25T03:33:04.843 INFO:teuthology.orchestra.run.smithi002.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-08-25T03:33:04.843 INFO:teuthology.orchestra.run.smithi002.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-08-25T03:33:04.844 INFO:teuthology.orchestra.run.smithi002.stdout: client mount timeout = 600 2023-08-25T03:33:04.844 INFO:teuthology.orchestra.run.smithi002.stdout: debug client = 20 2023-08-25T03:33:04.844 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2023-08-25T03:33:04.844 INFO:teuthology.orchestra.run.smithi002.stdout: pid file = "" 2023-08-25T03:33:04.844 INFO:teuthology.orchestra.run.smithi002.stdout: rados mon op timeout = 900 2023-08-25T03:33:04.844 INFO:teuthology.orchestra.run.smithi002.stdout: rados osd op timeout = 900 2023-08-25T03:33:04.844 INFO:teuthology.orchestra.run.smithi002.stdout:[mon.a] 2023-08-25T03:33:04.845 INFO:teuthology.orchestra.run.smithi002.stdout:[mon.b] 2023-08-25T03:33:04.845 INFO:teuthology.orchestra.run.smithi002.stdout:[mon.c] 2023-08-25T03:33:04.845 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.a] 2023-08-25T03:33:04.845 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.c] 2023-08-25T03:33:04.845 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.b] 2023-08-25T03:33:04.845 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.d] 2023-08-25T03:33:04.845 INFO:teuthology.orchestra.run.smithi002.stdout:[mds] 2023-08-25T03:33:04.845 INFO:teuthology.orchestra.run.smithi002.stdout: debug mds = 20 2023-08-25T03:33:04.846 INFO:teuthology.orchestra.run.smithi002.stdout: debug mds balancer = 20 2023-08-25T03:33:04.846 INFO:teuthology.orchestra.run.smithi002.stdout: debug ms = 1 2023-08-25T03:33:04.846 INFO:teuthology.orchestra.run.smithi002.stdout: mds debug frag = True 2023-08-25T03:33:04.846 INFO:teuthology.orchestra.run.smithi002.stdout: mds debug scatterstat = True 2023-08-25T03:33:04.846 INFO:teuthology.orchestra.run.smithi002.stdout: mds op complaint time = 180 2023-08-25T03:33:04.846 INFO:teuthology.orchestra.run.smithi002.stdout: mds verify scatter = True 2023-08-25T03:33:04.846 INFO:teuthology.orchestra.run.smithi002.stdout: osd op complaint time = 180 2023-08-25T03:33:04.847 INFO:teuthology.orchestra.run.smithi002.stdout: rados mon op timeout = 900 2023-08-25T03:33:04.847 INFO:teuthology.orchestra.run.smithi002.stdout: rados osd op timeout = 900 2023-08-25T03:33:04.848 INFO:teuthology.orchestra.run.smithi031.stdout:[global] 2023-08-25T03:33:04.849 INFO:teuthology.orchestra.run.smithi031.stdout: chdir = "" 2023-08-25T03:33:04.849 INFO:teuthology.orchestra.run.smithi031.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-08-25T03:33:04.849 INFO:teuthology.orchestra.run.smithi031.stdout: auth supported = cephx 2023-08-25T03:33:04.849 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.849 INFO:teuthology.orchestra.run.smithi031.stdout: filestore xattr use omap = true 2023-08-25T03:33:04.849 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.849 INFO:teuthology.orchestra.run.smithi031.stdout: mon clock drift allowed = 1.000 2023-08-25T03:33:04.850 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.850 INFO:teuthology.orchestra.run.smithi031.stdout: osd crush chooseleaf type = 0 2023-08-25T03:33:04.850 INFO:teuthology.orchestra.run.smithi031.stdout: auth debug = true 2023-08-25T03:33:04.850 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.850 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on old message = true 2023-08-25T03:33:04.850 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on bug = true 2023-08-25T03:33:04.851 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.851 INFO:teuthology.orchestra.run.smithi031.stdout: mon max pg per osd = 10000 # >= luminous 2023-08-25T03:33:04.851 INFO:teuthology.orchestra.run.smithi031.stdout: mon pg warn max object skew = 0 2023-08-25T03:33:04.851 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.851 INFO:teuthology.orchestra.run.smithi031.stdout: # disable pg_autoscaler by default for new pools 2023-08-25T03:33:04.851 INFO:teuthology.orchestra.run.smithi031.stdout: osd_pool_default_pg_autoscale_mode = off 2023-08-25T03:33:04.851 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.851 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default size = 2 2023-08-25T03:33:04.852 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.852 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow primary affinity = true 2023-08-25T03:33:04.852 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow pg remap = true 2023-08-25T03:33:04.852 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on legacy crush tunables = false 2023-08-25T03:33:04.852 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on crush straw calc version zero = false 2023-08-25T03:33:04.852 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on no sortbitwise = false 2023-08-25T03:33:04.852 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on osd down out interval zero = false 2023-08-25T03:33:04.852 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on too few osds = false 2023-08-25T03:33:04.853 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-08-25T03:33:04.853 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_no_redundancy = false 2023-08-25T03:33:04.853 INFO:teuthology.orchestra.run.smithi031.stdout: mon_allow_pool_size_one = true 2023-08-25T03:33:04.853 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.853 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-08-25T03:33:04.853 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.853 INFO:teuthology.orchestra.run.smithi031.stdout: osd default data pool replay window = 5 2023-08-25T03:33:04.854 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.854 INFO:teuthology.orchestra.run.smithi031.stdout: mon allow pool delete = true 2023-08-25T03:33:04.854 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.854 INFO:teuthology.orchestra.run.smithi031.stdout: mon cluster log file level = debug 2023-08-25T03:33:04.854 INFO:teuthology.orchestra.run.smithi031.stdout: debug asserts on shutdown = true 2023-08-25T03:33:04.854 INFO:teuthology.orchestra.run.smithi031.stdout: mon health detail to clog = false 2023-08-25T03:33:04.854 INFO:teuthology.orchestra.run.smithi031.stdout: mon host = "172.21.15.2,172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790]" 2023-08-25T03:33:04.854 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on skipped message = False 2023-08-25T03:33:04.855 INFO:teuthology.orchestra.run.smithi031.stdout: fsid = dc54136d-3683-431b-8a68-994ec618505c 2023-08-25T03:33:04.855 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.855 INFO:teuthology.orchestra.run.smithi031.stdout:[osd] 2023-08-25T03:33:04.855 INFO:teuthology.orchestra.run.smithi031.stdout: osd journal size = 100 2023-08-25T03:33:04.855 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.855 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub load threshold = 5.0 2023-08-25T03:33:04.855 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub max interval = 600 2023-08-25T03:33:04.855 INFO:teuthology.orchestra.run.smithi031.stdout: osd mclock profile = high_recovery_ops 2023-08-25T03:33:04.856 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.856 INFO:teuthology.orchestra.run.smithi031.stdout: osd recover clone overlap = true 2023-08-25T03:33:04.856 INFO:teuthology.orchestra.run.smithi031.stdout: osd recovery max chunk = 1048576 2023-08-25T03:33:04.856 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.856 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug shutdown = true 2023-08-25T03:33:04.856 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug op order = true 2023-08-25T03:33:04.856 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug verify stray on activate = true 2023-08-25T03:33:04.856 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.857 INFO:teuthology.orchestra.run.smithi031.stdout: osd open classes on start = true 2023-08-25T03:33:04.857 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug pg log writeout = true 2023-08-25T03:33:04.857 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.857 INFO:teuthology.orchestra.run.smithi031.stdout: osd deep scrub update digest min age = 30 2023-08-25T03:33:04.857 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.857 INFO:teuthology.orchestra.run.smithi031.stdout: osd map max advance = 10 2023-08-25T03:33:04.857 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.857 INFO:teuthology.orchestra.run.smithi031.stdout: journal zero on create = true 2023-08-25T03:33:04.858 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.858 INFO:teuthology.orchestra.run.smithi031.stdout: filestore ondisk finisher threads = 3 2023-08-25T03:33:04.858 INFO:teuthology.orchestra.run.smithi031.stdout: filestore apply finisher threads = 3 2023-08-25T03:33:04.858 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.858 INFO:teuthology.orchestra.run.smithi031.stdout: bdev debug aio = true 2023-08-25T03:33:04.858 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug misdirected ops = true 2023-08-25T03:33:04.858 INFO:teuthology.orchestra.run.smithi031.stdout: bdev async discard = True 2023-08-25T03:33:04.858 INFO:teuthology.orchestra.run.smithi031.stdout: bdev enable discard = True 2023-08-25T03:33:04.859 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore allocator = bitmap 2023-08-25T03:33:04.859 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore block size = 96636764160 2023-08-25T03:33:04.859 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore fsck on mount = True 2023-08-25T03:33:04.859 INFO:teuthology.orchestra.run.smithi031.stdout: debug bluefs = 1/20 2023-08-25T03:33:04.859 INFO:teuthology.orchestra.run.smithi031.stdout: debug bluestore = 1/20 2023-08-25T03:33:04.859 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2023-08-25T03:33:04.859 INFO:teuthology.orchestra.run.smithi031.stdout: debug osd = 20 2023-08-25T03:33:04.859 INFO:teuthology.orchestra.run.smithi031.stdout: debug rocksdb = 4/10 2023-08-25T03:33:04.860 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd backfillfull_ratio = 0.85 2023-08-25T03:33:04.860 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd full ratio = 0.9 2023-08-25T03:33:04.860 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd nearfull ratio = 0.8 2023-08-25T03:33:04.860 INFO:teuthology.orchestra.run.smithi031.stdout: osd failsafe full ratio = 0.95 2023-08-25T03:33:04.860 INFO:teuthology.orchestra.run.smithi031.stdout: osd objectstore = bluestore 2023-08-25T03:33:04.860 INFO:teuthology.orchestra.run.smithi031.stdout: osd op complaint time = 180 2023-08-25T03:33:04.860 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.861 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr] 2023-08-25T03:33:04.861 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2023-08-25T03:33:04.861 INFO:teuthology.orchestra.run.smithi031.stdout: debug mgr = 20 2023-08-25T03:33:04.861 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2023-08-25T03:33:04.861 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2023-08-25T03:33:04.861 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2023-08-25T03:33:04.861 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2023-08-25T03:33:04.861 INFO:teuthology.orchestra.run.smithi031.stdout: mgr/telemetry/nag = false 2023-08-25T03:33:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.862 INFO:teuthology.orchestra.run.smithi031.stdout:[mon] 2023-08-25T03:33:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2023-08-25T03:33:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2023-08-25T03:33:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: debug paxos = 20 2023-08-25T03:33:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2023-08-25T03:33:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: mon data avail warn = 5 2023-08-25T03:33:04.862 INFO:teuthology.orchestra.run.smithi031.stdout: mon mgr mkfs grace = 240 2023-08-25T03:33:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2023-08-25T03:33:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd reporter subtree level = osd 2023-08-25T03:33:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd prime pg temp = true 2023-08-25T03:33:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2023-08-25T03:33:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-08-25T03:33:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: auth mon ticket ttl = 660 # 11m 2023-08-25T03:33:04.863 INFO:teuthology.orchestra.run.smithi031.stdout: auth service ticket ttl = 240 # 4m 2023-08-25T03:33:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: # don't complain about insecure global_id in the test suite 2023-08-25T03:33:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-08-25T03:33:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-08-25T03:33:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: # 1m isn't quite enough 2023-08-25T03:33:04.864 INFO:teuthology.orchestra.run.smithi031.stdout: mon_down_mkfs_grace = 2m 2023-08-25T03:33:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_filestore_osds = false 2023-08-25T03:33:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: mon op complaint time = 120 2023-08-25T03:33:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: 2023-08-25T03:33:04.865 INFO:teuthology.orchestra.run.smithi031.stdout:[client] 2023-08-25T03:33:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: rgw cache enabled = true 2023-08-25T03:33:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable ops log = true 2023-08-25T03:33:04.865 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable usage log = true 2023-08-25T03:33:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-08-25T03:33:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-08-25T03:33:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: client mount timeout = 600 2023-08-25T03:33:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: debug client = 20 2023-08-25T03:33:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2023-08-25T03:33:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: pid file = "" 2023-08-25T03:33:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: rados mon op timeout = 900 2023-08-25T03:33:04.866 INFO:teuthology.orchestra.run.smithi031.stdout: rados osd op timeout = 900 2023-08-25T03:33:04.867 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.a] 2023-08-25T03:33:04.867 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.b] 2023-08-25T03:33:04.867 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.c] 2023-08-25T03:33:04.867 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.a] 2023-08-25T03:33:04.867 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.c] 2023-08-25T03:33:04.867 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.b] 2023-08-25T03:33:04.867 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.d] 2023-08-25T03:33:04.867 INFO:teuthology.orchestra.run.smithi031.stdout:[mds] 2023-08-25T03:33:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: debug mds = 20 2023-08-25T03:33:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: debug mds balancer = 20 2023-08-25T03:33:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2023-08-25T03:33:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: mds debug frag = True 2023-08-25T03:33:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: mds debug scatterstat = True 2023-08-25T03:33:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: mds op complaint time = 180 2023-08-25T03:33:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: mds verify scatter = True 2023-08-25T03:33:04.868 INFO:teuthology.orchestra.run.smithi031.stdout: osd op complaint time = 180 2023-08-25T03:33:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: rados mon op timeout = 900 2023-08-25T03:33:04.869 INFO:teuthology.orchestra.run.smithi031.stdout: rados osd op timeout = 900 2023-08-25T03:33:04.869 INFO:tasks.ceph:Creating admin key on mon.a... 2023-08-25T03:33:04.869 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-08-25T03:33:04.947 INFO:tasks.ceph:Copying monmap to all nodes... 2023-08-25T03:33:04.947 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:33:04.947 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-08-25T03:33:04.964 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:33:04.965 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-08-25T03:33:05.020 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi002.front.sepia.ceph.com 2023-08-25T03:33:05.021 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:33:05.021 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd of=/etc/ceph/ceph.keyring 2023-08-25T03:33:05.021 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-08-25T03:33:05.098 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:33:05.098 DEBUG:teuthology.orchestra.run.smithi002:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-08-25T03:33:05.154 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi031.front.sepia.ceph.com 2023-08-25T03:33:05.154 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:33:05.155 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.keyring 2023-08-25T03:33:05.155 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-08-25T03:33:05.201 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:33:05.201 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-08-25T03:33:05.261 INFO:tasks.ceph:Setting up mon nodes... 2023-08-25T03:33:05.261 INFO:tasks.ceph:Setting up mgr nodes... 2023-08-25T03:33:05.261 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-08-25T03:33:05.317 INFO:teuthology.orchestra.run.smithi002.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-08-25T03:33:05.319 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-08-25T03:33:05.404 INFO:teuthology.orchestra.run.smithi031.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-08-25T03:33:05.408 INFO:tasks.ceph:Setting up mds nodes... 2023-08-25T03:33:05.408 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-08-25T03:33:05.464 INFO:teuthology.orchestra.run.smithi002.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-08-25T03:33:05.466 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-08-25T03:33:05.492 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-08-25T03:33:05.587 INFO:teuthology.orchestra.run.smithi002.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-08-25T03:33:05.590 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-08-25T03:33:05.617 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-08-25T03:33:05.680 INFO:teuthology.orchestra.run.smithi031.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-08-25T03:33:05.683 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-08-25T03:33:05.712 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-08-25T03:33:05.809 INFO:teuthology.orchestra.run.smithi031.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-08-25T03:33:05.811 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-08-25T03:33:05.838 INFO:tasks.ceph_client:Setting up client nodes... 2023-08-25T03:33:05.839 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-08-25T03:33:05.886 INFO:teuthology.orchestra.run.smithi002.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-08-25T03:33:05.899 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-08-25T03:33:05.943 INFO:teuthology.orchestra.run.smithi002.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-08-25T03:33:05.957 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-08-25T03:33:06.004 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-08-25T03:33:06.017 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-08-25T03:33:06.061 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-08-25T03:33:06.074 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-08-25T03:33:06.075 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi002.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-08-25T03:33:06.075 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-08-25T03:33:06.102 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-25T03:33:06.102 INFO:tasks.ceph:role: osd.0 2023-08-25T03:33:06.103 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi002.front.sepia.ceph.com 2023-08-25T03:33:06.103 DEBUG:teuthology.orchestra.run.smithi002:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-08-25T03:33:06.170 INFO:teuthology.orchestra.run.smithi002.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-08-25T03:33:06.170 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-25T03:33:06.170 INFO:teuthology.orchestra.run.smithi002.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-25T03:33:06.171 INFO:teuthology.orchestra.run.smithi002.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-08-25T03:33:06.171 INFO:teuthology.orchestra.run.smithi002.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-25T03:33:06.171 INFO:teuthology.orchestra.run.smithi002.stdout: = sunit=0 swidth=0 blks 2023-08-25T03:33:06.171 INFO:teuthology.orchestra.run.smithi002.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-25T03:33:06.171 INFO:teuthology.orchestra.run.smithi002.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-25T03:33:06.171 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-25T03:33:06.171 INFO:teuthology.orchestra.run.smithi002.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-25T03:33:06.194 INFO:teuthology.orchestra.run.smithi002.stdout:Discarding blocks...Done. 2023-08-25T03:33:06.196 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi002.front.sepia.ceph.com -o noatime 2023-08-25T03:33:06.196 DEBUG:teuthology.orchestra.run.smithi002:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-08-25T03:33:06.278 DEBUG:teuthology.orchestra.run.smithi002:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-08-25T03:33:06.349 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-08-25T03:33:06.417 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-25T03:33:06.418 INFO:tasks.ceph:role: osd.1 2023-08-25T03:33:06.418 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi002.front.sepia.ceph.com 2023-08-25T03:33:06.418 DEBUG:teuthology.orchestra.run.smithi002:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-08-25T03:33:06.484 INFO:teuthology.orchestra.run.smithi002.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-08-25T03:33:06.485 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-25T03:33:06.485 INFO:teuthology.orchestra.run.smithi002.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-25T03:33:06.485 INFO:teuthology.orchestra.run.smithi002.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-08-25T03:33:06.485 INFO:teuthology.orchestra.run.smithi002.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-25T03:33:06.485 INFO:teuthology.orchestra.run.smithi002.stdout: = sunit=0 swidth=0 blks 2023-08-25T03:33:06.485 INFO:teuthology.orchestra.run.smithi002.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-25T03:33:06.485 INFO:teuthology.orchestra.run.smithi002.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-25T03:33:06.486 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-25T03:33:06.486 INFO:teuthology.orchestra.run.smithi002.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-25T03:33:06.509 INFO:teuthology.orchestra.run.smithi002.stdout:Discarding blocks...Done. 2023-08-25T03:33:06.511 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi002.front.sepia.ceph.com -o noatime 2023-08-25T03:33:06.511 DEBUG:teuthology.orchestra.run.smithi002:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-08-25T03:33:06.592 DEBUG:teuthology.orchestra.run.smithi002:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-08-25T03:33:06.662 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-08-25T03:33:06.730 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-25T03:33:06.730 INFO:tasks.ceph:role: osd.2 2023-08-25T03:33:06.730 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi002.front.sepia.ceph.com 2023-08-25T03:33:06.731 DEBUG:teuthology.orchestra.run.smithi002:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-08-25T03:33:06.796 INFO:teuthology.orchestra.run.smithi002.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-08-25T03:33:06.796 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-25T03:33:06.796 INFO:teuthology.orchestra.run.smithi002.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-25T03:33:06.796 INFO:teuthology.orchestra.run.smithi002.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-08-25T03:33:06.797 INFO:teuthology.orchestra.run.smithi002.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-25T03:33:06.797 INFO:teuthology.orchestra.run.smithi002.stdout: = sunit=0 swidth=0 blks 2023-08-25T03:33:06.797 INFO:teuthology.orchestra.run.smithi002.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-25T03:33:06.797 INFO:teuthology.orchestra.run.smithi002.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-25T03:33:06.797 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-25T03:33:06.797 INFO:teuthology.orchestra.run.smithi002.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-25T03:33:06.821 INFO:teuthology.orchestra.run.smithi002.stdout:Discarding blocks...Done. 2023-08-25T03:33:06.822 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi002.front.sepia.ceph.com -o noatime 2023-08-25T03:33:06.823 DEBUG:teuthology.orchestra.run.smithi002:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-08-25T03:33:06.903 DEBUG:teuthology.orchestra.run.smithi002:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-08-25T03:33:06.975 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-08-25T03:33:07.042 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-08-25T03:33:07.043 INFO:tasks.ceph:role: osd.3 2023-08-25T03:33:07.043 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi002.front.sepia.ceph.com 2023-08-25T03:33:07.043 DEBUG:teuthology.orchestra.run.smithi002:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-08-25T03:33:07.109 INFO:teuthology.orchestra.run.smithi002.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-08-25T03:33:07.109 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-25T03:33:07.109 INFO:teuthology.orchestra.run.smithi002.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-25T03:33:07.109 INFO:teuthology.orchestra.run.smithi002.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-08-25T03:33:07.109 INFO:teuthology.orchestra.run.smithi002.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-25T03:33:07.110 INFO:teuthology.orchestra.run.smithi002.stdout: = sunit=0 swidth=0 blks 2023-08-25T03:33:07.110 INFO:teuthology.orchestra.run.smithi002.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-25T03:33:07.110 INFO:teuthology.orchestra.run.smithi002.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-25T03:33:07.110 INFO:teuthology.orchestra.run.smithi002.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-25T03:33:07.110 INFO:teuthology.orchestra.run.smithi002.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-25T03:33:07.135 INFO:teuthology.orchestra.run.smithi002.stdout:Discarding blocks...Done. 2023-08-25T03:33:07.137 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi002.front.sepia.ceph.com -o noatime 2023-08-25T03:33:07.137 DEBUG:teuthology.orchestra.run.smithi002:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-08-25T03:33:07.217 DEBUG:teuthology.orchestra.run.smithi002:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-08-25T03:33:07.288 DEBUG:teuthology.orchestra.run.smithi002:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-25T03:33:07.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:07.500+0000 7f7f59433640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-08-25T03:33:07.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:07.500+0000 7f7f59433640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-08-25T03:33:07.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:07.500+0000 7f7f59433640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-08-25T03:33:07.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:07.500+0000 7f7f59433640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-08-25T03:33:07.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:07.500+0000 7f7f59433640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-08-25T03:33:07.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:07.501+0000 7f7f59433640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-08-25T03:33:10.368 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-08-25T03:33:10.402 DEBUG:teuthology.orchestra.run.smithi002:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-25T03:33:10.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:10.495+0000 7f5da5dc6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-08-25T03:33:10.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:10.495+0000 7f5da5dc6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-08-25T03:33:10.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:10.495+0000 7f5da5dc6640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-08-25T03:33:10.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:10.495+0000 7f5da5dc6640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-08-25T03:33:10.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:10.495+0000 7f5da5dc6640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-08-25T03:33:10.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:10.495+0000 7f5da5dc6640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-08-25T03:33:13.367 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-08-25T03:33:13.400 DEBUG:teuthology.orchestra.run.smithi002:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-25T03:33:13.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:13.491+0000 7f671494d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-08-25T03:33:13.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:13.491+0000 7f671494d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-08-25T03:33:13.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:13.491+0000 7f671494d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-08-25T03:33:13.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:13.491+0000 7f671494d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-08-25T03:33:13.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:13.491+0000 7f671494d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-08-25T03:33:13.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:13.492+0000 7f671494d640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-08-25T03:33:16.360 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-08-25T03:33:16.393 DEBUG:teuthology.orchestra.run.smithi002:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-25T03:33:16.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:16.488+0000 7f3726ced640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-08-25T03:33:16.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:16.489+0000 7f3726ced640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-08-25T03:33:16.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:16.489+0000 7f3726ced640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-08-25T03:33:16.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:16.489+0000 7f3726ced640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-08-25T03:33:16.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:16.489+0000 7f3726ced640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-08-25T03:33:16.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:16.489+0000 7f3726ced640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-08-25T03:33:19.354 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-08-25T03:33:19.390 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-08-25T03:33:19.428 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-25T03:33:19.429 INFO:tasks.ceph:role: osd.4 2023-08-25T03:33:19.429 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com 2023-08-25T03:33:19.429 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-08-25T03:33:19.513 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-08-25T03:33:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-25T03:33:19.513 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-25T03:33:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-08-25T03:33:19.514 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-25T03:33:19.514 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2023-08-25T03:33:19.514 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-25T03:33:19.514 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-25T03:33:19.515 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-25T03:33:19.515 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-25T03:33:19.943 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2023-08-25T03:33:19.944 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2023-08-25T03:33:19.945 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-08-25T03:33:20.130 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-08-25T03:33:20.168 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-08-25T03:33:20.239 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-25T03:33:20.240 INFO:tasks.ceph:role: osd.5 2023-08-25T03:33:20.240 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com 2023-08-25T03:33:20.241 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-08-25T03:33:20.317 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-08-25T03:33:20.317 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-25T03:33:20.317 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-25T03:33:20.317 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-08-25T03:33:20.318 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-25T03:33:20.318 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2023-08-25T03:33:20.318 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-25T03:33:20.318 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-25T03:33:20.318 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-25T03:33:20.318 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-25T03:33:20.732 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2023-08-25T03:33:20.734 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2023-08-25T03:33:20.734 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-08-25T03:33:20.841 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-08-25T03:33:20.875 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-08-25T03:33:20.945 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-25T03:33:20.946 INFO:tasks.ceph:role: osd.6 2023-08-25T03:33:20.946 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi031.front.sepia.ceph.com 2023-08-25T03:33:20.946 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-08-25T03:33:21.023 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-08-25T03:33:21.024 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-25T03:33:21.024 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-25T03:33:21.024 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-08-25T03:33:21.024 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-25T03:33:21.024 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2023-08-25T03:33:21.025 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-25T03:33:21.025 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-25T03:33:21.025 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-25T03:33:21.025 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-25T03:33:21.465 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2023-08-25T03:33:21.467 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2023-08-25T03:33:21.467 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-08-25T03:33:21.639 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-08-25T03:33:21.675 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-08-25T03:33:21.747 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-08-25T03:33:21.748 INFO:tasks.ceph:role: osd.7 2023-08-25T03:33:21.748 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi031.front.sepia.ceph.com 2023-08-25T03:33:21.749 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-08-25T03:33:21.827 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-08-25T03:33:21.827 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2023-08-25T03:33:21.828 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-08-25T03:33:21.828 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-08-25T03:33:21.828 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-08-25T03:33:21.829 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2023-08-25T03:33:21.829 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-08-25T03:33:21.829 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-08-25T03:33:21.829 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-08-25T03:33:21.830 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-08-25T03:33:22.264 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2023-08-25T03:33:22.266 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2023-08-25T03:33:22.268 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-08-25T03:33:22.448 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-08-25T03:33:22.487 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-25T03:33:22.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:22.596+0000 7fa7ec381640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-08-25T03:33:22.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:22.597+0000 7fa7ec381640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-08-25T03:33:22.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:22.597+0000 7fa7ec381640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-08-25T03:33:22.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:22.597+0000 7fa7ec381640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-08-25T03:33:22.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:22.597+0000 7fa7ec381640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-08-25T03:33:22.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:22.598+0000 7fa7ec381640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-08-25T03:33:25.468 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-08-25T03:33:25.502 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-25T03:33:25.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:25.597+0000 7f4fddd17640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-08-25T03:33:25.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:25.597+0000 7f4fddd17640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-08-25T03:33:25.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:25.597+0000 7f4fddd17640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-08-25T03:33:25.600 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:25.597+0000 7f4fddd17640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-08-25T03:33:25.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:25.597+0000 7f4fddd17640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-08-25T03:33:25.601 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:25.598+0000 7f4fddd17640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-08-25T03:33:28.468 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-08-25T03:33:28.503 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-25T03:33:28.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:28.597+0000 7f0c51149640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-08-25T03:33:28.598 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:28.597+0000 7f0c51149640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-08-25T03:33:28.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:28.597+0000 7f0c51149640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-08-25T03:33:28.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:28.597+0000 7f0c51149640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-08-25T03:33:28.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:28.597+0000 7f0c51149640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-08-25T03:33:28.599 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:28.598+0000 7f0c51149640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-08-25T03:33:31.474 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-08-25T03:33:31.508 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-08-25T03:33:31.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:31.600+0000 7f67034c5640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-08-25T03:33:31.602 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:31.600+0000 7f67034c5640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-08-25T03:33:31.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:31.600+0000 7f67034c5640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-08-25T03:33:31.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:31.600+0000 7f67034c5640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-08-25T03:33:31.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:31.600+0000 7f67034c5640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-08-25T03:33:31.603 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:31.601+0000 7f67034c5640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-08-25T03:33:32.747 DEBUG:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T03:33:32.749 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T03:33:34.479 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-08-25T03:33:34.512 INFO:tasks.ceph:Reading keys from all nodes... 2023-08-25T03:33:34.512 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:33:34.513 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-08-25T03:33:34.546 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:33:34.547 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-08-25T03:33:34.620 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:33:34.620 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-08-25T03:33:34.691 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:33:34.692 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-08-25T03:33:34.768 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:33:34.790 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-08-25T03:33:34.822 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:33:34.823 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-08-25T03:33:34.898 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:33:34.898 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-08-25T03:33:34.970 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:33:34.971 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-08-25T03:33:35.005 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:33:35.006 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-08-25T03:33:35.078 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:33:35.079 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-08-25T03:33:35.150 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:33:35.151 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-08-25T03:33:35.222 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:33:35.223 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-08-25T03:33:35.293 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:33:35.293 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-08-25T03:33:35.361 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:33:35.362 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-08-25T03:33:35.430 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:33:35.431 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-08-25T03:33:35.451 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:33:35.452 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-08-25T03:33:35.512 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:33:35.512 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-08-25T03:33:35.530 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:33:35.530 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-08-25T03:33:35.588 INFO:tasks.ceph:Adding keys to all mons... 2023-08-25T03:33:35.589 DEBUG:teuthology.orchestra.run.smithi002:> sudo tee -a /etc/ceph/ceph.keyring 2023-08-25T03:33:35.592 DEBUG:teuthology.orchestra.run.smithi031:> sudo tee -a /etc/ceph/ceph.keyring 2023-08-25T03:33:35.631 INFO:teuthology.orchestra.run.smithi002.stdout:[mgr.y] 2023-08-25T03:33:35.632 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQDxIOhkaC3sEhAAEGY1vTnEmbinFzeG81KHiQ== 2023-08-25T03:33:35.632 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.a] 2023-08-25T03:33:35.633 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQDxIOhkPRSpGxAAlym3fRx8TUwZVWGws/30Nw== 2023-08-25T03:33:35.633 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.c] 2023-08-25T03:33:35.633 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQDxIOhk2hoLIxAA/n01Ior/9O72xdsI5timoA== 2023-08-25T03:33:35.634 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.0] 2023-08-25T03:33:35.634 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQDzIOhkYU/wHRAA516XnQ537/DrH2XeVOhs2Q== 2023-08-25T03:33:35.635 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.1] 2023-08-25T03:33:35.635 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQD2IOhkEdSbHRAAIduQDl/OxgzmFmMV83S38g== 2023-08-25T03:33:35.635 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.2] 2023-08-25T03:33:35.635 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQD5IOhkm9dlHRAAGL6kwSQgXPQ+RJt+Uqj4xQ== 2023-08-25T03:33:35.636 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.3] 2023-08-25T03:33:35.636 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQD8IOhk5jg9HRAAIMJhGJ3s7PbQ6XYoDbQ2kA== 2023-08-25T03:33:35.636 INFO:teuthology.orchestra.run.smithi002.stdout:[mgr.x] 2023-08-25T03:33:35.637 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQDxIOhkS50iGBAAkESO8E2kBz5meycjbPmMFw== 2023-08-25T03:33:35.637 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.b] 2023-08-25T03:33:35.637 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQDxIOhkNAqPKBAA6WWkiOL5E6fAboTSSPpkjA== 2023-08-25T03:33:35.638 INFO:teuthology.orchestra.run.smithi002.stdout:[mds.d] 2023-08-25T03:33:35.638 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQDxIOhk4pE9MBAAelxvHaPZaiXLMIVPxXu5bQ== 2023-08-25T03:33:35.638 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.4] 2023-08-25T03:33:35.638 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQACIehk/emkIxAAsj0+YCvCJXSBujajtCOWSQ== 2023-08-25T03:33:35.639 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.5] 2023-08-25T03:33:35.639 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQAFIehkQGGsIxAAZg0CHqCjmN74rhDKmTlirQ== 2023-08-25T03:33:35.639 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.6] 2023-08-25T03:33:35.640 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQAIIehksCOrIxAAZFtoY9baekBEIgaRWo+5UA== 2023-08-25T03:33:35.640 INFO:teuthology.orchestra.run.smithi002.stdout:[osd.7] 2023-08-25T03:33:35.640 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQALIehkhhnYIxAAzQQ2qxOVDdUKc9a0C3YH4A== 2023-08-25T03:33:35.641 INFO:teuthology.orchestra.run.smithi002.stdout:[client.0] 2023-08-25T03:33:35.641 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQDxIOhkOijPNBAAx276DY0meH7ng2YxV3536A== 2023-08-25T03:33:35.641 INFO:teuthology.orchestra.run.smithi002.stdout:[client.2] 2023-08-25T03:33:35.641 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQDxIOhkKjdAOBAATR/CiHdKxI+68Sd2hI5Mcw== 2023-08-25T03:33:35.642 INFO:teuthology.orchestra.run.smithi002.stdout:[client.1] 2023-08-25T03:33:35.642 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQDyIOhkQlM+ABAA6hnKw9yLbHTeZWxIjmru/w== 2023-08-25T03:33:35.642 INFO:teuthology.orchestra.run.smithi002.stdout:[client.3] 2023-08-25T03:33:35.643 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQDyIOhkk4GpAxAATAWzeGdDpAann/fbqCtIzw== 2023-08-25T03:33:35.655 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr.y] 2023-08-25T03:33:35.656 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDxIOhkaC3sEhAAEGY1vTnEmbinFzeG81KHiQ== 2023-08-25T03:33:35.656 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.a] 2023-08-25T03:33:35.656 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDxIOhkPRSpGxAAlym3fRx8TUwZVWGws/30Nw== 2023-08-25T03:33:35.656 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.c] 2023-08-25T03:33:35.656 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDxIOhk2hoLIxAA/n01Ior/9O72xdsI5timoA== 2023-08-25T03:33:35.656 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.0] 2023-08-25T03:33:35.656 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDzIOhkYU/wHRAA516XnQ537/DrH2XeVOhs2Q== 2023-08-25T03:33:35.657 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.1] 2023-08-25T03:33:35.657 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQD2IOhkEdSbHRAAIduQDl/OxgzmFmMV83S38g== 2023-08-25T03:33:35.657 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.2] 2023-08-25T03:33:35.657 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQD5IOhkm9dlHRAAGL6kwSQgXPQ+RJt+Uqj4xQ== 2023-08-25T03:33:35.657 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.3] 2023-08-25T03:33:35.657 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQD8IOhk5jg9HRAAIMJhGJ3s7PbQ6XYoDbQ2kA== 2023-08-25T03:33:35.657 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr.x] 2023-08-25T03:33:35.658 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDxIOhkS50iGBAAkESO8E2kBz5meycjbPmMFw== 2023-08-25T03:33:35.658 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.b] 2023-08-25T03:33:35.658 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDxIOhkNAqPKBAA6WWkiOL5E6fAboTSSPpkjA== 2023-08-25T03:33:35.658 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.d] 2023-08-25T03:33:35.658 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDxIOhk4pE9MBAAelxvHaPZaiXLMIVPxXu5bQ== 2023-08-25T03:33:35.658 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.4] 2023-08-25T03:33:35.658 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQACIehk/emkIxAAsj0+YCvCJXSBujajtCOWSQ== 2023-08-25T03:33:35.658 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.5] 2023-08-25T03:33:35.659 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQAFIehkQGGsIxAAZg0CHqCjmN74rhDKmTlirQ== 2023-08-25T03:33:35.659 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.6] 2023-08-25T03:33:35.659 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQAIIehksCOrIxAAZFtoY9baekBEIgaRWo+5UA== 2023-08-25T03:33:35.659 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.7] 2023-08-25T03:33:35.659 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQALIehkhhnYIxAAzQQ2qxOVDdUKc9a0C3YH4A== 2023-08-25T03:33:35.659 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2023-08-25T03:33:35.659 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDxIOhkOijPNBAAx276DY0meH7ng2YxV3536A== 2023-08-25T03:33:35.659 INFO:teuthology.orchestra.run.smithi031.stdout:[client.2] 2023-08-25T03:33:35.660 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDxIOhkKjdAOBAATR/CiHdKxI+68Sd2hI5Mcw== 2023-08-25T03:33:35.660 INFO:teuthology.orchestra.run.smithi031.stdout:[client.1] 2023-08-25T03:33:35.660 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDyIOhkQlM+ABAA6hnKw9yLbHTeZWxIjmru/w== 2023-08-25T03:33:35.660 INFO:teuthology.orchestra.run.smithi031.stdout:[client.3] 2023-08-25T03:33:35.660 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDyIOhkk4GpAxAATAWzeGdDpAann/fbqCtIzw== 2023-08-25T03:33:35.661 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-08-25T03:33:35.687 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-08-25T03:33:35.756 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-25T03:33:35.758 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-25T03:33:35.816 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-25T03:33:35.818 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-25T03:33:35.876 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T03:33:35.878 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T03:33:35.932 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T03:33:35.934 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T03:33:35.988 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T03:33:35.990 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T03:33:36.044 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T03:33:36.046 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T03:33:36.100 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-08-25T03:33:36.102 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-08-25T03:33:36.156 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-25T03:33:36.159 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-25T03:33:36.216 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-25T03:33:36.218 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-08-25T03:33:36.274 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T03:33:36.277 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T03:33:36.330 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T03:33:36.333 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T03:33:36.386 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T03:33:36.388 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T03:33:36.443 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T03:33:36.445 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-08-25T03:33:36.499 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-25T03:33:36.502 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-25T03:33:36.559 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-25T03:33:36.561 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-25T03:33:36.618 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-25T03:33:36.621 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-25T03:33:36.678 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-25T03:33:36.680 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-08-25T03:33:36.738 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-08-25T03:33:36.738 DEBUG:teuthology.orchestra.run.smithi002:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-08-25T03:33:36.765 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-08-25T03:33:36.865 DEBUG:teuthology.orchestra.run.smithi002:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-08-25T03:33:36.892 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-08-25T03:33:36.919 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-08-25T03:33:37.019 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-08-25T03:33:37.045 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-08-25T03:33:37.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-08-25T03:33:37.212 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-08-25T03:33:37.239 DEBUG:teuthology.orchestra.run.smithi002:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-08-25T03:33:37.241 DEBUG:teuthology.orchestra.run.smithi031:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-08-25T03:33:37.295 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-08-25T03:33:37.296 INFO:tasks.ceph.mon.a:Restarting daemon 2023-08-25T03:33:37.296 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-08-25T03:33:37.299 INFO:tasks.ceph.mon.a:Started 2023-08-25T03:33:37.299 INFO:tasks.ceph.mon.b:Restarting daemon 2023-08-25T03:33:37.299 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-08-25T03:33:37.338 INFO:tasks.ceph.mon.b:Started 2023-08-25T03:33:37.338 INFO:tasks.ceph.mon.c:Restarting daemon 2023-08-25T03:33:37.339 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-08-25T03:33:37.340 INFO:tasks.ceph.mon.c:Started 2023-08-25T03:33:37.341 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-08-25T03:33:37.341 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-08-25T03:33:37.341 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-08-25T03:33:37.343 INFO:tasks.ceph.mgr.y:Started 2023-08-25T03:33:37.344 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-08-25T03:33:37.344 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-08-25T03:33:37.345 INFO:tasks.ceph.mgr.x:Started 2023-08-25T03:33:37.346 INFO:tasks.ceph:Setting crush tunables to default 2023-08-25T03:33:37.346 DEBUG:teuthology.orchestra.run.smithi002:> sudo ceph --cluster ceph osd crush tunables default 2023-08-25T03:33:37.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.472+0000 7f4f062f2700 1 Processor -- start 2023-08-25T03:33:37.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.472+0000 7f4f062f2700 1 -- start start 2023-08-25T03:33:37.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.473+0000 7f4f062f2700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f4f0006a660 0x7f4f00072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:37.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.473+0000 7f4f062f2700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f0006aa40 0x7f4f0015f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:37.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.473+0000 7f4f062f2700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f000b1e30 con 0x7f4f00066050 2023-08-25T03:33:37.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.473+0000 7f4f062f2700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f000b12b0 con 0x7f4f0006a660 2023-08-25T03:33:37.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.473+0000 7f4f062f2700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f000b1590 con 0x7f4f0006aa40 2023-08-25T03:33:37.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.473+0000 7f4efffff700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f4f0006a660 0x7f4f00072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:37.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.473+0000 7f4efffff700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f4f0006a660 0x7f4f00072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58464/0 (socket says 172.21.15.2:58464) 2023-08-25T03:33:37.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.473+0000 7f4efffff700 1 -- 172.21.15.2:0/3383851005 learned_addr learned my addr 172.21.15.2:0/3383851005 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:33:37.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.473+0000 7f4f0488f700 1 --2- 172.21.15.2:0/3383851005 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f0006aa40 0x7f4f0015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:37.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.473+0000 7f4efffff700 1 -- 172.21.15.2:0/3383851005 >> v2:172.21.15.2:3300/0 conn(0x7f4f0006a660 msgr2=0x7f4f00072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-08-25T03:33:37.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.473+0000 7f4efffff700 1 -- 172.21.15.2:0/3383851005 >> v2:172.21.15.2:3300/0 conn(0x7f4f0006a660 msgr2=0x7f4f00072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-25T03:33:37.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.473+0000 7f4efffff700 1 --2- 172.21.15.2:0/3383851005 >> v2:172.21.15.2:3300/0 conn(0x7f4f0006a660 0x7f4f00072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-25T03:33:37.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.473+0000 7f4efffff700 1 --2- 172.21.15.2:0/3383851005 >> v2:172.21.15.2:3300/0 conn(0x7f4f0006a660 0x7f4f00072330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-25T03:33:37.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.473+0000 7f4f0488f700 1 -- 172.21.15.2:0/3383851005 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f0006aa40 msgr2=0x7f4f0015f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-08-25T03:33:37.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.473+0000 7f4f0488f700 1 -- 172.21.15.2:0/3383851005 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f0006aa40 msgr2=0x7f4f0015f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-08-25T03:33:37.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.473+0000 7f4f0488f700 1 --2- 172.21.15.2:0/3383851005 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f0006aa40 0x7f4f0015f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-08-25T03:33:37.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.473+0000 7f4f0488f700 1 --2- 172.21.15.2:0/3383851005 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f0006aa40 0x7f4f0015f910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-08-25T03:33:37.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.660+0000 7f4efeffd700 1 -- 172.21.15.2:0/3383851005 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2077729824 0 0) 0x7f4f000b1e30 con 0x7f4f00066050 2023-08-25T03:33:37.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.660+0000 7f4efeffd700 1 -- 172.21.15.2:0/3383851005 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4eec0035b0 con 0x7f4f00066050 2023-08-25T03:33:37.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.660+0000 7f4efeffd700 1 -- 172.21.15.2:0/3383851005 <== mon.1 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2296600544 0 0) 0x7f4eec0035b0 con 0x7f4f00066050 2023-08-25T03:33:37.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.660+0000 7f4efeffd700 1 -- 172.21.15.2:0/3383851005 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f0006aa40 msgr2=0x7f4f0015f910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-25T03:33:37.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.660+0000 7f4efeffd700 1 --2- 172.21.15.2:0/3383851005 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f0006aa40 0x7f4f0015f910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:37.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.660+0000 7f4efeffd700 1 -- 172.21.15.2:0/3383851005 >> v2:172.21.15.2:3300/0 conn(0x7f4f0006a660 msgr2=0x7f4f00072330 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-25T03:33:37.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.660+0000 7f4efeffd700 1 --2- 172.21.15.2:0/3383851005 >> v2:172.21.15.2:3300/0 conn(0x7f4f0006a660 0x7f4f00072330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:37.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.660+0000 7f4efeffd700 1 -- 172.21.15.2:0/3383851005 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f000b1870 con 0x7f4f00066050 2023-08-25T03:33:37.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.660+0000 7f4efeffd700 1 -- 172.21.15.2:0/3383851005 <== mon.1 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f4ee8001670 con 0x7f4f00066050 2023-08-25T03:33:37.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.660+0000 7f4efeffd700 1 -- 172.21.15.2:0/3383851005 >> v1:172.21.15.31:6789/0 conn(0x7f4f00066050 legacy=0x7f4f0015f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:37.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.660+0000 7f4efeffd700 1 --2- 172.21.15.2:0/3383851005 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f0006aa40 0x7f4eec003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:37.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.660+0000 7f4efeffd700 1 --2- 172.21.15.2:0/3383851005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eec004080 0x7f4eec004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:37.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.660+0000 7f4efeffd700 1 --2- 172.21.15.2:0/3383851005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4eec004d90 0x7f4eec005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:37.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.660+0000 7f4efeffd700 1 -- 172.21.15.2:0/3383851005 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f000b12b0 con 0x7f4eec004080 2023-08-25T03:33:37.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.660+0000 7f4efeffd700 1 -- 172.21.15.2:0/3383851005 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f000b1590 con 0x7f4eec004d90 2023-08-25T03:33:37.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.660+0000 7f4efeffd700 1 -- 172.21.15.2:0/3383851005 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec00bc20 con 0x7f4f0006aa40 2023-08-25T03:33:37.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.661+0000 7f4efffff700 1 --2- 172.21.15.2:0/3383851005 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f0006aa40 0x7f4eec003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:37.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.661+0000 7f4eff7fe700 1 --2- 172.21.15.2:0/3383851005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eec004080 0x7f4eec004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:37.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.661+0000 7f4f0488f700 1 --2- 172.21.15.2:0/3383851005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4eec004d90 0x7f4eec005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:37.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.661+0000 7f4efffff700 1 -- 172.21.15.2:0/3383851005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4eec004d90 msgr2=0x7f4eec005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:33:37.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.661+0000 7f4efffff700 1 --2- 172.21.15.2:0/3383851005 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4eec004d90 0x7f4eec005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:37.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.661+0000 7f4efffff700 1 -- 172.21.15.2:0/3383851005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eec004080 msgr2=0x7f4eec004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:33:37.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.661+0000 7f4efffff700 1 --2- 172.21.15.2:0/3383851005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eec004080 0x7f4eec004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:37.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.661+0000 7f4efffff700 1 -- 172.21.15.2:0/3383851005 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eec0035b0 con 0x7f4f0006aa40 2023-08-25T03:33:37.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.661+0000 7f4eff7fe700 1 --2- 172.21.15.2:0/3383851005 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4eec004080 0x7f4eec004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:33:37.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.661+0000 7f4efffff700 1 --2- 172.21.15.2:0/3383851005 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f0006aa40 0x7f4eec003a00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009910 tx=0x7f4eec005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efec919368e45760 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:37.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.663+0000 7f4efeffd700 1 -- 172.21.15.2:0/3383851005 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0017070 con 0x7f4f0006aa40 2023-08-25T03:33:37.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.663+0000 7f4efeffd700 1 -- 172.21.15.2:0/3383851005 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef0005a20 con 0x7f4f0006aa40 2023-08-25T03:33:37.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.663+0000 7f4efeffd700 1 -- 172.21.15.2:0/3383851005 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef000c070 con 0x7f4f0006aa40 2023-08-25T03:33:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.664+0000 7f4f062f2700 1 -- 172.21.15.2:0/3383851005 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f0006aa40 msgr2=0x7f4eec003a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.664+0000 7f4f062f2700 1 --2- 172.21.15.2:0/3383851005 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f0006aa40 0x7f4eec003a00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009910 tx=0x7f4eec005880 comp rx=0 tx=0).stop 2023-08-25T03:33:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.664+0000 7f4f0488f700 1 -- 172.21.15.2:0/3383851005 reap_dead start 2023-08-25T03:33:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.664+0000 7f4f062f2700 1 -- 172.21.15.2:0/3383851005 shutdown_connections 2023-08-25T03:33:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.664+0000 7f4f062f2700 1 -- 172.21.15.2:0/3383851005 >> 172.21.15.2:0/3383851005 conn(0x7f4f000cf500 msgr2=0x7f4f000cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.664+0000 7f4f062f2700 1 -- 172.21.15.2:0/3383851005 shutdown_connections 2023-08-25T03:33:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.665+0000 7f4f062f2700 1 -- 172.21.15.2:0/3383851005 wait complete. 2023-08-25T03:33:37.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.666+0000 7f4f062f2700 1 Processor -- start 2023-08-25T03:33:37.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.666+0000 7f4f062f2700 1 -- start start 2023-08-25T03:33:37.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.667+0000 7f4f062f2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f0006aa40 0x7f4f00153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:37.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.667+0000 7f4f062f2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f000c6540 0x7f4f000ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:37.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.667+0000 7f4f062f2700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f00154be0 0x7f4f0015bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:37.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.667+0000 7f4f062f2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f00153bc0 con 0x7f4f000c6540 2023-08-25T03:33:37.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.667+0000 7f4f062f2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f000ae7e0 con 0x7f4f0006aa40 2023-08-25T03:33:37.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.667+0000 7f4f062f2700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f000c6a00 con 0x7f4f00154be0 2023-08-25T03:33:37.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.667+0000 7f4eff7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f000c6540 0x7f4f000ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:37.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.667+0000 7f4eff7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f000c6540 0x7f4f000ae2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58500/0 (socket says 172.21.15.2:58500) 2023-08-25T03:33:37.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.667+0000 7f4eff7fe700 1 -- 172.21.15.2:0/819912160 learned_addr learned my addr 172.21.15.2:0/819912160 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:33:37.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.667+0000 7f4efffff700 1 --2- 172.21.15.2:0/819912160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f0006aa40 0x7f4f00153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:37.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.667+0000 7f4f0488f700 1 --2- 172.21.15.2:0/819912160 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f00154be0 0x7f4f0015bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:37.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.668+0000 7f4eff7fe700 1 -- 172.21.15.2:0/819912160 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f00154be0 msgr2=0x7f4f0015bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:37.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.668+0000 7f4eff7fe700 1 --2- 172.21.15.2:0/819912160 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f00154be0 0x7f4f0015bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:37.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.668+0000 7f4eff7fe700 1 -- 172.21.15.2:0/819912160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f0006aa40 msgr2=0x7f4f00153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:37.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.668+0000 7f4eff7fe700 1 --2- 172.21.15.2:0/819912160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f0006aa40 0x7f4f00153680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:37.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.668+0000 7f4eff7fe700 1 -- 172.21.15.2:0/819912160 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f000c6ce0 con 0x7f4f000c6540 2023-08-25T03:33:37.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.668+0000 7f4eff7fe700 1 --2- 172.21.15.2:0/819912160 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f000c6540 0x7f4f000ae2a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4ee80017e0 tx=0x7f4ee800fdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:37.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.668+0000 7f4efcff9700 1 -- 172.21.15.2:0/819912160 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee80052e0 con 0x7f4f000c6540 2023-08-25T03:33:37.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.668+0000 7f4efcff9700 1 -- 172.21.15.2:0/819912160 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee8005440 con 0x7f4f000c6540 2023-08-25T03:33:37.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.668+0000 7f4f062f2700 1 -- 172.21.15.2:0/819912160 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f000c6fc0 con 0x7f4f000c6540 2023-08-25T03:33:37.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.668+0000 7f4efcff9700 1 -- 172.21.15.2:0/819912160 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee80038c0 con 0x7f4f000c6540 2023-08-25T03:33:37.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.668+0000 7f4f062f2700 1 -- 172.21.15.2:0/819912160 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f000adf00 con 0x7f4f000c6540 2023-08-25T03:33:37.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.669+0000 7f4efcff9700 1 -- 172.21.15.2:0/819912160 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4ee8003d20 con 0x7f4f000c6540 2023-08-25T03:33:37.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.669+0000 7f4efcff9700 1 -- 172.21.15.2:0/819912160 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4ee801d460 con 0x7f4f000c6540 2023-08-25T03:33:37.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.670+0000 7f4f062f2700 1 -- 172.21.15.2:0/819912160 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f000ac180 con 0x7f4f000c6540 2023-08-25T03:33:37.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.672+0000 7f4efcff9700 1 -- 172.21.15.2:0/819912160 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f4f000ac180 con 0x7f4f000c6540 2023-08-25T03:33:37.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.751+0000 7f4f062f2700 1 -- 172.21.15.2:0/819912160 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f4f000746d0 con 0x7f4f000c6540 2023-08-25T03:33:37.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.754+0000 7f4efcff9700 1 -- 172.21.15.2:0/819912160 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f4ee800d5e0 con 0x7f4f000c6540 2023-08-25T03:33:37.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.755+0000 7f4f062f2700 1 -- 172.21.15.2:0/819912160 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f000c6540 msgr2=0x7f4f000ae2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:37.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.755+0000 7f4f062f2700 1 --2- 172.21.15.2:0/819912160 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f000c6540 0x7f4f000ae2a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4ee80017e0 tx=0x7f4ee800fdc0 comp rx=0 tx=0).stop 2023-08-25T03:33:37.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.755+0000 7f4f062f2700 1 -- 172.21.15.2:0/819912160 shutdown_connections 2023-08-25T03:33:37.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.755+0000 7f4f062f2700 1 --2- 172.21.15.2:0/819912160 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f00154be0 0x7f4f0015bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:37.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.755+0000 7f4f062f2700 1 --2- 172.21.15.2:0/819912160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f0006aa40 0x7f4f00153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:37.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.755+0000 7f4f062f2700 1 --2- 172.21.15.2:0/819912160 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4f000c6540 0x7f4f000ae2a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:37.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.755+0000 7f4f062f2700 1 -- 172.21.15.2:0/819912160 >> 172.21.15.2:0/819912160 conn(0x7f4f000cf500 msgr2=0x7f4f000aee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:37.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.755+0000 7f4f062f2700 1 -- 172.21.15.2:0/819912160 shutdown_connections 2023-08-25T03:33:37.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:37.755+0000 7f4f062f2700 1 -- 172.21.15.2:0/819912160 wait complete. 2023-08-25T03:33:37.757 INFO:teuthology.orchestra.run.smithi002.stderr:adjusted tunables profile to default 2023-08-25T03:33:37.767 INFO:tasks.ceph:check_enable_crimson: False 2023-08-25T03:33:37.768 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-08-25T03:33:37.768 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:33:37.768 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-08-25T03:33:37.782 INFO:tasks.ceph.mgr.x.smithi031.stderr:2023-08-25T03:33:37.781+0000 7faad00e4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-25T03:33:37.787 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:37.787+0000 7f8f9b465200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-08-25T03:33:37.795 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:33:37.795 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-08-25T03:33:37.851 INFO:tasks.ceph.mgr.x.smithi031.stderr:2023-08-25T03:33:37.850+0000 7faad00e4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-25T03:33:37.857 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:37.856+0000 7f8f9b465200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-08-25T03:33:37.862 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:33:37.862 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-08-25T03:33:37.928 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:33:37.928 DEBUG:teuthology.orchestra.run.smithi002:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-08-25T03:33:37.930 INFO:tasks.ceph.mgr.x.smithi031.stderr:2023-08-25T03:33:37.929+0000 7faad00e4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-25T03:33:37.939 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:37.938+0000 7f8f9b465200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-08-25T03:33:37.965 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:33:37.965 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-08-25T03:33:37.992 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:33:37.993 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-08-25T03:33:38.013 INFO:tasks.ceph.mgr.x.smithi031.stderr:2023-08-25T03:33:38.012+0000 7faad00e4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-25T03:33:38.024 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:38.023+0000 7f8f9b465200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-08-25T03:33:38.040 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:33:38.040 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-08-25T03:33:38.110 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:33:38.110 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-08-25T03:33:38.170 INFO:tasks.ceph.mgr.x.smithi031.stderr:2023-08-25T03:33:38.169+0000 7faad00e4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-25T03:33:38.173 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:38.173+0000 7f8f9b465200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-08-25T03:33:38.176 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd new fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722 0 2023-08-25T03:33:38.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.343+0000 7f2671fab700 1 Processor -- start 2023-08-25T03:33:38.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.343+0000 7f2671fab700 1 -- start start 2023-08-25T03:33:38.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.344+0000 7f2671fab700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f266c0ae6b0 0x7f266c06a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:38.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.344+0000 7f2671fab700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f266c072370 0x7f266c06a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:38.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.344+0000 7f2671fab700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f266c0b0f80 con 0x7f266c065fe0 2023-08-25T03:33:38.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.344+0000 7f2671fab700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f266c0b1b00 con 0x7f266c0ae6b0 2023-08-25T03:33:38.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.344+0000 7f2671fab700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266c15e990 con 0x7f266c072370 2023-08-25T03:33:38.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.344+0000 7f266affd700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f266c072370 0x7f266c06a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:38.347 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.344+0000 7f266bfff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f266c065fe0 0x7f266c0b41b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43056/0 (socket says 172.21.15.31:43056) 2023-08-25T03:33:38.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.344+0000 7f266bfff700 1 -- 172.21.15.31:0/2991226494 learned_addr learned my addr 172.21.15.31:0/2991226494 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-08-25T03:33:38.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.344+0000 7f266b7fe700 1 --2- 172.21.15.31:0/2991226494 >> v2:172.21.15.2:3300/0 conn(0x7f266c0ae6b0 0x7f266c06a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:38.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.345+0000 7f266a7fc700 1 -- 172.21.15.31:0/2991226494 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4101105266 0 0) 0x7f266c0b0f80 con 0x7f266c065fe0 2023-08-25T03:33:38.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.345+0000 7f266affd700 1 -- 172.21.15.31:0/2991226494 >> v2:172.21.15.2:3300/0 conn(0x7f266c0ae6b0 msgr2=0x7f266c06a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:33:38.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.345+0000 7f266affd700 1 --2- 172.21.15.31:0/2991226494 >> v2:172.21.15.2:3300/0 conn(0x7f266c0ae6b0 0x7f266c06a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.345+0000 7f266affd700 1 -- 172.21.15.31:0/2991226494 >> v1:172.21.15.31:6789/0 conn(0x7f266c065fe0 legacy=0x7f266c0b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:38.348 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.345+0000 7f266affd700 1 -- 172.21.15.31:0/2991226494 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266c0b11d0 con 0x7f266c072370 2023-08-25T03:33:38.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.345+0000 7f266affd700 1 --2- 172.21.15.31:0/2991226494 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f266c072370 0x7f266c06a930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2654009840 tx=0x7f2654009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6eb3e7c80f9e119c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:38.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.345+0000 7f266a7fc700 1 -- 172.21.15.31:0/2991226494 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2654011830 con 0x7f266c072370 2023-08-25T03:33:38.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.345+0000 7f266a7fc700 1 -- 172.21.15.31:0/2991226494 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2654011990 con 0x7f266c072370 2023-08-25T03:33:38.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.345+0000 7f266a7fc700 1 -- 172.21.15.31:0/2991226494 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2654011d20 con 0x7f266c072370 2023-08-25T03:33:38.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.345+0000 7f2671fab700 1 -- 172.21.15.31:0/2991226494 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f266c072370 msgr2=0x7f266c06a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:38.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.345+0000 7f2671fab700 1 --2- 172.21.15.31:0/2991226494 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f266c072370 0x7f266c06a930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2654009840 tx=0x7f2654009b50 comp rx=0 tx=0).stop 2023-08-25T03:33:38.349 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.345+0000 7f2671fab700 1 -- 172.21.15.31:0/2991226494 shutdown_connections 2023-08-25T03:33:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.345+0000 7f2671fab700 1 --2- 172.21.15.31:0/2991226494 >> v2:172.21.15.2:3300/0 conn(0x7f266c0ae6b0 0x7f266c06a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.345+0000 7f2671fab700 1 --2- 172.21.15.31:0/2991226494 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f266c072370 0x7f266c06a930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.345+0000 7f2671fab700 1 -- 172.21.15.31:0/2991226494 >> 172.21.15.31:0/2991226494 conn(0x7f266c0cf500 msgr2=0x7f266c0cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.346+0000 7f2671fab700 1 -- 172.21.15.31:0/2991226494 shutdown_connections 2023-08-25T03:33:38.350 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.346+0000 7f2671fab700 1 -- 172.21.15.31:0/2991226494 wait complete. 2023-08-25T03:33:38.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.347+0000 7f2671fab700 1 Processor -- start 2023-08-25T03:33:38.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.347+0000 7f2671fab700 1 -- start start 2023-08-25T03:33:38.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.348+0000 7f2671fab700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f266c065fe0 0x7f266c0c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:38.352 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.348+0000 7f2671fab700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f266c072370 0x7f266c0b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:38.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.348+0000 7f2671fab700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f266c0ae6b0 0x7f266c1f67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:38.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.348+0000 7f2671fab700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266c1f6330 con 0x7f266c065fe0 2023-08-25T03:33:38.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.348+0000 7f2671fab700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266c1f6000 con 0x7f266c072370 2023-08-25T03:33:38.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.348+0000 7f2671fab700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266c0c9590 con 0x7f266c0ae6b0 2023-08-25T03:33:38.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.348+0000 7f266affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f266c072370 0x7f266c0b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:38.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.348+0000 7f266bfff700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f266c0ae6b0 0x7f266c1f67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:38.353 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.348+0000 7f266affd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f266c072370 0x7f266c0b4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41666/0 (socket says 172.21.15.31:41666) 2023-08-25T03:33:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.348+0000 7f266b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f266c065fe0 0x7f266c0c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.348+0000 7f266affd700 1 -- 172.21.15.31:0/2675719495 learned_addr learned my addr 172.21.15.31:0/2675719495 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-08-25T03:33:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.348+0000 7f266bfff700 1 -- 172.21.15.31:0/2675719495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f266c072370 msgr2=0x7f266c0b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.348+0000 7f266bfff700 1 --2- 172.21.15.31:0/2675719495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f266c072370 0x7f266c0b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.348+0000 7f266bfff700 1 -- 172.21.15.31:0/2675719495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f266c065fe0 msgr2=0x7f266c0c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.348+0000 7f266bfff700 1 --2- 172.21.15.31:0/2675719495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f266c065fe0 0x7f266c0c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.348+0000 7f266bfff700 1 -- 172.21.15.31:0/2675719495 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26540094a0 con 0x7f266c0ae6b0 2023-08-25T03:33:38.354 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.348+0000 7f266affd700 1 --2- 172.21.15.31:0/2675719495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f266c072370 0x7f266c0b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:33:38.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.348+0000 7f266bfff700 1 --2- 172.21.15.31:0/2675719495 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f266c0ae6b0 0x7f266c1f67c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f266400c430 tx=0x7f266400c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:38.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.348+0000 7f2663fff700 1 -- 172.21.15.31:0/2675719495 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2664004c80 con 0x7f266c0ae6b0 2023-08-25T03:33:38.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.348+0000 7f2663fff700 1 -- 172.21.15.31:0/2675719495 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2664004e00 con 0x7f266c0ae6b0 2023-08-25T03:33:38.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.348+0000 7f2671fab700 1 -- 172.21.15.31:0/2675719495 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f266c0c6bd0 con 0x7f266c0ae6b0 2023-08-25T03:33:38.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.348+0000 7f2663fff700 1 -- 172.21.15.31:0/2675719495 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2664007310 con 0x7f266c0ae6b0 2023-08-25T03:33:38.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.349+0000 7f2671fab700 1 -- 172.21.15.31:0/2675719495 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f266c0c7190 con 0x7f266c0ae6b0 2023-08-25T03:33:38.355 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.349+0000 7f2663fff700 1 -- 172.21.15.31:0/2675719495 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f266401d3f0 con 0x7f266c0ae6b0 2023-08-25T03:33:38.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.349+0000 7f2663fff700 1 -- 172.21.15.31:0/2675719495 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2664013700 con 0x7f266c0ae6b0 2023-08-25T03:33:38.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.349+0000 7f2671fab700 1 -- 172.21.15.31:0/2675719495 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f266c0c7380 con 0x7f266c0ae6b0 2023-08-25T03:33:38.356 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.352+0000 7f2663fff700 1 -- 172.21.15.31:0/2675719495 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f26640139c0 con 0x7f266c0ae6b0 2023-08-25T03:33:38.438 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.437+0000 7f2671fab700 1 -- 172.21.15.31:0/2675719495 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "osd new", "uuid": "fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722", "id": 0} v 0) v1 -- 0x7f266c1f64a0 con 0x7f266c0ae6b0 2023-08-25T03:33:38.441 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.440+0000 7f2663fff700 1 -- 172.21.15.31:0/2675719495 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2664013c30 con 0x7f266c0ae6b0 2023-08-25T03:33:38.442 INFO:teuthology.orchestra.run.smithi031.stdout:0 2023-08-25T03:33:38.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.441+0000 7f2671fab700 1 -- 172.21.15.31:0/2675719495 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f266c0ae6b0 msgr2=0x7f266c1f67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:38.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.441+0000 7f2671fab700 1 --2- 172.21.15.31:0/2675719495 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f266c0ae6b0 0x7f266c1f67c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f266400c430 tx=0x7f266400c740 comp rx=0 tx=0).stop 2023-08-25T03:33:38.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.441+0000 7f2671fab700 1 -- 172.21.15.31:0/2675719495 shutdown_connections 2023-08-25T03:33:38.443 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.441+0000 7f2671fab700 1 --2- 172.21.15.31:0/2675719495 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f266c065fe0 0x7f266c0c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.441+0000 7f2671fab700 1 --2- 172.21.15.31:0/2675719495 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f266c0ae6b0 0x7f266c1f67c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.441+0000 7f2671fab700 1 --2- 172.21.15.31:0/2675719495 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f266c072370 0x7f266c0b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.441+0000 7f2671fab700 1 -- 172.21.15.31:0/2675719495 >> 172.21.15.31:0/2675719495 conn(0x7f266c0cf500 msgr2=0x7f266c06c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:38.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.441+0000 7f2671fab700 1 -- 172.21.15.31:0/2675719495 shutdown_connections 2023-08-25T03:33:38.444 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.441+0000 7f2671fab700 1 -- 172.21.15.31:0/2675719495 wait complete. 2023-08-25T03:33:38.454 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd new c30c594a-88f7-4be9-9bd0-320bc6489fab 1 2023-08-25T03:33:38.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.576+0000 7f2fbee11700 1 Processor -- start 2023-08-25T03:33:38.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.577+0000 7f2fbee11700 1 -- start start 2023-08-25T03:33:38.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.577+0000 7f2fbee11700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2fb80cfb80 0x7f2fb80cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:38.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.577+0000 7f2fbee11700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2fb80cdd30 0x7f2fb806d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:38.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.577+0000 7f2fbee11700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb80d0000 con 0x7f2fb815b170 2023-08-25T03:33:38.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.577+0000 7f2fbee11700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb80ce1c0 con 0x7f2fb80cfb80 2023-08-25T03:33:38.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.577+0000 7f2fbee11700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb80cf6f0 con 0x7f2fb80cdd30 2023-08-25T03:33:38.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.578+0000 7f2fbd3ae700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2fb80cdd30 0x7f2fb806d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:38.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.578+0000 7f2fb7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2fb80cfb80 0x7f2fb80cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:38.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.578+0000 7f2fbcbad700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f2fb815b170 0x7f2fb80cc640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43064/0 (socket says 172.21.15.31:43064) 2023-08-25T03:33:38.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.578+0000 7f2fbd3ae700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2fb80cdd30 0x7f2fb806d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.31:58370/0 (socket says 172.21.15.31:58370) 2023-08-25T03:33:38.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.578+0000 7f2fb7fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2fb80cfb80 0x7f2fb80cd7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41672/0 (socket says 172.21.15.31:41672) 2023-08-25T03:33:38.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.578+0000 7f2fbcbad700 1 -- 172.21.15.31:0/2392971247 learned_addr learned my addr 172.21.15.31:0/2392971247 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-08-25T03:33:38.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.578+0000 7f2fbd3ae700 1 -- 172.21.15.31:0/2392971247 >> v2:172.21.15.31:3300/0 conn(0x7f2fb80cfb80 msgr2=0x7f2fb80cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:33:38.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.578+0000 7f2fbd3ae700 1 --2- 172.21.15.31:0/2392971247 >> v2:172.21.15.31:3300/0 conn(0x7f2fb80cfb80 0x7f2fb80cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.578+0000 7f2fbd3ae700 1 -- 172.21.15.31:0/2392971247 >> v1:172.21.15.31:6789/0 conn(0x7f2fb815b170 legacy=0x7f2fb80cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:38.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.578+0000 7f2fbd3ae700 1 -- 172.21.15.31:0/2392971247 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb80d1d50 con 0x7f2fb80cdd30 2023-08-25T03:33:38.583 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.578+0000 7f2fb7fff700 1 --2- 172.21.15.31:0/2392971247 >> v2:172.21.15.31:3300/0 conn(0x7f2fb80cfb80 0x7f2fb80cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:33:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.578+0000 7f2fbd3ae700 1 --2- 172.21.15.31:0/2392971247 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2fb80cdd30 0x7f2fb806d9f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0009d90 tx=0x7f2fb000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a2ef10d96ecf99 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.578+0000 7f2fb77fe700 1 -- 172.21.15.31:0/2392971247 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb000e070 con 0x7f2fb80cdd30 2023-08-25T03:33:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.578+0000 7f2fb77fe700 1 -- 172.21.15.31:0/2392971247 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fb000bb20 con 0x7f2fb80cdd30 2023-08-25T03:33:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.578+0000 7f2fb77fe700 1 -- 172.21.15.31:0/2392971247 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0012430 con 0x7f2fb80cdd30 2023-08-25T03:33:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.579+0000 7f2fbee11700 1 -- 172.21.15.31:0/2392971247 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2fb80cdd30 msgr2=0x7f2fb806d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.579+0000 7f2fbee11700 1 --2- 172.21.15.31:0/2392971247 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2fb80cdd30 0x7f2fb806d9f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0009d90 tx=0x7f2fb000b330 comp rx=0 tx=0).stop 2023-08-25T03:33:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.579+0000 7f2fbee11700 1 -- 172.21.15.31:0/2392971247 shutdown_connections 2023-08-25T03:33:38.584 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.579+0000 7f2fbee11700 1 --2- 172.21.15.31:0/2392971247 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2fb80cdd30 0x7f2fb806d9f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.579+0000 7f2fbee11700 1 --2- 172.21.15.31:0/2392971247 >> v2:172.21.15.31:3300/0 conn(0x7f2fb80cfb80 0x7f2fb80cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.579+0000 7f2fbee11700 1 -- 172.21.15.31:0/2392971247 >> 172.21.15.31:0/2392971247 conn(0x7f2fb809a200 msgr2=0x7f2fb80d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:38.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.579+0000 7f2fbee11700 1 -- 172.21.15.31:0/2392971247 shutdown_connections 2023-08-25T03:33:38.585 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.579+0000 7f2fbee11700 1 -- 172.21.15.31:0/2392971247 wait complete. 2023-08-25T03:33:38.586 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:38.579+0000 7f8f9b465200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-25T03:33:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.580+0000 7f2fbee11700 1 Processor -- start 2023-08-25T03:33:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.580+0000 7f2fbee11700 1 -- start start 2023-08-25T03:33:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.581+0000 7f2fbee11700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fb80cdd30 0x7f2fb806c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.581+0000 7f2fbee11700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2fb80cfb80 0x7f2fb806b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.581+0000 7f2fbee11700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fb815b170 0x7f2fb815d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.581+0000 7f2fbee11700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb806c290 con 0x7f2fb80cdd30 2023-08-25T03:33:38.588 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.581+0000 7f2fbee11700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb806b9f0 con 0x7f2fb815b170 2023-08-25T03:33:38.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.581+0000 7f2fbee11700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb806bcd0 con 0x7f2fb80cfb80 2023-08-25T03:33:38.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.581+0000 7f2fb7fff700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2fb80cfb80 0x7f2fb806b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:38.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.581+0000 7f2fbd3ae700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fb815b170 0x7f2fb815d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:38.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.581+0000 7f2fb7fff700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2fb80cfb80 0x7f2fb806b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.31:58374/0 (socket says 172.21.15.31:58374) 2023-08-25T03:33:38.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.581+0000 7f2fb7fff700 1 -- 172.21.15.31:0/3348471408 learned_addr learned my addr 172.21.15.31:0/3348471408 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-08-25T03:33:38.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.581+0000 7f2fbd3ae700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fb815b170 0x7f2fb815d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41680/0 (socket says 172.21.15.31:41680) 2023-08-25T03:33:38.589 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.581+0000 7f2fbcbad700 1 --2- 172.21.15.31:0/3348471408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fb80cdd30 0x7f2fb806c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:38.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.582+0000 7f2fb7fff700 1 -- 172.21.15.31:0/3348471408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fb815b170 msgr2=0x7f2fb815d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:38.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.582+0000 7f2fb7fff700 1 --2- 172.21.15.31:0/3348471408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fb815b170 0x7f2fb815d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.582+0000 7f2fb7fff700 1 -- 172.21.15.31:0/3348471408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fb80cdd30 msgr2=0x7f2fb806c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:38.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.582+0000 7f2fb7fff700 1 --2- 172.21.15.31:0/3348471408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fb80cdd30 0x7f2fb806c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.582+0000 7f2fb7fff700 1 -- 172.21.15.31:0/3348471408 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa40094a0 con 0x7f2fb80cfb80 2023-08-25T03:33:38.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.582+0000 7f2fbd3ae700 1 --2- 172.21.15.31:0/3348471408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fb815b170 0x7f2fb815d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:33:38.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.582+0000 7f2fbcbad700 1 --2- 172.21.15.31:0/3348471408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fb80cdd30 0x7f2fb806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:33:38.590 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.582+0000 7f2fb7fff700 1 --2- 172.21.15.31:0/3348471408 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2fb80cfb80 0x7f2fb806b2f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400ebe0 tx=0x7f2fa400eef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:38.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.582+0000 7f2fb57fa700 1 -- 172.21.15.31:0/3348471408 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4009800 con 0x7f2fb80cfb80 2023-08-25T03:33:38.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.582+0000 7f2fb57fa700 1 -- 172.21.15.31:0/3348471408 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa400c910 con 0x7f2fb80cfb80 2023-08-25T03:33:38.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.582+0000 7f2fb57fa700 1 -- 172.21.15.31:0/3348471408 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa400cc80 con 0x7f2fb80cfb80 2023-08-25T03:33:38.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.582+0000 7f2fbee11700 1 -- 172.21.15.31:0/3348471408 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb0009a70 con 0x7f2fb80cfb80 2023-08-25T03:33:38.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.582+0000 7f2fbee11700 1 -- 172.21.15.31:0/3348471408 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb81f61c0 con 0x7f2fb80cfb80 2023-08-25T03:33:38.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.582+0000 7f2fb57fa700 1 -- 172.21.15.31:0/3348471408 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2fa4005d20 con 0x7f2fb80cfb80 2023-08-25T03:33:38.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.582+0000 7f2fb57fa700 1 -- 172.21.15.31:0/3348471408 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f2fa4013820 con 0x7f2fb80cfb80 2023-08-25T03:33:38.591 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.583+0000 7f2fbee11700 1 -- 172.21.15.31:0/3348471408 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb81f6020 con 0x7f2fb80cfb80 2023-08-25T03:33:38.592 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.585+0000 7f2fb57fa700 1 -- 172.21.15.31:0/3348471408 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f2fb81f6020 con 0x7f2fb80cfb80 2023-08-25T03:33:38.598 INFO:tasks.ceph.mgr.x.smithi031.stderr:2023-08-25T03:33:38.597+0000 7faad00e4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-08-25T03:33:38.673 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.672+0000 7f2fbee11700 1 -- 172.21.15.31:0/3348471408 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c30c594a-88f7-4be9-9bd0-320bc6489fab", "id": 1} v 0) v1 -- 0x7f2fb80746d0 con 0x7f2fb80cfb80 2023-08-25T03:33:38.676 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.675+0000 7f2fb57fa700 1 -- 172.21.15.31:0/3348471408 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c30c594a-88f7-4be9-9bd0-320bc6489fab", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2fa4021060 con 0x7f2fb80cfb80 2023-08-25T03:33:38.676 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-08-25T03:33:38.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.676+0000 7f2fbee11700 1 -- 172.21.15.31:0/3348471408 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2fb80cfb80 msgr2=0x7f2fb806b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:38.677 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.676+0000 7f2fbee11700 1 --2- 172.21.15.31:0/3348471408 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2fb80cfb80 0x7f2fb806b2f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400ebe0 tx=0x7f2fa400eef0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.676+0000 7f2fbee11700 1 -- 172.21.15.31:0/3348471408 shutdown_connections 2023-08-25T03:33:38.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.676+0000 7f2fbee11700 1 --2- 172.21.15.31:0/3348471408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2fb80cdd30 0x7f2fb806c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.676+0000 7f2fbee11700 1 --2- 172.21.15.31:0/3348471408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fb815b170 0x7f2fb815d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.676+0000 7f2fbee11700 1 --2- 172.21.15.31:0/3348471408 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2fb80cfb80 0x7f2fb806b2f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.676+0000 7f2fbee11700 1 -- 172.21.15.31:0/3348471408 >> 172.21.15.31:0/3348471408 conn(0x7f2fb809a200 msgr2=0x7f2fb815f200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:38.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.676+0000 7f2fbee11700 1 -- 172.21.15.31:0/3348471408 shutdown_connections 2023-08-25T03:33:38.678 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.676+0000 7f2fbee11700 1 -- 172.21.15.31:0/3348471408 wait complete. 2023-08-25T03:33:38.690 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd new adab17e2-ecf8-4c8a-9c9c-26c70154856f 2 2023-08-25T03:33:38.703 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:38.703+0000 7f8f9b465200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-25T03:33:38.726 INFO:tasks.ceph.mgr.x.smithi031.stderr:2023-08-25T03:33:38.725+0000 7faad00e4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-08-25T03:33:38.796 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:38.796+0000 7f8f9b465200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-25T03:33:38.815 INFO:tasks.ceph.mgr.x.smithi031.stderr:2023-08-25T03:33:38.814+0000 7faad00e4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-08-25T03:33:38.816 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.816+0000 7f51cf469700 1 Processor -- start 2023-08-25T03:33:38.817 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.816+0000 7f51cf469700 1 -- start start 2023-08-25T03:33:38.818 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.817+0000 7f51cf469700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51c81607f0 0x7f51c8070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.817+0000 7f51cf469700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f51c80af180 0x7f51c80ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.817+0000 7f51cf469700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f51c806a1e0 0x7f51c806a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.817+0000 7f51cf469700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c80713a0 con 0x7f51c806a1e0 2023-08-25T03:33:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.817+0000 7f51cf469700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c80718a0 con 0x7f51c80af180 2023-08-25T03:33:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.817+0000 7f51cf469700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c806ab70 con 0x7f51c81607f0 2023-08-25T03:33:38.819 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.817+0000 7f51cd205700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51c81607f0 0x7f51c8070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:38.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.817+0000 7f51cd205700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51c81607f0 0x7f51c8070e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.31:58378/0 (socket says 172.21.15.31:58378) 2023-08-25T03:33:38.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.817+0000 7f51cd205700 1 -- 172.21.15.31:0/2790357426 learned_addr learned my addr 172.21.15.31:0/2790357426 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-08-25T03:33:38.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.817+0000 7f51cda06700 1 --2- 172.21.15.31:0/2790357426 >> v2:172.21.15.2:3300/0 conn(0x7f51c806a1e0 0x7f51c806a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:38.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.817+0000 7f51cca04700 1 --2- 172.21.15.31:0/2790357426 >> v2:172.21.15.31:3300/0 conn(0x7f51c80af180 0x7f51c80ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:38.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.817+0000 7f51cd205700 1 -- 172.21.15.31:0/2790357426 >> v2:172.21.15.31:3300/0 conn(0x7f51c80af180 msgr2=0x7f51c80ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:33:38.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.817+0000 7f51cd205700 1 --2- 172.21.15.31:0/2790357426 >> v2:172.21.15.31:3300/0 conn(0x7f51c80af180 0x7f51c80ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.820 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.817+0000 7f51cd205700 1 -- 172.21.15.31:0/2790357426 >> v2:172.21.15.2:3300/0 conn(0x7f51c806a1e0 msgr2=0x7f51c806a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:33:38.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.817+0000 7f51cd205700 1 --2- 172.21.15.31:0/2790357426 >> v2:172.21.15.2:3300/0 conn(0x7f51c806a1e0 0x7f51c806a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.817+0000 7f51cd205700 1 -- 172.21.15.31:0/2790357426 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c8160bd0 con 0x7f51c81607f0 2023-08-25T03:33:38.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.817+0000 7f51cd205700 1 --2- 172.21.15.31:0/2790357426 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51c81607f0 0x7f51c8070e60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f51c806cc00 tx=0x7f51c4009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43468ded0d06991d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:38.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.818+0000 7f51bffff700 1 -- 172.21.15.31:0/2790357426 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c4011830 con 0x7f51c81607f0 2023-08-25T03:33:38.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.818+0000 7f51bffff700 1 -- 172.21.15.31:0/2790357426 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c4011990 con 0x7f51c81607f0 2023-08-25T03:33:38.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.818+0000 7f51bffff700 1 -- 172.21.15.31:0/2790357426 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c4011d20 con 0x7f51c81607f0 2023-08-25T03:33:38.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.818+0000 7f51cf469700 1 -- 172.21.15.31:0/2790357426 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51c81607f0 msgr2=0x7f51c8070e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:38.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.818+0000 7f51cf469700 1 --2- 172.21.15.31:0/2790357426 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51c81607f0 0x7f51c8070e60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f51c806cc00 tx=0x7f51c4009bd0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.818+0000 7f51cf469700 1 -- 172.21.15.31:0/2790357426 shutdown_connections 2023-08-25T03:33:38.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.818+0000 7f51cf469700 1 --2- 172.21.15.31:0/2790357426 >> v2:172.21.15.2:3300/0 conn(0x7f51c806a1e0 0x7f51c806a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.818+0000 7f51cf469700 1 --2- 172.21.15.31:0/2790357426 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51c81607f0 0x7f51c8070e60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.818+0000 7f51cf469700 1 --2- 172.21.15.31:0/2790357426 >> v2:172.21.15.31:3300/0 conn(0x7f51c80af180 0x7f51c80ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.818+0000 7f51cf469700 1 -- 172.21.15.31:0/2790357426 >> 172.21.15.31:0/2790357426 conn(0x7f51c80cfd90 msgr2=0x7f51c80d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:38.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.818+0000 7f51cf469700 1 -- 172.21.15.31:0/2790357426 shutdown_connections 2023-08-25T03:33:38.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.818+0000 7f51cf469700 1 -- 172.21.15.31:0/2790357426 wait complete. 2023-08-25T03:33:38.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.820+0000 7f51cf469700 1 Processor -- start 2023-08-25T03:33:38.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.820+0000 7f51cf469700 1 -- start start 2023-08-25T03:33:38.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.822+0000 7f51cf469700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51c806a1e0 0x7f51c80c5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:38.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.822+0000 7f51cf469700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51c80af180 0x7f51c80c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:38.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.822+0000 7f51cf469700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51c81607f0 0x7f51c80c7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:38.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.822+0000 7f51cf469700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c80c4fc0 con 0x7f51c81607f0 2023-08-25T03:33:38.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.822+0000 7f51cf469700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c80c53e0 con 0x7f51c80af180 2023-08-25T03:33:38.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.822+0000 7f51cf469700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c80c4340 con 0x7f51c806a1e0 2023-08-25T03:33:38.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.822+0000 7f51cca04700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51c80af180 0x7f51c80c38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:38.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.822+0000 7f51cd205700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51c806a1e0 0x7f51c80c5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:38.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.822+0000 7f51cca04700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51c80af180 0x7f51c80c38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41690/0 (socket says 172.21.15.31:41690) 2023-08-25T03:33:38.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.822+0000 7f51cca04700 1 -- 172.21.15.31:0/1289295594 learned_addr learned my addr 172.21.15.31:0/1289295594 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-08-25T03:33:38.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.822+0000 7f51cda06700 1 --2- 172.21.15.31:0/1289295594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51c81607f0 0x7f51c80c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:38.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.822+0000 7f51cd205700 1 -- 172.21.15.31:0/1289295594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51c80af180 msgr2=0x7f51c80c38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:38.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.822+0000 7f51cd205700 1 --2- 172.21.15.31:0/1289295594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51c80af180 0x7f51c80c38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.822+0000 7f51cd205700 1 -- 172.21.15.31:0/1289295594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51c81607f0 msgr2=0x7f51c80c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:38.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.822+0000 7f51cd205700 1 --2- 172.21.15.31:0/1289295594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51c81607f0 0x7f51c80c7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.822+0000 7f51cd205700 1 -- 172.21.15.31:0/1289295594 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c4009550 con 0x7f51c806a1e0 2023-08-25T03:33:38.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.822+0000 7f51cca04700 1 --2- 172.21.15.31:0/1289295594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51c80af180 0x7f51c80c38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:33:38.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.822+0000 7f51cd205700 1 --2- 172.21.15.31:0/1289295594 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51c806a1e0 0x7f51c80c5f70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f51c4016000 tx=0x7f51c4004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:38.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.822+0000 7f51bdffb700 1 -- 172.21.15.31:0/1289295594 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c4004450 con 0x7f51c806a1e0 2023-08-25T03:33:38.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.823+0000 7f51bdffb700 1 -- 172.21.15.31:0/1289295594 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c40045d0 con 0x7f51c806a1e0 2023-08-25T03:33:38.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.823+0000 7f51cf469700 1 -- 172.21.15.31:0/1289295594 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c80c4620 con 0x7f51c806a1e0 2023-08-25T03:33:38.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.823+0000 7f51bdffb700 1 -- 172.21.15.31:0/1289295594 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c40038e0 con 0x7f51c806a1e0 2023-08-25T03:33:38.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.823+0000 7f51cf469700 1 -- 172.21.15.31:0/1289295594 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c80c4be0 con 0x7f51c806a1e0 2023-08-25T03:33:38.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.823+0000 7f51bdffb700 1 -- 172.21.15.31:0/1289295594 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f51c4003d40 con 0x7f51c806a1e0 2023-08-25T03:33:38.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.823+0000 7f51bdffb700 1 -- 172.21.15.31:0/1289295594 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f51c4021030 con 0x7f51c806a1e0 2023-08-25T03:33:38.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.824+0000 7f51cf469700 1 -- 172.21.15.31:0/1289295594 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c80746d0 con 0x7f51c806a1e0 2023-08-25T03:33:38.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.827+0000 7f51bdffb700 1 -- 172.21.15.31:0/1289295594 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f51c4045b00 con 0x7f51c806a1e0 2023-08-25T03:33:38.881 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:38.880+0000 7f8f9b465200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-25T03:33:38.913 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.912+0000 7f51cf469700 1 -- 172.21.15.31:0/1289295594 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "osd new", "uuid": "adab17e2-ecf8-4c8a-9c9c-26c70154856f", "id": 2} v 0) v1 -- 0x7f51c80ceaa0 con 0x7f51c806a1e0 2023-08-25T03:33:38.914 INFO:tasks.ceph.mgr.x.smithi031.stderr:2023-08-25T03:33:38.913+0000 7faad00e4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-08-25T03:33:38.916 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.915+0000 7f51bdffb700 1 -- 172.21.15.31:0/1289295594 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "adab17e2-ecf8-4c8a-9c9c-26c70154856f", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f51c4034030 con 0x7f51c806a1e0 2023-08-25T03:33:38.916 INFO:teuthology.orchestra.run.smithi031.stdout:2 2023-08-25T03:33:38.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.917+0000 7f51cf469700 1 -- 172.21.15.31:0/1289295594 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51c806a1e0 msgr2=0x7f51c80c5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:38.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.917+0000 7f51cf469700 1 --2- 172.21.15.31:0/1289295594 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51c806a1e0 0x7f51c80c5f70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f51c4016000 tx=0x7f51c4004750 comp rx=0 tx=0).stop 2023-08-25T03:33:38.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.917+0000 7f51cf469700 1 -- 172.21.15.31:0/1289295594 shutdown_connections 2023-08-25T03:33:38.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.917+0000 7f51cf469700 1 --2- 172.21.15.31:0/1289295594 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51c81607f0 0x7f51c80c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.918 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.917+0000 7f51cf469700 1 --2- 172.21.15.31:0/1289295594 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51c806a1e0 0x7f51c80c5f70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.917+0000 7f51cf469700 1 --2- 172.21.15.31:0/1289295594 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51c80af180 0x7f51c80c38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:38.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.917+0000 7f51cf469700 1 -- 172.21.15.31:0/1289295594 >> 172.21.15.31:0/1289295594 conn(0x7f51c80cfd90 msgr2=0x7f51c80afb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:38.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.917+0000 7f51cf469700 1 -- 172.21.15.31:0/1289295594 shutdown_connections 2023-08-25T03:33:38.919 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:38.917+0000 7f51cf469700 1 -- 172.21.15.31:0/1289295594 wait complete. 2023-08-25T03:33:38.934 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd new a670f4b5-ba0b-4a48-a845-afdb45e74c5f 3 2023-08-25T03:33:39.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.098+0000 7f597cb54700 1 Processor -- start 2023-08-25T03:33:39.099 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.098+0000 7f597cb54700 1 -- start start 2023-08-25T03:33:39.100 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.099+0000 7f597cb54700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f597815b170 0x7f59780cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:39.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.099+0000 7f597cb54700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f59780cc640 0x7f59780ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:39.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.099+0000 7f597cb54700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59780cf6f0 con 0x7f59780cd7f0 2023-08-25T03:33:39.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.099+0000 7f597cb54700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59780d1d50 con 0x7f597815b170 2023-08-25T03:33:39.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.099+0000 7f597cb54700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597815b550 con 0x7f59780cc640 2023-08-25T03:33:39.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.099+0000 7f5975d9b700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f59780cc640 0x7f59780ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:39.101 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.099+0000 7f5975d9b700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f59780cc640 0x7f59780ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.31:58390/0 (socket says 172.21.15.31:58390) 2023-08-25T03:33:39.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.099+0000 7f5975d9b700 1 -- 172.21.15.31:0/1315259694 learned_addr learned my addr 172.21.15.31:0/1315259694 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-08-25T03:33:39.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.099+0000 7f597659c700 1 --2- 172.21.15.31:0/1315259694 >> v2:172.21.15.2:3300/0 conn(0x7f597815b170 0x7f59780cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:39.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.099+0000 7f5975d9b700 1 -- 172.21.15.31:0/1315259694 >> v2:172.21.15.2:3300/0 conn(0x7f597815b170 msgr2=0x7f59780cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:33:39.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.099+0000 7f5975d9b700 1 --2- 172.21.15.31:0/1315259694 >> v2:172.21.15.2:3300/0 conn(0x7f597815b170 0x7f59780cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.099+0000 7f5975d9b700 1 -- 172.21.15.31:0/1315259694 >> v1:172.21.15.31:6789/0 conn(0x7f59780cd7f0 legacy=0x7f597806d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:39.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.099+0000 7f5975d9b700 1 -- 172.21.15.31:0/1315259694 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59780ceae0 con 0x7f59780cc640 2023-08-25T03:33:39.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.099+0000 7f597559a700 1 -- 172.21.15.31:0/1315259694 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1625229614 0 0) 0x7f59780cf6f0 con 0x7f59780cd7f0 2023-08-25T03:33:39.102 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.099+0000 7f5975d9b700 1 --2- 172.21.15.31:0/1315259694 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f59780cc640 0x7f59780ccac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f596c0098f0 tx=0x7f596c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27b8f65ca1893848 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:39.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.100+0000 7f597559a700 1 -- 172.21.15.31:0/1315259694 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c011830 con 0x7f59780cc640 2023-08-25T03:33:39.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.100+0000 7f597559a700 1 -- 172.21.15.31:0/1315259694 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596c011990 con 0x7f59780cc640 2023-08-25T03:33:39.103 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.100+0000 7f597559a700 1 -- 172.21.15.31:0/1315259694 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c011d20 con 0x7f59780cc640 2023-08-25T03:33:39.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.100+0000 7f597cb54700 1 -- 172.21.15.31:0/1315259694 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f59780cc640 msgr2=0x7f59780ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:39.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.100+0000 7f597cb54700 1 --2- 172.21.15.31:0/1315259694 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f59780cc640 0x7f59780ccac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f596c0098f0 tx=0x7f596c009c00 comp rx=0 tx=0).stop 2023-08-25T03:33:39.105 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.100+0000 7f597cb54700 1 -- 172.21.15.31:0/1315259694 shutdown_connections 2023-08-25T03:33:39.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.100+0000 7f597cb54700 1 --2- 172.21.15.31:0/1315259694 >> v2:172.21.15.2:3300/0 conn(0x7f597815b170 0x7f59780cfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.100+0000 7f597cb54700 1 --2- 172.21.15.31:0/1315259694 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f59780cc640 0x7f59780ccac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.100+0000 7f597cb54700 1 -- 172.21.15.31:0/1315259694 >> 172.21.15.31:0/1315259694 conn(0x7f597809a200 msgr2=0x7f59780d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:39.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.100+0000 7f597cb54700 1 -- 172.21.15.31:0/1315259694 shutdown_connections 2023-08-25T03:33:39.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.100+0000 7f597cb54700 1 -- 172.21.15.31:0/1315259694 wait complete. 2023-08-25T03:33:39.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.102+0000 7f597cb54700 1 Processor -- start 2023-08-25T03:33:39.106 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.102+0000 7f597cb54700 1 -- start start 2023-08-25T03:33:39.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.103+0000 7f597cb54700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f59780cc640 0x7f5978157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:39.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.103+0000 7f597cb54700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59780cd7f0 0x7f597806bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:39.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.103+0000 7f597cb54700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f597815b170 0x7f59780c6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:39.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.103+0000 7f597cb54700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59780c77c0 con 0x7f59780cd7f0 2023-08-25T03:33:39.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.103+0000 7f597cb54700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59780c7200 con 0x7f597815b170 2023-08-25T03:33:39.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.103+0000 7f597cb54700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59780c74e0 con 0x7f59780cc640 2023-08-25T03:33:39.107 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.103+0000 7f597659c700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f59780cc640 0x7f5978157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:39.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.103+0000 7f597659c700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f59780cc640 0x7f5978157690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.31:58396/0 (socket says 172.21.15.31:58396) 2023-08-25T03:33:39.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.103+0000 7f597659c700 1 -- 172.21.15.31:0/115219651 learned_addr learned my addr 172.21.15.31:0/115219651 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-08-25T03:33:39.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.103+0000 7f5976d9d700 1 --2- 172.21.15.31:0/115219651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f597815b170 0x7f59780c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:39.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.103+0000 7f5975d9b700 1 --2- 172.21.15.31:0/115219651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59780cd7f0 0x7f597806bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:39.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.103+0000 7f597659c700 1 -- 172.21.15.31:0/115219651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f597815b170 msgr2=0x7f59780c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:39.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.103+0000 7f597659c700 1 --2- 172.21.15.31:0/115219651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f597815b170 0x7f59780c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.103+0000 7f597659c700 1 -- 172.21.15.31:0/115219651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59780cd7f0 msgr2=0x7f597806bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:39.108 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.103+0000 7f597659c700 1 --2- 172.21.15.31:0/115219651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59780cd7f0 0x7f597806bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.103+0000 7f597659c700 1 -- 172.21.15.31:0/115219651 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c009550 con 0x7f59780cc640 2023-08-25T03:33:39.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.103+0000 7f597659c700 1 --2- 172.21.15.31:0/115219651 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f59780cc640 0x7f5978157690 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f596800e990 tx=0x7f596800eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:39.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.103+0000 7f5966ffd700 1 -- 172.21.15.31:0/115219651 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59680096f0 con 0x7f59780cc640 2023-08-25T03:33:39.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.103+0000 7f5966ffd700 1 -- 172.21.15.31:0/115219651 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596800c690 con 0x7f59780cc640 2023-08-25T03:33:39.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.103+0000 7f597cb54700 1 -- 172.21.15.31:0/115219651 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f597806b420 con 0x7f59780cc640 2023-08-25T03:33:39.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.103+0000 7f5966ffd700 1 -- 172.21.15.31:0/115219651 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596800ca00 con 0x7f59780cc640 2023-08-25T03:33:39.109 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.103+0000 7f597cb54700 1 -- 172.21.15.31:0/115219651 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f597806c160 con 0x7f59780cc640 2023-08-25T03:33:39.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.104+0000 7f5966ffd700 1 -- 172.21.15.31:0/115219651 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5968013470 con 0x7f59780cc640 2023-08-25T03:33:39.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.104+0000 7f5966ffd700 1 -- 172.21.15.31:0/115219651 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f596801ca20 con 0x7f59780cc640 2023-08-25T03:33:39.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.104+0000 7f597cb54700 1 -- 172.21.15.31:0/115219651 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59780746d0 con 0x7f59780cc640 2023-08-25T03:33:39.110 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.106+0000 7f5966ffd700 1 -- 172.21.15.31:0/115219651 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f596803bb00 con 0x7f59780cc640 2023-08-25T03:33:39.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.192+0000 7f597cb54700 1 -- 172.21.15.31:0/115219651 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a670f4b5-ba0b-4a48-a845-afdb45e74c5f", "id": 3} v 0) v1 -- 0x7f59780b6bd0 con 0x7f59780cc640 2023-08-25T03:33:39.196 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.195+0000 7f5966ffd700 1 -- 172.21.15.31:0/115219651 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a670f4b5-ba0b-4a48-a845-afdb45e74c5f", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f596802a030 con 0x7f59780cc640 2023-08-25T03:33:39.196 INFO:teuthology.orchestra.run.smithi031.stdout:3 2023-08-25T03:33:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.196+0000 7f597cb54700 1 -- 172.21.15.31:0/115219651 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f59780cc640 msgr2=0x7f5978157690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.196+0000 7f597cb54700 1 --2- 172.21.15.31:0/115219651 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f59780cc640 0x7f5978157690 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f596800e990 tx=0x7f596800eca0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.196+0000 7f597cb54700 1 -- 172.21.15.31:0/115219651 shutdown_connections 2023-08-25T03:33:39.197 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.196+0000 7f597cb54700 1 --2- 172.21.15.31:0/115219651 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f597815b170 0x7f59780c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.196+0000 7f597cb54700 1 --2- 172.21.15.31:0/115219651 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f59780cc640 0x7f5978157690 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.196+0000 7f597cb54700 1 --2- 172.21.15.31:0/115219651 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f59780cd7f0 0x7f597806bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.196+0000 7f597cb54700 1 -- 172.21.15.31:0/115219651 >> 172.21.15.31:0/115219651 conn(0x7f597809a200 msgr2=0x7f59780afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:39.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.196+0000 7f597cb54700 1 -- 172.21.15.31:0/115219651 shutdown_connections 2023-08-25T03:33:39.198 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.196+0000 7f597cb54700 1 -- 172.21.15.31:0/115219651 wait complete. 2023-08-25T03:33:39.209 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd new 534f7781-ab2e-4166-91fe-efd954dc1345 4 2023-08-25T03:33:39.323 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:39.322+0000 7f8f9b465200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-25T03:33:39.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.332+0000 7f5981812700 1 Processor -- start 2023-08-25T03:33:39.333 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.332+0000 7f5981812700 1 -- start start 2023-08-25T03:33:39.334 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.333+0000 7f5981812700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f597c15b170 0x7f597c0cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:39.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.333+0000 7f5981812700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f597c0cf6f0 con 0x7f597c0cdf00 2023-08-25T03:33:39.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.333+0000 7f5981812700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f597c0d1d50 con 0x7f597c0cc640 2023-08-25T03:33:39.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.333+0000 7f5981812700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f597c15b550 con 0x7f597c15b170 2023-08-25T03:33:39.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.333+0000 7f597a7fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f597c0cc640 0x7f597c0cd7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43102/0 (socket says 172.21.15.31:43102) 2023-08-25T03:33:39.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.333+0000 7f597a7fc700 1 -- 172.21.15.31:0/2051356669 learned_addr learned my addr 172.21.15.31:0/2051356669 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-08-25T03:33:39.335 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.333+0000 7f597affd700 1 --2- 172.21.15.31:0/2051356669 >> v2:172.21.15.2:3300/0 conn(0x7f597c15b170 0x7f597c0cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:39.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.333+0000 7f5979ffb700 1 -- 172.21.15.31:0/2051356669 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3107733459 0 0) 0x7f597c0d1d50 con 0x7f597c0cc640 2023-08-25T03:33:39.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.334+0000 7f5979ffb700 1 -- 172.21.15.31:0/2051356669 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59640035b0 con 0x7f597c0cc640 2023-08-25T03:33:39.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.334+0000 7f5979ffb700 1 -- 172.21.15.31:0/2051356669 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 233923343 0 0) 0x7f597c0cf6f0 con 0x7f597c0cdf00 2023-08-25T03:33:39.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.334+0000 7f5979ffb700 1 -- 172.21.15.31:0/2051356669 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f597c0d1d50 con 0x7f597c0cdf00 2023-08-25T03:33:39.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.334+0000 7f597affd700 1 -- 172.21.15.31:0/2051356669 >> v1:172.21.15.31:6789/0 conn(0x7f597c0cc640 legacy=0x7f597c0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:39.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.334+0000 7f597affd700 1 -- 172.21.15.31:0/2051356669 >> v1:172.21.15.2:6789/0 conn(0x7f597c0cdf00 legacy=0x7f597c06d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:39.336 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.334+0000 7f597affd700 1 -- 172.21.15.31:0/2051356669 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f597c0ceae0 con 0x7f597c15b170 2023-08-25T03:33:39.337 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.334+0000 7f597affd700 1 --2- 172.21.15.31:0/2051356669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f597c15b170 0x7f597c0cfb80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f596c009820 tx=0x7f596c009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a560445c10d18d6b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:39.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.334+0000 7f5979ffb700 1 -- 172.21.15.31:0/2051356669 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c005e30 con 0x7f597c15b170 2023-08-25T03:33:39.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.334+0000 7f5979ffb700 1 -- 172.21.15.31:0/2051356669 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596c006030 con 0x7f597c15b170 2023-08-25T03:33:39.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.334+0000 7f5979ffb700 1 -- 172.21.15.31:0/2051356669 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c0063c0 con 0x7f597c15b170 2023-08-25T03:33:39.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.335+0000 7f5981812700 1 -- 172.21.15.31:0/2051356669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f597c15b170 msgr2=0x7f597c0cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:39.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.335+0000 7f5981812700 1 --2- 172.21.15.31:0/2051356669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f597c15b170 0x7f597c0cfb80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f596c009820 tx=0x7f596c009b30 comp rx=0 tx=0).stop 2023-08-25T03:33:39.338 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.335+0000 7f5981812700 1 -- 172.21.15.31:0/2051356669 shutdown_connections 2023-08-25T03:33:39.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.335+0000 7f5981812700 1 --2- 172.21.15.31:0/2051356669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f597c15b170 0x7f597c0cfb80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.335+0000 7f5981812700 1 -- 172.21.15.31:0/2051356669 >> 172.21.15.31:0/2051356669 conn(0x7f597c09a200 msgr2=0x7f597c0d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:39.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.335+0000 7f5981812700 1 -- 172.21.15.31:0/2051356669 shutdown_connections 2023-08-25T03:33:39.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.335+0000 7f5981812700 1 -- 172.21.15.31:0/2051356669 wait complete. 2023-08-25T03:33:39.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.336+0000 7f5981812700 1 Processor -- start 2023-08-25T03:33:39.339 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.337+0000 7f5981812700 1 -- start start 2023-08-25T03:33:39.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.337+0000 7f5981812700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f597c0cc640 0x7f597c15c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:39.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.338+0000 7f5981812700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f597c0cdf00 0x7f597c1f5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:39.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.338+0000 7f5981812700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f597c15b170 0x7f597c1f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:39.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.338+0000 7f5981812700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597c06b800 con 0x7f597c15b170 2023-08-25T03:33:39.341 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.338+0000 7f5981812700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f597c15ce90 con 0x7f597c0cdf00 2023-08-25T03:33:39.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.338+0000 7f5981812700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597c15d170 con 0x7f597c0cc640 2023-08-25T03:33:39.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.338+0000 7f597a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f597c0cdf00 0x7f597c1f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:39.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.338+0000 7f597a7fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f597c0cdf00 0x7f597c1f5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41714/0 (socket says 172.21.15.31:41714) 2023-08-25T03:33:39.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.338+0000 7f597a7fc700 1 -- 172.21.15.31:0/1036365508 learned_addr learned my addr 172.21.15.31:0/1036365508 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-08-25T03:33:39.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.338+0000 7f597affd700 1 --2- 172.21.15.31:0/1036365508 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f597c0cc640 0x7f597c15c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:39.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.338+0000 7f597b7fe700 1 --2- 172.21.15.31:0/1036365508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f597c15b170 0x7f597c1f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:39.342 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.338+0000 7f597affd700 1 -- 172.21.15.31:0/1036365508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f597c0cdf00 msgr2=0x7f597c1f5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:39.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.338+0000 7f597affd700 1 --2- 172.21.15.31:0/1036365508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f597c0cdf00 0x7f597c1f5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.338+0000 7f597affd700 1 -- 172.21.15.31:0/1036365508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f597c15b170 msgr2=0x7f597c1f5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:39.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.338+0000 7f597affd700 1 --2- 172.21.15.31:0/1036365508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f597c15b170 0x7f597c1f5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.338+0000 7f597affd700 1 -- 172.21.15.31:0/1036365508 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c009430 con 0x7f597c0cc640 2023-08-25T03:33:39.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.338+0000 7f597a7fc700 1 --2- 172.21.15.31:0/1036365508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f597c0cdf00 0x7f597c1f5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:33:39.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.338+0000 7f597affd700 1 --2- 172.21.15.31:0/1036365508 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f597c0cc640 0x7f597c15c590 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f596c015000 tx=0x7f596c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:39.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.338+0000 7f5963fff700 1 -- 172.21.15.31:0/1036365508 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c0067f0 con 0x7f597c0cc640 2023-08-25T03:33:39.343 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.338+0000 7f5963fff700 1 -- 172.21.15.31:0/1036365508 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596c020070 con 0x7f597c0cc640 2023-08-25T03:33:39.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.338+0000 7f5981812700 1 -- 172.21.15.31:0/1036365508 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f597c06b490 con 0x7f597c0cc640 2023-08-25T03:33:39.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.338+0000 7f5963fff700 1 -- 172.21.15.31:0/1036365508 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596c025670 con 0x7f597c0cc640 2023-08-25T03:33:39.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.339+0000 7f5981812700 1 -- 172.21.15.31:0/1036365508 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f597c0b08d0 con 0x7f597c0cc640 2023-08-25T03:33:39.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.339+0000 7f5963fff700 1 -- 172.21.15.31:0/1036365508 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f596c022030 con 0x7f597c0cc640 2023-08-25T03:33:39.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.339+0000 7f5963fff700 1 -- 172.21.15.31:0/1036365508 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f596c011930 con 0x7f597c0cc640 2023-08-25T03:33:39.344 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.340+0000 7f5981812700 1 -- 172.21.15.31:0/1036365508 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f597c06b2f0 con 0x7f597c0cc640 2023-08-25T03:33:39.345 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.342+0000 7f5963fff700 1 -- 172.21.15.31:0/1036365508 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f597c06b2f0 con 0x7f597c0cc640 2023-08-25T03:33:39.382 INFO:tasks.ceph.mgr.x.smithi031.stderr:2023-08-25T03:33:39.382+0000 7faad00e4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-08-25T03:33:39.427 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.426+0000 7f5981812700 1 -- 172.21.15.31:0/1036365508 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "osd new", "uuid": "534f7781-ab2e-4166-91fe-efd954dc1345", "id": 4} v 0) v1 -- 0x7f597c0746d0 con 0x7f597c0cc640 2023-08-25T03:33:39.430 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.429+0000 7f5963fff700 1 -- 172.21.15.31:0/1036365508 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "534f7781-ab2e-4166-91fe-efd954dc1345", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f596c025c10 con 0x7f597c0cc640 2023-08-25T03:33:39.431 INFO:teuthology.orchestra.run.smithi031.stdout:4 2023-08-25T03:33:39.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.430+0000 7f5981812700 1 -- 172.21.15.31:0/1036365508 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f597c0cc640 msgr2=0x7f597c15c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:39.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.430+0000 7f5981812700 1 --2- 172.21.15.31:0/1036365508 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f597c0cc640 0x7f597c15c590 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f596c015000 tx=0x7f596c004750 comp rx=0 tx=0).stop 2023-08-25T03:33:39.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.431+0000 7f5981812700 1 -- 172.21.15.31:0/1036365508 shutdown_connections 2023-08-25T03:33:39.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.431+0000 7f5981812700 1 --2- 172.21.15.31:0/1036365508 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f597c15b170 0x7f597c1f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.432 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.431+0000 7f5981812700 1 --2- 172.21.15.31:0/1036365508 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f597c0cc640 0x7f597c15c590 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.431+0000 7f5981812700 1 --2- 172.21.15.31:0/1036365508 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f597c0cdf00 0x7f597c1f5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.431+0000 7f5981812700 1 -- 172.21.15.31:0/1036365508 >> 172.21.15.31:0/1036365508 conn(0x7f597c09a200 msgr2=0x7f597c0afb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:39.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.431+0000 7f5981812700 1 -- 172.21.15.31:0/1036365508 shutdown_connections 2023-08-25T03:33:39.433 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.431+0000 7f5981812700 1 -- 172.21.15.31:0/1036365508 wait complete. 2023-08-25T03:33:39.445 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd new c78da803-e437-4ab8-9158-e3f41948e55d 5 2023-08-25T03:33:39.574 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.568+0000 7f095a751700 1 Processor -- start 2023-08-25T03:33:39.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.568+0000 7f095a751700 1 -- start start 2023-08-25T03:33:39.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.569+0000 7f095a751700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f09540cfd80 0x7f095406bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:39.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.569+0000 7f095a751700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09540d1b40 con 0x7f095415d360 2023-08-25T03:33:39.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.569+0000 7f095a751700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f095406e0e0 con 0x7f095406e570 2023-08-25T03:33:39.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.569+0000 7f095a751700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09540d0620 con 0x7f09540cfd80 2023-08-25T03:33:39.575 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.569+0000 7f0958cee700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f095406e570 0x7f095406ea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43116/0 (socket says 172.21.15.31:43116) 2023-08-25T03:33:39.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.569+0000 7f0958cee700 1 -- 172.21.15.31:0/3056347437 learned_addr learned my addr 172.21.15.31:0/3056347437 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-08-25T03:33:39.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.569+0000 7f09537fe700 1 --2- 172.21.15.31:0/3056347437 >> v2:172.21.15.2:3300/0 conn(0x7f09540cfd80 0x7f095406bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:39.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.569+0000 7f0952ffd700 1 -- 172.21.15.31:0/3056347437 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1549560500 0 0) 0x7f095406e0e0 con 0x7f095406e570 2023-08-25T03:33:39.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.569+0000 7f0952ffd700 1 -- 172.21.15.31:0/3056347437 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f093c0035b0 con 0x7f095406e570 2023-08-25T03:33:39.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.569+0000 7f0952ffd700 1 -- 172.21.15.31:0/3056347437 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1797418942 0 0) 0x7f09540d1b40 con 0x7f095415d360 2023-08-25T03:33:39.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.569+0000 7f0952ffd700 1 -- 172.21.15.31:0/3056347437 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f095406e0e0 con 0x7f095415d360 2023-08-25T03:33:39.576 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.569+0000 7f09537fe700 1 -- 172.21.15.31:0/3056347437 >> v1:172.21.15.31:6789/0 conn(0x7f095406e570 legacy=0x7f095406ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.569+0000 7f09537fe700 1 -- 172.21.15.31:0/3056347437 >> v1:172.21.15.2:6789/0 conn(0x7f095415d360 legacy=0x7f095406b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.569+0000 7f09537fe700 1 -- 172.21.15.31:0/3056347437 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09540d0340 con 0x7f09540cfd80 2023-08-25T03:33:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.570+0000 7f09537fe700 1 --2- 172.21.15.31:0/3056347437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09540cfd80 0x7f095406bc10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0948009840 tx=0x7f0948009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1448f8bb348936f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.570+0000 7f0952ffd700 1 -- 172.21.15.31:0/3056347437 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0948011830 con 0x7f09540cfd80 2023-08-25T03:33:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.570+0000 7f0952ffd700 1 -- 172.21.15.31:0/3056347437 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0948011990 con 0x7f09540cfd80 2023-08-25T03:33:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.570+0000 7f0952ffd700 1 -- 172.21.15.31:0/3056347437 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0948011d20 con 0x7f09540cfd80 2023-08-25T03:33:39.577 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.570+0000 7f095a751700 1 -- 172.21.15.31:0/3056347437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09540cfd80 msgr2=0x7f095406bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:39.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.570+0000 7f095a751700 1 --2- 172.21.15.31:0/3056347437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09540cfd80 0x7f095406bc10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0948009840 tx=0x7f0948009b50 comp rx=0 tx=0).stop 2023-08-25T03:33:39.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.570+0000 7f095a751700 1 -- 172.21.15.31:0/3056347437 shutdown_connections 2023-08-25T03:33:39.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.570+0000 7f095a751700 1 --2- 172.21.15.31:0/3056347437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09540cfd80 0x7f095406bc10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.570+0000 7f095a751700 1 -- 172.21.15.31:0/3056347437 >> 172.21.15.31:0/3056347437 conn(0x7f095409a200 msgr2=0x7f09540d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:39.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.570+0000 7f095a751700 1 -- 172.21.15.31:0/3056347437 shutdown_connections 2023-08-25T03:33:39.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.570+0000 7f095a751700 1 -- 172.21.15.31:0/3056347437 wait complete. 2023-08-25T03:33:39.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.572+0000 7f095a751700 1 Processor -- start 2023-08-25T03:33:39.578 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.572+0000 7f095a751700 1 -- start start 2023-08-25T03:33:39.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.572+0000 7f095a751700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f095406e570 0x7f09541f2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:39.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.572+0000 7f095a751700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09540cfd80 0x7f09541f5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:39.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.572+0000 7f095a751700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f095415d360 0x7f09541f1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:39.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.572+0000 7f095a751700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09541f6a80 con 0x7f09540cfd80 2023-08-25T03:33:39.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.572+0000 7f095a751700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095409cd80 con 0x7f095415d360 2023-08-25T03:33:39.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.572+0000 7f095a751700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09541f5fd0 con 0x7f095406e570 2023-08-25T03:33:39.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.573+0000 7f0958cee700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f095415d360 0x7f09541f1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:39.579 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.573+0000 7f0953fff700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f095406e570 0x7f09541f2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:39.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.573+0000 7f0953fff700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f095406e570 0x7f09541f2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.31:58420/0 (socket says 172.21.15.31:58420) 2023-08-25T03:33:39.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.573+0000 7f0958cee700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f095415d360 0x7f09541f1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41728/0 (socket says 172.21.15.31:41728) 2023-08-25T03:33:39.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.573+0000 7f0953fff700 1 -- 172.21.15.31:0/1160219246 learned_addr learned my addr 172.21.15.31:0/1160219246 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-08-25T03:33:39.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.573+0000 7f09537fe700 1 --2- 172.21.15.31:0/1160219246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09540cfd80 0x7f09541f5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:39.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.573+0000 7f0953fff700 1 -- 172.21.15.31:0/1160219246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f095415d360 msgr2=0x7f09541f1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:39.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.573+0000 7f0953fff700 1 --2- 172.21.15.31:0/1160219246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f095415d360 0x7f09541f1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.580 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.573+0000 7f0953fff700 1 -- 172.21.15.31:0/1160219246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09540cfd80 msgr2=0x7f09541f5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:39.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.573+0000 7f0953fff700 1 --2- 172.21.15.31:0/1160219246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09540cfd80 0x7f09541f5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.573+0000 7f0953fff700 1 -- 172.21.15.31:0/1160219246 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09480094a0 con 0x7f095406e570 2023-08-25T03:33:39.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.573+0000 7f0953fff700 1 --2- 172.21.15.31:0/1160219246 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f095406e570 0x7f09541f2ff0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f094400c700 tx=0x7f094400ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:39.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.573+0000 7f0950ff9700 1 -- 172.21.15.31:0/1160219246 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09440052d0 con 0x7f095406e570 2023-08-25T03:33:39.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.573+0000 7f0950ff9700 1 -- 172.21.15.31:0/1160219246 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0944005430 con 0x7f095406e570 2023-08-25T03:33:39.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.573+0000 7f095a751700 1 -- 172.21.15.31:0/1160219246 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09541f23b0 con 0x7f095406e570 2023-08-25T03:33:39.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.573+0000 7f0950ff9700 1 -- 172.21.15.31:0/1160219246 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09440038f0 con 0x7f095406e570 2023-08-25T03:33:39.581 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.573+0000 7f095a751700 1 -- 172.21.15.31:0/1160219246 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09541f61e0 con 0x7f095406e570 2023-08-25T03:33:39.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.573+0000 7f0950ff9700 1 -- 172.21.15.31:0/1160219246 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0944003d50 con 0x7f095406e570 2023-08-25T03:33:39.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.573+0000 7f0950ff9700 1 -- 172.21.15.31:0/1160219246 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f0944011db0 con 0x7f095406e570 2023-08-25T03:33:39.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.574+0000 7f095a751700 1 -- 172.21.15.31:0/1160219246 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09540746d0 con 0x7f095406e570 2023-08-25T03:33:39.582 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.576+0000 7f0950ff9700 1 -- 172.21.15.31:0/1160219246 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f09541f23b0 con 0x7f095406e570 2023-08-25T03:33:39.663 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.662+0000 7f095a751700 1 -- 172.21.15.31:0/1160219246 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c78da803-e437-4ab8-9158-e3f41948e55d", "id": 5} v 0) v1 -- 0x7f09540ce620 con 0x7f095406e570 2023-08-25T03:33:39.666 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.665+0000 7f0950ff9700 1 -- 172.21.15.31:0/1160219246 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c78da803-e437-4ab8-9158-e3f41948e55d", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0944017070 con 0x7f095406e570 2023-08-25T03:33:39.666 INFO:teuthology.orchestra.run.smithi031.stdout:5 2023-08-25T03:33:39.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.666+0000 7f095a751700 1 -- 172.21.15.31:0/1160219246 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f095406e570 msgr2=0x7f09541f2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:39.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.666+0000 7f095a751700 1 --2- 172.21.15.31:0/1160219246 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f095406e570 0x7f09541f2ff0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f094400c700 tx=0x7f094400ca10 comp rx=0 tx=0).stop 2023-08-25T03:33:39.667 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.666+0000 7f095a751700 1 -- 172.21.15.31:0/1160219246 shutdown_connections 2023-08-25T03:33:39.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.666+0000 7f095a751700 1 --2- 172.21.15.31:0/1160219246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f095415d360 0x7f09541f1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.666+0000 7f095a751700 1 --2- 172.21.15.31:0/1160219246 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f095406e570 0x7f09541f2ff0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.666+0000 7f095a751700 1 --2- 172.21.15.31:0/1160219246 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09540cfd80 0x7f09541f5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.666+0000 7f095a751700 1 -- 172.21.15.31:0/1160219246 >> 172.21.15.31:0/1160219246 conn(0x7f095409a200 msgr2=0x7f09540b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:39.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.666+0000 7f095a751700 1 -- 172.21.15.31:0/1160219246 shutdown_connections 2023-08-25T03:33:39.668 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.666+0000 7f095a751700 1 -- 172.21.15.31:0/1160219246 wait complete. 2023-08-25T03:33:39.679 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd new c3989d6e-cf32-4380-9c0f-be95f940c268 6 2023-08-25T03:33:39.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.820+0000 7f6bc94a4700 1 Processor -- start 2023-08-25T03:33:39.821 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.820+0000 7f6bc94a4700 1 -- start start 2023-08-25T03:33:39.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.821+0000 7f6bc94a4700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f6bc415d2f0 0x7f6bc40d1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:39.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.821+0000 7f6bc94a4700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6bc415d6d0 0x7f6bc40b6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:39.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.821+0000 7f6bc94a4700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bc40cead0 con 0x7f6bc40d2510 2023-08-25T03:33:39.822 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.821+0000 7f6bc94a4700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bc40d1b40 con 0x7f6bc415d2f0 2023-08-25T03:33:39.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.821+0000 7f6bc94a4700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc40ce7f0 con 0x7f6bc415d6d0 2023-08-25T03:33:39.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.821+0000 7f6bc2ffd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f6bc415d2f0 0x7f6bc40d1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:39.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.821+0000 7f6bc27fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6bc40d2510 0x7f6bc40cf860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.31:43128/0 (socket says 172.21.15.31:43128) 2023-08-25T03:33:39.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.821+0000 7f6bc27fc700 1 -- 172.21.15.31:0/1481856397 learned_addr learned my addr 172.21.15.31:0/1481856397 (peer_addr_for_me v1:172.21.15.31:0/0) 2023-08-25T03:33:39.823 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.821+0000 7f6bc37fe700 1 --2- 172.21.15.31:0/1481856397 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6bc415d6d0 0x7f6bc40b6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.822+0000 7f6bc1ffb700 1 -- 172.21.15.31:0/1481856397 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 326300717 0 0) 0x7f6bc40cead0 con 0x7f6bc40d2510 2023-08-25T03:33:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.822+0000 7f6bc37fe700 1 -- 172.21.15.31:0/1481856397 >> v2:172.21.15.2:3300/0 conn(0x7f6bc415d2f0 msgr2=0x7f6bc40d1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:33:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.822+0000 7f6bc37fe700 1 --2- 172.21.15.31:0/1481856397 >> v2:172.21.15.2:3300/0 conn(0x7f6bc415d2f0 0x7f6bc40d1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.822+0000 7f6bc37fe700 1 -- 172.21.15.31:0/1481856397 >> v1:172.21.15.31:6789/0 conn(0x7f6bc40d2510 legacy=0x7f6bc40cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:39.824 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.822+0000 7f6bc37fe700 1 -- 172.21.15.31:0/1481856397 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc40d14f0 con 0x7f6bc415d6d0 2023-08-25T03:33:39.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.822+0000 7f6bc37fe700 1 --2- 172.21.15.31:0/1481856397 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6bc415d6d0 0x7f6bc40b6510 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009840 tx=0x7f6bbc009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=419a5111ee18ad72 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:39.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.822+0000 7f6bc1ffb700 1 -- 172.21.15.31:0/1481856397 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc011830 con 0x7f6bc415d6d0 2023-08-25T03:33:39.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.822+0000 7f6bc1ffb700 1 -- 172.21.15.31:0/1481856397 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bbc011990 con 0x7f6bc415d6d0 2023-08-25T03:33:39.825 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.822+0000 7f6bc1ffb700 1 -- 172.21.15.31:0/1481856397 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc011d20 con 0x7f6bc415d6d0 2023-08-25T03:33:39.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.823+0000 7f6bc94a4700 1 -- 172.21.15.31:0/1481856397 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6bc415d6d0 msgr2=0x7f6bc40b6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:39.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.823+0000 7f6bc94a4700 1 --2- 172.21.15.31:0/1481856397 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6bc415d6d0 0x7f6bc40b6510 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009840 tx=0x7f6bbc009b50 comp rx=0 tx=0).stop 2023-08-25T03:33:39.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.823+0000 7f6bc94a4700 1 -- 172.21.15.31:0/1481856397 shutdown_connections 2023-08-25T03:33:39.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.823+0000 7f6bc94a4700 1 --2- 172.21.15.31:0/1481856397 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6bc415d6d0 0x7f6bc40b6510 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.823+0000 7f6bc94a4700 1 --2- 172.21.15.31:0/1481856397 >> v2:172.21.15.2:3300/0 conn(0x7f6bc415d2f0 0x7f6bc40d1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.824+0000 7f6bc94a4700 1 -- 172.21.15.31:0/1481856397 >> 172.21.15.31:0/1481856397 conn(0x7f6bc406ab30 msgr2=0x7f6bc406af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:39.826 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.824+0000 7f6bc94a4700 1 -- 172.21.15.31:0/1481856397 shutdown_connections 2023-08-25T03:33:39.827 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.824+0000 7f6bc94a4700 1 -- 172.21.15.31:0/1481856397 wait complete. 2023-08-25T03:33:39.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.826+0000 7f6bc94a4700 1 Processor -- start 2023-08-25T03:33:39.828 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.826+0000 7f6bc94a4700 1 -- start start 2023-08-25T03:33:39.829 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.828+0000 7f6bc94a4700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6bc40d2510 0x7f6bc40b3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:39.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.828+0000 7f6bc94a4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bc415d2f0 0x7f6bc4159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:39.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.828+0000 7f6bc94a4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc415d6d0 0x7f6bc4151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:39.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.828+0000 7f6bc94a4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc40b4580 con 0x7f6bc415d6d0 2023-08-25T03:33:39.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.828+0000 7f6bc27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bc415d2f0 0x7f6bc4159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:39.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.828+0000 7f6bc37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc415d6d0 0x7f6bc4151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:39.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.828+0000 7f6bc2ffd700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6bc40d2510 0x7f6bc40b3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:39.830 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.828+0000 7f6bc27fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bc415d2f0 0x7f6bc4159020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41736/0 (socket says 172.21.15.31:41736) 2023-08-25T03:33:39.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.828+0000 7f6bc27fc700 1 -- 172.21.15.31:0/4274131404 learned_addr learned my addr 172.21.15.31:0/4274131404 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-08-25T03:33:39.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.829+0000 7f6bc94a4700 1 -- 172.21.15.31:0/4274131404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc4159560 con 0x7f6bc415d2f0 2023-08-25T03:33:39.831 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.829+0000 7f6bc94a4700 1 -- 172.21.15.31:0/4274131404 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc40b4820 con 0x7f6bc40d2510 2023-08-25T03:33:39.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.829+0000 7f6bc27fc700 1 -- 172.21.15.31:0/4274131404 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6bc40d2510 msgr2=0x7f6bc40b3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:39.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.829+0000 7f6bc27fc700 1 --2- 172.21.15.31:0/4274131404 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6bc40d2510 0x7f6bc40b3bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.829+0000 7f6bc27fc700 1 -- 172.21.15.31:0/4274131404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc415d6d0 msgr2=0x7f6bc4151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:39.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.829+0000 7f6bc27fc700 1 --2- 172.21.15.31:0/4274131404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc415d6d0 0x7f6bc4151480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.829+0000 7f6bc27fc700 1 -- 172.21.15.31:0/4274131404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bbc0094a0 con 0x7f6bc415d2f0 2023-08-25T03:33:39.832 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.829+0000 7f6bc37fe700 1 --2- 172.21.15.31:0/4274131404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc415d6d0 0x7f6bc4151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:33:39.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.830+0000 7f6bc27fc700 1 --2- 172.21.15.31:0/4274131404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bc415d2f0 0x7f6bc4159020 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8001eb0 tx=0x7f6bb800c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:39.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.830+0000 7f6bab7fe700 1 -- 172.21.15.31:0/4274131404 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8004c80 con 0x7f6bc415d2f0 2023-08-25T03:33:39.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.830+0000 7f6bab7fe700 1 -- 172.21.15.31:0/4274131404 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb8004de0 con 0x7f6bc415d2f0 2023-08-25T03:33:39.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.830+0000 7f6bc94a4700 1 -- 172.21.15.31:0/4274131404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc40b4b00 con 0x7f6bc415d2f0 2023-08-25T03:33:39.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.830+0000 7f6bab7fe700 1 -- 172.21.15.31:0/4274131404 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8007310 con 0x7f6bc415d2f0 2023-08-25T03:33:39.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.830+0000 7f6bc94a4700 1 -- 172.21.15.31:0/4274131404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc40b4240 con 0x7f6bc415d2f0 2023-08-25T03:33:39.833 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.831+0000 7f6bab7fe700 1 -- 172.21.15.31:0/4274131404 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6bb801d3f0 con 0x7f6bc415d2f0 2023-08-25T03:33:39.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.831+0000 7f6bab7fe700 1 -- 172.21.15.31:0/4274131404 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f6bb80139b0 con 0x7f6bc415d2f0 2023-08-25T03:33:39.834 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.832+0000 7f6bc94a4700 1 -- 172.21.15.31:0/4274131404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bc40746d0 con 0x7f6bc415d2f0 2023-08-25T03:33:39.838 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.837+0000 7f6bab7fe700 1 -- 172.21.15.31:0/4274131404 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f6bb8013bc0 con 0x7f6bc415d2f0 2023-08-25T03:33:39.936 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.935+0000 7f6bc94a4700 1 -- 172.21.15.31:0/4274131404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c3989d6e-cf32-4380-9c0f-be95f940c268", "id": 6} v 0) v1 -- 0x7f6bc40d0f70 con 0x7f6bc415d2f0 2023-08-25T03:33:39.944 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.943+0000 7f6bab7fe700 1 -- 172.21.15.31:0/4274131404 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c3989d6e-cf32-4380-9c0f-be95f940c268", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6bb8013e30 con 0x7f6bc415d2f0 2023-08-25T03:33:39.944 INFO:teuthology.orchestra.run.smithi031.stdout:6 2023-08-25T03:33:39.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.944+0000 7f6bc94a4700 1 -- 172.21.15.31:0/4274131404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bc415d2f0 msgr2=0x7f6bc4159020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:39.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.944+0000 7f6bc94a4700 1 --2- 172.21.15.31:0/4274131404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bc415d2f0 0x7f6bc4159020 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8001eb0 tx=0x7f6bb800c800 comp rx=0 tx=0).stop 2023-08-25T03:33:39.945 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.944+0000 7f6bc94a4700 1 -- 172.21.15.31:0/4274131404 shutdown_connections 2023-08-25T03:33:39.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.944+0000 7f6bc94a4700 1 --2- 172.21.15.31:0/4274131404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6bc415d6d0 0x7f6bc4151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.944+0000 7f6bc94a4700 1 --2- 172.21.15.31:0/4274131404 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6bc40d2510 0x7f6bc40b3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.946 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.944+0000 7f6bc94a4700 1 --2- 172.21.15.31:0/4274131404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bc415d2f0 0x7f6bc4159020 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:39.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.944+0000 7f6bc94a4700 1 -- 172.21.15.31:0/4274131404 >> 172.21.15.31:0/4274131404 conn(0x7f6bc406ab30 msgr2=0x7f6bc40b86e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:39.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.944+0000 7f6bc94a4700 1 -- 172.21.15.31:0/4274131404 shutdown_connections 2023-08-25T03:33:39.947 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:39.944+0000 7f6bc94a4700 1 -- 172.21.15.31:0/4274131404 wait complete. 2023-08-25T03:33:39.958 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd new fab11910-b768-4930-addd-8cf4e2ef03dd 7 2023-08-25T03:33:39.996 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:39.996+0000 7f8f9b465200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-25T03:33:40.072 INFO:tasks.ceph.mgr.x.smithi031.stderr:2023-08-25T03:33:40.071+0000 7faad00e4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-08-25T03:33:40.075 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:40.067+0000 7f8f9b465200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-25T03:33:40.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.084+0000 7f40ee265700 1 Processor -- start 2023-08-25T03:33:40.085 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.084+0000 7f40ee265700 1 -- start start 2023-08-25T03:33:40.086 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.084+0000 7f40ee265700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f40e80af180 0x7f40e8070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:40.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.085+0000 7f40ee265700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40e8160bd0 con 0x7f40e81607f0 2023-08-25T03:33:40.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.085+0000 7f40ee265700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40e80cdd00 con 0x7f40e806a1e0 2023-08-25T03:33:40.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.085+0000 7f40ee265700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e80713a0 con 0x7f40e80af180 2023-08-25T03:33:40.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.085+0000 7f40e77fe700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f40e80af180 0x7f40e8070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:40.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.085+0000 7f40e77fe700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f40e80af180 0x7f40e8070e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.31:58440/0 (socket says 172.21.15.31:58440) 2023-08-25T03:33:40.087 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.085+0000 7f40e77fe700 1 -- 172.21.15.31:0/2895057374 learned_addr learned my addr 172.21.15.31:0/2895057374 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-08-25T03:33:40.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.085+0000 7f40e77fe700 1 -- 172.21.15.31:0/2895057374 >> v1:172.21.15.31:6789/0 conn(0x7f40e806a1e0 legacy=0x7f40e806ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:40.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.085+0000 7f40e77fe700 1 -- 172.21.15.31:0/2895057374 >> v1:172.21.15.2:6789/0 conn(0x7f40e81607f0 legacy=0x7f40e80ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:40.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.085+0000 7f40e77fe700 1 -- 172.21.15.31:0/2895057374 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e80cda00 con 0x7f40e80af180 2023-08-25T03:33:40.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.085+0000 7f40e77fe700 1 --2- 172.21.15.31:0/2895057374 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f40e80af180 0x7f40e8070e60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f40dc009920 tx=0x7f40dc009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d022dbe0809ad661 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:40.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.085+0000 7f40e6ffd700 1 -- 172.21.15.31:0/2895057374 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40dc00ba60 con 0x7f40e80af180 2023-08-25T03:33:40.088 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.085+0000 7f40e6ffd700 1 -- 172.21.15.31:0/2895057374 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40dc00bbc0 con 0x7f40e80af180 2023-08-25T03:33:40.089 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.085+0000 7f40e6ffd700 1 -- 172.21.15.31:0/2895057374 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40dc009e70 con 0x7f40e80af180 2023-08-25T03:33:40.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.086+0000 7f40ee265700 1 -- 172.21.15.31:0/2895057374 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f40e80af180 msgr2=0x7f40e8070e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:40.091 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.086+0000 7f40ee265700 1 --2- 172.21.15.31:0/2895057374 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f40e80af180 0x7f40e8070e60 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f40dc009920 tx=0x7f40dc009c30 comp rx=0 tx=0).stop 2023-08-25T03:33:40.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.086+0000 7f40ee265700 1 -- 172.21.15.31:0/2895057374 shutdown_connections 2023-08-25T03:33:40.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.086+0000 7f40ee265700 1 --2- 172.21.15.31:0/2895057374 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f40e80af180 0x7f40e8070e60 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:40.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.086+0000 7f40ee265700 1 -- 172.21.15.31:0/2895057374 >> 172.21.15.31:0/2895057374 conn(0x7f40e80cfd90 msgr2=0x7f40e80d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:40.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.086+0000 7f40ee265700 1 -- 172.21.15.31:0/2895057374 shutdown_connections 2023-08-25T03:33:40.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.086+0000 7f40ee265700 1 -- 172.21.15.31:0/2895057374 wait complete. 2023-08-25T03:33:40.092 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.087+0000 7f40ee265700 1 Processor -- start 2023-08-25T03:33:40.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.087+0000 7f40ee265700 1 -- start start 2023-08-25T03:33:40.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.088+0000 7f40ee265700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40e806a1e0 0x7f40e80c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:40.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.088+0000 7f40ee265700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e80af180 0x7f40e80c7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:40.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.088+0000 7f40ee265700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f40e81607f0 0x7f40e80c6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:40.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.088+0000 7f40ee265700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e81f3250 con 0x7f40e806a1e0 2023-08-25T03:33:40.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.088+0000 7f40ee265700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40e80c6790 con 0x7f40e80af180 2023-08-25T03:33:40.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.088+0000 7f40ee265700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40e815bdf0 con 0x7f40e81607f0 2023-08-25T03:33:40.093 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.088+0000 7f40e77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e80af180 0x7f40e80c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:40.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.088+0000 7f40e7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40e806a1e0 0x7f40e80c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:40.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.088+0000 7f40e77fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e80af180 0x7f40e80c7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:41750/0 (socket says 172.21.15.31:41750) 2023-08-25T03:33:40.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.088+0000 7f40e77fe700 1 -- 172.21.15.31:0/1497671145 learned_addr learned my addr 172.21.15.31:0/1497671145 (peer_addr_for_me v2:172.21.15.31:0/0) 2023-08-25T03:33:40.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.088+0000 7f40ec802700 1 --2- 172.21.15.31:0/1497671145 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f40e81607f0 0x7f40e80c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:40.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.089+0000 7f40e7fff700 1 -- 172.21.15.31:0/1497671145 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f40e81607f0 msgr2=0x7f40e80c6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:40.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.089+0000 7f40e7fff700 1 --2- 172.21.15.31:0/1497671145 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f40e81607f0 0x7f40e80c6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:40.094 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.089+0000 7f40e7fff700 1 -- 172.21.15.31:0/1497671145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e80af180 msgr2=0x7f40e80c7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:40.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.089+0000 7f40e7fff700 1 --2- 172.21.15.31:0/1497671145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e80af180 0x7f40e80c7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:40.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.089+0000 7f40e7fff700 1 -- 172.21.15.31:0/1497671145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40dc009550 con 0x7f40e806a1e0 2023-08-25T03:33:40.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.089+0000 7f40ec802700 1 --2- 172.21.15.31:0/1497671145 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f40e81607f0 0x7f40e80c6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:33:40.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.089+0000 7f40e7fff700 1 --2- 172.21.15.31:0/1497671145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40e806a1e0 0x7f40e80c6e60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f40d800c400 tx=0x7f40d800c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:40.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.089+0000 7f40e4ff9700 1 -- 172.21.15.31:0/1497671145 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d80052d0 con 0x7f40e806a1e0 2023-08-25T03:33:40.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.089+0000 7f40e4ff9700 1 -- 172.21.15.31:0/1497671145 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d8005430 con 0x7f40e806a1e0 2023-08-25T03:33:40.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.089+0000 7f40ee265700 1 -- 172.21.15.31:0/1497671145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40e81f2940 con 0x7f40e806a1e0 2023-08-25T03:33:40.095 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.089+0000 7f40e4ff9700 1 -- 172.21.15.31:0/1497671145 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d80038a0 con 0x7f40e806a1e0 2023-08-25T03:33:40.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.089+0000 7f40ee265700 1 -- 172.21.15.31:0/1497671145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40e81f2c00 con 0x7f40e806a1e0 2023-08-25T03:33:40.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.089+0000 7f40e4ff9700 1 -- 172.21.15.31:0/1497671145 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f40d8003d00 con 0x7f40e806a1e0 2023-08-25T03:33:40.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.089+0000 7f40e4ff9700 1 -- 172.21.15.31:0/1497671145 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f40d8012a80 con 0x7f40e806a1e0 2023-08-25T03:33:40.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.090+0000 7f40ee265700 1 -- 172.21.15.31:0/1497671145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40e80746d0 con 0x7f40e806a1e0 2023-08-25T03:33:40.096 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.093+0000 7f40e4ff9700 1 -- 172.21.15.31:0/1497671145 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f40d8017070 con 0x7f40e806a1e0 2023-08-25T03:33:40.154 INFO:tasks.ceph.mgr.x.smithi031.stderr:2023-08-25T03:33:40.153+0000 7faad00e4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-08-25T03:33:40.188 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.187+0000 7f40ee265700 1 -- 172.21.15.31:0/1497671145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd new", "uuid": "fab11910-b768-4930-addd-8cf4e2ef03dd", "id": 7} v 0) v1 -- 0x7f40e80ce7c0 con 0x7f40e806a1e0 2023-08-25T03:33:40.191 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.190+0000 7f40e4ff9700 1 -- 172.21.15.31:0/1497671145 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "fab11910-b768-4930-addd-8cf4e2ef03dd", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f40d801c440 con 0x7f40e806a1e0 2023-08-25T03:33:40.192 INFO:teuthology.orchestra.run.smithi031.stdout:7 2023-08-25T03:33:40.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.191+0000 7f40ee265700 1 -- 172.21.15.31:0/1497671145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40e806a1e0 msgr2=0x7f40e80c6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:40.193 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.191+0000 7f40ee265700 1 --2- 172.21.15.31:0/1497671145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40e806a1e0 0x7f40e80c6e60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f40d800c400 tx=0x7f40d800c710 comp rx=0 tx=0).stop 2023-08-25T03:33:40.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.191+0000 7f40ee265700 1 -- 172.21.15.31:0/1497671145 shutdown_connections 2023-08-25T03:33:40.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.191+0000 7f40ee265700 1 --2- 172.21.15.31:0/1497671145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f40e806a1e0 0x7f40e80c6e60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:40.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.191+0000 7f40ee265700 1 --2- 172.21.15.31:0/1497671145 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f40e81607f0 0x7f40e80c6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:40.194 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.191+0000 7f40ee265700 1 --2- 172.21.15.31:0/1497671145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f40e80af180 0x7f40e80c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:40.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.191+0000 7f40ee265700 1 -- 172.21.15.31:0/1497671145 >> 172.21.15.31:0/1497671145 conn(0x7f40e80cfd90 msgr2=0x7f40e80cd270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:40.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.191+0000 7f40ee265700 1 -- 172.21.15.31:0/1497671145 shutdown_connections 2023-08-25T03:33:40.195 INFO:teuthology.orchestra.run.smithi031.stderr:2023-08-25T03:33:40.191+0000 7f40ee265700 1 -- 172.21.15.31:0/1497671145 wait complete. 2023-08-25T03:33:40.204 INFO:tasks.ceph.osd.0:Restarting daemon 2023-08-25T03:33:40.205 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-08-25T03:33:40.207 INFO:tasks.ceph.osd.0:Started 2023-08-25T03:33:40.208 INFO:tasks.ceph.osd.1:Restarting daemon 2023-08-25T03:33:40.208 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-08-25T03:33:40.210 INFO:tasks.ceph.osd.1:Started 2023-08-25T03:33:40.210 INFO:tasks.ceph.osd.2:Restarting daemon 2023-08-25T03:33:40.211 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-08-25T03:33:40.213 INFO:tasks.ceph.osd.2:Started 2023-08-25T03:33:40.213 INFO:tasks.ceph.osd.3:Restarting daemon 2023-08-25T03:33:40.213 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-08-25T03:33:40.215 INFO:tasks.ceph.osd.3:Started 2023-08-25T03:33:40.216 INFO:tasks.ceph.osd.4:Restarting daemon 2023-08-25T03:33:40.216 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-08-25T03:33:40.218 INFO:tasks.ceph.osd.4:Started 2023-08-25T03:33:40.218 INFO:tasks.ceph.osd.5:Restarting daemon 2023-08-25T03:33:40.219 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-08-25T03:33:40.221 INFO:tasks.ceph.osd.5:Started 2023-08-25T03:33:40.222 INFO:tasks.ceph.osd.6:Restarting daemon 2023-08-25T03:33:40.222 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-08-25T03:33:40.224 INFO:tasks.ceph.osd.6:Started 2023-08-25T03:33:40.225 INFO:tasks.ceph.osd.7:Restarting daemon 2023-08-25T03:33:40.225 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-08-25T03:33:40.227 INFO:tasks.ceph.osd.7:Started 2023-08-25T03:33:40.227 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:33:40.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.386+0000 7fb19414d700 1 Processor -- start 2023-08-25T03:33:40.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.386+0000 7fb19414d700 1 -- start start 2023-08-25T03:33:40.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.387+0000 7fb19414d700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb18c06cfe0 0x7fb18c0ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:40.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.387+0000 7fb19414d700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb18c0cf120 con 0x7fb18c0d0720 2023-08-25T03:33:40.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.387+0000 7fb19414d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb18c0b29f0 con 0x7fb18c15d1f0 2023-08-25T03:33:40.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.387+0000 7fb19414d700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c0d0b90 con 0x7fb18c06cfe0 2023-08-25T03:33:40.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.387+0000 7fb1926ea700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fb18c0d0720 0x7fb18c15d5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45204/0 (socket says 172.21.15.2:45204) 2023-08-25T03:33:40.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.387+0000 7fb1926ea700 1 -- 172.21.15.2:0/2203710285 learned_addr learned my addr 172.21.15.2:0/2203710285 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:33:40.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.387+0000 7fb1916e8700 1 --2- 172.21.15.2:0/2203710285 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb18c06cfe0 0x7fb18c0ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:40.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.387+0000 7fb190ee7700 1 -- 172.21.15.2:0/2203710285 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3255035625 0 0) 0x7fb18c0cf120 con 0x7fb18c0d0720 2023-08-25T03:33:40.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.387+0000 7fb190ee7700 1 -- 172.21.15.2:0/2203710285 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1780035b0 con 0x7fb18c0d0720 2023-08-25T03:33:40.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.387+0000 7fb190ee7700 1 -- 172.21.15.2:0/2203710285 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3696759062 0 0) 0x7fb18c0b29f0 con 0x7fb18c15d1f0 2023-08-25T03:33:40.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.387+0000 7fb190ee7700 1 -- 172.21.15.2:0/2203710285 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb18c0cf120 con 0x7fb18c15d1f0 2023-08-25T03:33:40.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.387+0000 7fb1916e8700 1 -- 172.21.15.2:0/2203710285 >> v1:172.21.15.31:6789/0 conn(0x7fb18c15d1f0 legacy=0x7fb18c06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:40.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.387+0000 7fb1916e8700 1 -- 172.21.15.2:0/2203710285 >> v1:172.21.15.2:6789/0 conn(0x7fb18c0d0720 legacy=0x7fb18c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:40.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.388+0000 7fb1916e8700 1 -- 172.21.15.2:0/2203710285 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c0b2cd0 con 0x7fb18c06cfe0 2023-08-25T03:33:40.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.388+0000 7fb1916e8700 1 --2- 172.21.15.2:0/2203710285 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb18c06cfe0 0x7fb18c0ceab0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb17c009870 tx=0x7fb17c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5ac5025b0d66b1f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:40.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.388+0000 7fb190ee7700 1 -- 172.21.15.2:0/2203710285 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c011830 con 0x7fb18c06cfe0 2023-08-25T03:33:40.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.388+0000 7fb190ee7700 1 -- 172.21.15.2:0/2203710285 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17c011990 con 0x7fb18c06cfe0 2023-08-25T03:33:40.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.388+0000 7fb190ee7700 1 -- 172.21.15.2:0/2203710285 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c011d20 con 0x7fb18c06cfe0 2023-08-25T03:33:40.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.388+0000 7fb19414d700 1 -- 172.21.15.2:0/2203710285 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb18c06cfe0 msgr2=0x7fb18c0ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:40.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.388+0000 7fb19414d700 1 --2- 172.21.15.2:0/2203710285 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb18c06cfe0 0x7fb18c0ceab0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb17c009870 tx=0x7fb17c009b80 comp rx=0 tx=0).stop 2023-08-25T03:33:40.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.388+0000 7fb19414d700 1 -- 172.21.15.2:0/2203710285 shutdown_connections 2023-08-25T03:33:40.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.388+0000 7fb19414d700 1 --2- 172.21.15.2:0/2203710285 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb18c06cfe0 0x7fb18c0ceab0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:40.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.388+0000 7fb19414d700 1 -- 172.21.15.2:0/2203710285 >> 172.21.15.2:0/2203710285 conn(0x7fb18c0d0f70 msgr2=0x7fb18c0d1350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:40.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.389+0000 7fb19414d700 1 -- 172.21.15.2:0/2203710285 shutdown_connections 2023-08-25T03:33:40.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.389+0000 7fb19414d700 1 -- 172.21.15.2:0/2203710285 wait complete. 2023-08-25T03:33:40.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.390+0000 7fb19414d700 1 Processor -- start 2023-08-25T03:33:40.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.390+0000 7fb19414d700 1 -- start start 2023-08-25T03:33:40.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.390+0000 7fb19414d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb18c06cfe0 0x7fb18c1f3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:40.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.390+0000 7fb19414d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb18c0d0720 0x7fb18c153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:40.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.390+0000 7fb19414d700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb18c15d1f0 0x7fb18c0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:40.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.390+0000 7fb19414d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c1f3d20 con 0x7fb18c0d0720 2023-08-25T03:33:40.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.390+0000 7fb19414d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c15bdc0 con 0x7fb18c06cfe0 2023-08-25T03:33:40.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.390+0000 7fb19414d700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c15c0a0 con 0x7fb18c15d1f0 2023-08-25T03:33:40.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.391+0000 7fb1916e8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb18c0d0720 0x7fb18c153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:40.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.391+0000 7fb1916e8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb18c0d0720 0x7fb18c153270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58542/0 (socket says 172.21.15.2:58542) 2023-08-25T03:33:40.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.391+0000 7fb1916e8700 1 -- 172.21.15.2:0/3458039135 learned_addr learned my addr 172.21.15.2:0/3458039135 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:33:40.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.391+0000 7fb191ee9700 1 --2- 172.21.15.2:0/3458039135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb18c06cfe0 0x7fb18c1f3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:40.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.391+0000 7fb1926ea700 1 --2- 172.21.15.2:0/3458039135 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb18c15d1f0 0x7fb18c0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:40.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.391+0000 7fb1916e8700 1 -- 172.21.15.2:0/3458039135 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb18c15d1f0 msgr2=0x7fb18c0ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:40.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.391+0000 7fb1916e8700 1 --2- 172.21.15.2:0/3458039135 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb18c15d1f0 0x7fb18c0ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:40.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.391+0000 7fb1916e8700 1 -- 172.21.15.2:0/3458039135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb18c06cfe0 msgr2=0x7fb18c1f3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:40.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.391+0000 7fb1916e8700 1 --2- 172.21.15.2:0/3458039135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb18c06cfe0 0x7fb18c1f3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:40.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.391+0000 7fb1916e8700 1 -- 172.21.15.2:0/3458039135 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb17c0094a0 con 0x7fb18c0d0720 2023-08-25T03:33:40.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.391+0000 7fb1916e8700 1 --2- 172.21.15.2:0/3458039135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb18c0d0720 0x7fb18c153270 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0090b0 tx=0x7fb17c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:40.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.391+0000 7fb1827fc700 1 -- 172.21.15.2:0/3458039135 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c01a6e0 con 0x7fb18c0d0720 2023-08-25T03:33:40.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.391+0000 7fb1827fc700 1 -- 172.21.15.2:0/3458039135 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb17c01a840 con 0x7fb18c0d0720 2023-08-25T03:33:40.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.391+0000 7fb19414d700 1 -- 172.21.15.2:0/3458039135 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb18c15c380 con 0x7fb18c0d0720 2023-08-25T03:33:40.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.391+0000 7fb1827fc700 1 -- 172.21.15.2:0/3458039135 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c01abb0 con 0x7fb18c0d0720 2023-08-25T03:33:40.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.391+0000 7fb19414d700 1 -- 172.21.15.2:0/3458039135 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb18c0ad770 con 0x7fb18c0d0720 2023-08-25T03:33:40.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.391+0000 7fb1827fc700 1 -- 172.21.15.2:0/3458039135 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb17c022070 con 0x7fb18c0d0720 2023-08-25T03:33:40.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.391+0000 7fb1827fc700 1 -- 172.21.15.2:0/3458039135 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fb17c01ad10 con 0x7fb18c0d0720 2023-08-25T03:33:40.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.392+0000 7fb19414d700 1 -- 172.21.15.2:0/3458039135 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb18c1f3e90 con 0x7fb18c0d0720 2023-08-25T03:33:40.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.394+0000 7fb1827fc700 1 -- 172.21.15.2:0/3458039135 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7fb17c029070 con 0x7fb18c0d0720 2023-08-25T03:33:40.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.465+0000 7fb19414d700 1 -- 172.21.15.2:0/3458039135 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb18c070150 con 0x7fb18c0d0720 2023-08-25T03:33:40.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.466+0000 7fb1827fc700 1 -- 172.21.15.2:0/3458039135 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fb17c015020 con 0x7fb18c0d0720 2023-08-25T03:33:40.467 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:40.468 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":10,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:33:40.189548+0000","last_up_change":"0.000000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:33:40.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.467+0000 7fb19414d700 1 -- 172.21.15.2:0/3458039135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb18c0d0720 msgr2=0x7fb18c153270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:40.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.467+0000 7fb19414d700 1 --2- 172.21.15.2:0/3458039135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb18c0d0720 0x7fb18c153270 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb17c0090b0 tx=0x7fb17c004750 comp rx=0 tx=0).stop 2023-08-25T03:33:40.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.467+0000 7fb19414d700 1 -- 172.21.15.2:0/3458039135 shutdown_connections 2023-08-25T03:33:40.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.467+0000 7fb19414d700 1 --2- 172.21.15.2:0/3458039135 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb18c15d1f0 0x7fb18c0ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:40.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.467+0000 7fb19414d700 1 --2- 172.21.15.2:0/3458039135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb18c06cfe0 0x7fb18c1f3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:40.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.467+0000 7fb19414d700 1 --2- 172.21.15.2:0/3458039135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb18c0d0720 0x7fb18c153270 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:40.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.467+0000 7fb19414d700 1 -- 172.21.15.2:0/3458039135 >> 172.21.15.2:0/3458039135 conn(0x7fb18c0d0f70 msgr2=0x7fb18c160dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:40.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.467+0000 7fb19414d700 1 -- 172.21.15.2:0/3458039135 shutdown_connections 2023-08-25T03:33:40.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:40.467+0000 7fb19414d700 1 -- 172.21.15.2:0/3458039135 wait complete. 2023-08-25T03:33:40.479 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-08-25T03:33:40.480 INFO:tasks.ceph:Waiting for OSDs to come up 2023-08-25T03:33:40.578 INFO:tasks.ceph.osd.1.smithi002.stderr:2023-08-25T03:33:40.577+0000 7f3d631da640 -1 Falling back to public interface 2023-08-25T03:33:40.828 INFO:tasks.ceph.osd.5.smithi031.stderr:2023-08-25T03:33:40.827+0000 7fc7402b2640 -1 Falling back to public interface 2023-08-25T03:33:40.840 INFO:tasks.ceph.osd.6.smithi031.stderr:2023-08-25T03:33:40.839+0000 7ff83e325640 -1 Falling back to public interface 2023-08-25T03:33:40.843 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:40.843+0000 7f8f9b465200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-25T03:33:40.844 INFO:tasks.ceph.osd.0.smithi002.stderr:2023-08-25T03:33:40.843+0000 7fec7352e640 -1 Falling back to public interface 2023-08-25T03:33:40.849 INFO:tasks.ceph.osd.7.smithi031.stderr:2023-08-25T03:33:40.848+0000 7f4273194640 -1 Falling back to public interface 2023-08-25T03:33:40.853 INFO:tasks.ceph.osd.2.smithi002.stderr:2023-08-25T03:33:40.852+0000 7f2581244640 -1 Falling back to public interface 2023-08-25T03:33:40.855 INFO:tasks.ceph.osd.3.smithi002.stderr:2023-08-25T03:33:40.854+0000 7f61802e3640 -1 Falling back to public interface 2023-08-25T03:33:40.855 INFO:tasks.ceph.osd.4.smithi031.stderr:2023-08-25T03:33:40.854+0000 7f964cf46640 -1 Falling back to public interface 2023-08-25T03:33:40.914 INFO:tasks.ceph.mgr.x.smithi031.stderr:2023-08-25T03:33:40.913+0000 7faad00e4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-08-25T03:33:40.929 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:40.928+0000 7f8f9b465200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-25T03:33:40.995 INFO:tasks.ceph.mgr.x.smithi031.stderr:2023-08-25T03:33:40.994+0000 7faad00e4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-08-25T03:33:41.076 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:41.075+0000 7f8f9b465200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-25T03:33:41.144 INFO:tasks.ceph.mgr.x.smithi031.stderr:2023-08-25T03:33:41.143+0000 7faad00e4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-08-25T03:33:41.284 DEBUG:teuthology.orchestra.run.smithi002:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-08-25T03:33:41.324 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:41.323+0000 7f8f9b465200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-25T03:33:41.389 INFO:tasks.ceph.mgr.x.smithi031.stderr:2023-08-25T03:33:41.389+0000 7faad00e4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-08-25T03:33:41.401 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.400+0000 7f5400c4b700 1 Processor -- start 2023-08-25T03:33:41.401 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.400+0000 7f5400c4b700 1 -- start start 2023-08-25T03:33:41.402 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.401+0000 7f5400c4b700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f53fc159810 0x7f53fc0cb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:41.403 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.401+0000 7f5400c4b700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53fc070850 0x7f53fc070ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:41.403 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.401+0000 7f5400c4b700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53fc0c9fc0 con 0x7f53fc06af80 2023-08-25T03:33:41.403 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.401+0000 7f5400c4b700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53fc071220 con 0x7f53fc159810 2023-08-25T03:33:41.403 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.401+0000 7f5400c4b700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53fc0ca260 con 0x7f53fc070850 2023-08-25T03:33:41.403 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.401+0000 7f53fa59c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f53fc159810 0x7f53fc0cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:41.403 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.401+0000 7f53f9d9b700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f53fc06af80 0x7f53fc0cdb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45212/0 (socket says 172.21.15.2:45212) 2023-08-25T03:33:41.404 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.401+0000 7f53fa59c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f53fc159810 0x7f53fc0cb020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58546/0 (socket says 172.21.15.2:58546) 2023-08-25T03:33:41.404 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.401+0000 7f53f9d9b700 1 -- 172.21.15.2:0/1977849353 learned_addr learned my addr 172.21.15.2:0/1977849353 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:33:41.404 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.401+0000 7f53fad9d700 1 --2- 172.21.15.2:0/1977849353 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53fc070850 0x7f53fc070ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:41.404 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.402+0000 7f53fa59c700 1 -- 172.21.15.2:0/1977849353 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53fc070850 msgr2=0x7f53fc070ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:33:41.404 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.402+0000 7f53fa59c700 1 --2- 172.21.15.2:0/1977849353 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53fc070850 0x7f53fc070ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:41.404 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.402+0000 7f53fa59c700 1 -- 172.21.15.2:0/1977849353 >> v1:172.21.15.2:6789/0 conn(0x7f53fc06af80 legacy=0x7f53fc0cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:41.404 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.402+0000 7f53f959a700 1 -- 172.21.15.2:0/1977849353 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 367447252 0 0) 0x7f53fc0c9fc0 con 0x7f53fc06af80 2023-08-25T03:33:41.405 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.402+0000 7f53fa59c700 1 -- 172.21.15.2:0/1977849353 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53fc0ca540 con 0x7f53fc159810 2023-08-25T03:33:41.405 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.402+0000 7f53fa59c700 1 --2- 172.21.15.2:0/1977849353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53fc159810 0x7f53fc0cb020 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009840 tx=0x7f53ec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3443c20647e0224a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:41.405 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.402+0000 7f53f959a700 1 -- 172.21.15.2:0/1977849353 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ec010800 con 0x7f53fc159810 2023-08-25T03:33:41.405 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.402+0000 7f53f959a700 1 -- 172.21.15.2:0/1977849353 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53ec010960 con 0x7f53fc159810 2023-08-25T03:33:41.405 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.402+0000 7f53f959a700 1 -- 172.21.15.2:0/1977849353 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ec010cf0 con 0x7f53fc159810 2023-08-25T03:33:41.407 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.403+0000 7f5400c4b700 1 -- 172.21.15.2:0/1977849353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53fc159810 msgr2=0x7f53fc0cb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:41.407 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.403+0000 7f5400c4b700 1 --2- 172.21.15.2:0/1977849353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53fc159810 0x7f53fc0cb020 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f53ec009840 tx=0x7f53ec009b50 comp rx=0 tx=0).stop 2023-08-25T03:33:41.408 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.403+0000 7f5400c4b700 1 -- 172.21.15.2:0/1977849353 shutdown_connections 2023-08-25T03:33:41.408 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.403+0000 7f5400c4b700 1 --2- 172.21.15.2:0/1977849353 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53fc070850 0x7f53fc070ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:41.408 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.403+0000 7f5400c4b700 1 --2- 172.21.15.2:0/1977849353 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53fc159810 0x7f53fc0cb020 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:41.408 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.403+0000 7f5400c4b700 1 -- 172.21.15.2:0/1977849353 >> 172.21.15.2:0/1977849353 conn(0x7f53fc0cf690 msgr2=0x7f53fc0cfaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:41.408 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.403+0000 7f5400c4b700 1 -- 172.21.15.2:0/1977849353 shutdown_connections 2023-08-25T03:33:41.408 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.403+0000 7f5400c4b700 1 -- 172.21.15.2:0/1977849353 wait complete. 2023-08-25T03:33:41.409 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.404+0000 7f5400c4b700 1 Processor -- start 2023-08-25T03:33:41.409 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.404+0000 7f5400c4b700 1 -- start start 2023-08-25T03:33:41.409 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.405+0000 7f5400c4b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53fc06af80 0x7f53fc13e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:41.409 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.405+0000 7f5400c4b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53fc070850 0x7f53fc15c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:41.409 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.405+0000 7f5400c4b700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53fc159810 0x7f53fc1f16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:41.409 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.405+0000 7f5400c4b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53fc156830 con 0x7f53fc070850 2023-08-25T03:33:41.409 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.405+0000 7f5400c4b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53fc155bb0 con 0x7f53fc06af80 2023-08-25T03:33:41.410 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.405+0000 7f5400c4b700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53fc155e90 con 0x7f53fc159810 2023-08-25T03:33:41.410 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.405+0000 7f53f9d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53fc070850 0x7f53fc15c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:41.410 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.405+0000 7f53f9d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53fc070850 0x7f53fc15c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58552/0 (socket says 172.21.15.2:58552) 2023-08-25T03:33:41.410 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.405+0000 7f53f9d9b700 1 -- 172.21.15.2:0/3916468430 learned_addr learned my addr 172.21.15.2:0/3916468430 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:33:41.410 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.405+0000 7f53fad9d700 1 --2- 172.21.15.2:0/3916468430 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53fc159810 0x7f53fc1f16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:41.410 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.405+0000 7f53fa59c700 1 --2- 172.21.15.2:0/3916468430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53fc06af80 0x7f53fc13e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:41.410 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.406+0000 7f53f9d9b700 1 -- 172.21.15.2:0/3916468430 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53fc159810 msgr2=0x7f53fc1f16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:41.410 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.406+0000 7f53f9d9b700 1 --2- 172.21.15.2:0/3916468430 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53fc159810 0x7f53fc1f16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:41.411 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.406+0000 7f53f9d9b700 1 -- 172.21.15.2:0/3916468430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53fc06af80 msgr2=0x7f53fc13e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:41.411 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.406+0000 7f53f9d9b700 1 --2- 172.21.15.2:0/3916468430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53fc06af80 0x7f53fc13e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:41.411 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.406+0000 7f53f9d9b700 1 -- 172.21.15.2:0/3916468430 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec0094a0 con 0x7f53fc070850 2023-08-25T03:33:41.411 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.406+0000 7f53f9d9b700 1 --2- 172.21.15.2:0/3916468430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53fc070850 0x7f53fc15c770 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f53f000c540 tx=0x7f53f000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:41.411 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.406+0000 7f53eaffd700 1 -- 172.21.15.2:0/3916468430 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f0004c80 con 0x7f53fc070850 2023-08-25T03:33:41.411 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.406+0000 7f53eaffd700 1 -- 172.21.15.2:0/3916468430 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53f0004de0 con 0x7f53fc070850 2023-08-25T03:33:41.411 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.406+0000 7f5400c4b700 1 -- 172.21.15.2:0/3916468430 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53fc156170 con 0x7f53fc070850 2023-08-25T03:33:41.411 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.406+0000 7f53eaffd700 1 -- 172.21.15.2:0/3916468430 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53f0007310 con 0x7f53fc070850 2023-08-25T03:33:41.412 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.406+0000 7f5400c4b700 1 -- 172.21.15.2:0/3916468430 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53fc13e9a0 con 0x7f53fc070850 2023-08-25T03:33:41.412 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.407+0000 7f53eaffd700 1 -- 172.21.15.2:0/3916468430 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f53f001d3f0 con 0x7f53fc070850 2023-08-25T03:33:41.412 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.407+0000 7f5400c4b700 1 -- 172.21.15.2:0/3916468430 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53fc0746d0 con 0x7f53fc070850 2023-08-25T03:33:41.412 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.407+0000 7f53eaffd700 1 -- 172.21.15.2:0/3916468430 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f53f0013a90 con 0x7f53fc070850 2023-08-25T03:33:41.412 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.410+0000 7f53eaffd700 1 -- 172.21.15.2:0/3916468430 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72429 (secure 0 0 0) 0x7f53f0013d30 con 0x7f53fc070850 2023-08-25T03:33:41.483 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.483+0000 7f5400c4b700 1 -- 172.21.15.2:0/3916468430 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53fc156970 con 0x7f53fc070850 2023-08-25T03:33:41.484 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.484+0000 7f53eaffd700 1 -- 172.21.15.2:0/3916468430 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f53fc156970 con 0x7f53fc070850 2023-08-25T03:33:41.484 INFO:teuthology.misc.health.smithi002.stdout: 2023-08-25T03:33:41.485 INFO:teuthology.misc.health.smithi002.stdout:{"epoch":10,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:33:40.189548+0000","last_up_change":"0.000000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:33:41.485 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.485+0000 7f5400c4b700 1 -- 172.21.15.2:0/3916468430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53fc070850 msgr2=0x7f53fc15c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:41.486 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.485+0000 7f5400c4b700 1 --2- 172.21.15.2:0/3916468430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53fc070850 0x7f53fc15c770 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f53f000c540 tx=0x7f53f000c850 comp rx=0 tx=0).stop 2023-08-25T03:33:41.486 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.485+0000 7f5400c4b700 1 -- 172.21.15.2:0/3916468430 shutdown_connections 2023-08-25T03:33:41.486 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.485+0000 7f5400c4b700 1 --2- 172.21.15.2:0/3916468430 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53fc159810 0x7f53fc1f16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:41.486 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.485+0000 7f5400c4b700 1 --2- 172.21.15.2:0/3916468430 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53fc06af80 0x7f53fc13e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:41.486 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.485+0000 7f5400c4b700 1 --2- 172.21.15.2:0/3916468430 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53fc070850 0x7f53fc15c770 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:41.486 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.485+0000 7f5400c4b700 1 -- 172.21.15.2:0/3916468430 >> 172.21.15.2:0/3916468430 conn(0x7f53fc0cf690 msgr2=0x7f53fc0cd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:41.486 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.485+0000 7f5400c4b700 1 -- 172.21.15.2:0/3916468430 shutdown_connections 2023-08-25T03:33:41.487 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:41.485+0000 7f5400c4b700 1 -- 172.21.15.2:0/3916468430 wait complete. 2023-08-25T03:33:41.496 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-08-25T03:33:41.548 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:41.548+0000 7f8f9b465200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-25T03:33:41.611 INFO:tasks.ceph.mgr.x.smithi031.stderr:2023-08-25T03:33:41.610+0000 7faad00e4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-08-25T03:33:41.689 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:41.688+0000 7f8f9b465200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-25T03:33:41.752 INFO:tasks.ceph.mgr.x.smithi031.stderr:2023-08-25T03:33:41.751+0000 7faad00e4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-08-25T03:33:41.917 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:41.917+0000 7f8f9b465200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-25T03:33:41.980 INFO:tasks.ceph.mgr.x.smithi031.stderr:2023-08-25T03:33:41.980+0000 7faad00e4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-08-25T03:33:42.080 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:42.079+0000 7f8f9b465200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-25T03:33:42.143 INFO:tasks.ceph.mgr.x.smithi031.stderr:2023-08-25T03:33:42.142+0000 7faad00e4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-08-25T03:33:42.201 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:42.200+0000 7f8f9b465200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-25T03:33:42.264 INFO:tasks.ceph.mgr.x.smithi031.stderr:2023-08-25T03:33:42.263+0000 7faad00e4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-08-25T03:33:42.536 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:42.535+0000 7f8f9b465200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-25T03:33:42.600 INFO:tasks.ceph.mgr.x.smithi031.stderr:2023-08-25T03:33:42.599+0000 7faad00e4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-08-25T03:33:42.640 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:42.639+0000 7f8f9b465200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-25T03:33:42.704 INFO:tasks.ceph.mgr.x.smithi031.stderr:2023-08-25T03:33:42.703+0000 7faad00e4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-08-25T03:33:42.834 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:42.833+0000 7f8f9b465200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-25T03:33:42.897 INFO:tasks.ceph.mgr.x.smithi031.stderr:2023-08-25T03:33:42.896+0000 7faad00e4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-08-25T03:33:43.478 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:43.478+0000 7f8f9b465200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-25T03:33:43.538 INFO:tasks.ceph.mgr.x.smithi031.stderr:2023-08-25T03:33:43.536+0000 7faad00e4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-08-25T03:33:44.109 INFO:tasks.ceph.osd.1.smithi002.stderr:2023-08-25T03:33:44.108+0000 7f3d631da640 -1 osd.1 0 log_to_monitors true 2023-08-25T03:33:44.338 INFO:tasks.ceph.osd.5.smithi031.stderr:2023-08-25T03:33:44.337+0000 7fc7402b2640 -1 osd.5 0 log_to_monitors true 2023-08-25T03:33:44.361 INFO:tasks.ceph.osd.0.smithi002.stderr:2023-08-25T03:33:44.361+0000 7fec7352e640 -1 osd.0 0 log_to_monitors true 2023-08-25T03:33:44.374 INFO:tasks.ceph.osd.2.smithi002.stderr:2023-08-25T03:33:44.374+0000 7f2581244640 -1 osd.2 0 log_to_monitors true 2023-08-25T03:33:44.380 INFO:tasks.ceph.osd.3.smithi002.stderr:2023-08-25T03:33:44.380+0000 7f61802e3640 -1 osd.3 0 log_to_monitors true 2023-08-25T03:33:44.441 INFO:tasks.ceph.osd.6.smithi031.stderr:2023-08-25T03:33:44.440+0000 7ff83e325640 -1 osd.6 0 log_to_monitors true 2023-08-25T03:33:44.457 INFO:tasks.ceph.osd.7.smithi031.stderr:2023-08-25T03:33:44.455+0000 7f4273194640 -1 osd.7 0 log_to_monitors true 2023-08-25T03:33:44.459 INFO:tasks.ceph.osd.4.smithi031.stderr:2023-08-25T03:33:44.458+0000 7f964cf46640 -1 osd.4 0 log_to_monitors true 2023-08-25T03:33:44.503 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:44.503+0000 7f8f5346e700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-08-25T03:33:44.504 INFO:tasks.ceph.mgr.y.smithi002.stderr:2023-08-25T03:33:44.503+0000 7f8f5346e700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-08-25T03:33:46.943 INFO:tasks.ceph.osd.0.smithi002.stderr:2023-08-25T03:33:46.942+0000 7fec670e5700 -1 osd.0 0 waiting for initial osdmap 2023-08-25T03:33:46.956 INFO:tasks.ceph.osd.0.smithi002.stderr:2023-08-25T03:33:46.956+0000 7fec5f4c0700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T03:33:46.967 INFO:tasks.ceph.osd.3.smithi002.stderr:2023-08-25T03:33:46.966+0000 7f6173e9a700 -1 osd.3 0 waiting for initial osdmap 2023-08-25T03:33:46.976 INFO:tasks.ceph.osd.2.smithi002.stderr:2023-08-25T03:33:46.976+0000 7f2574dfb700 -1 osd.2 0 waiting for initial osdmap 2023-08-25T03:33:47.013 INFO:tasks.ceph.osd.2.smithi002.stderr:2023-08-25T03:33:47.012+0000 7f256d1d6700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T03:33:47.014 INFO:tasks.ceph.osd.3.smithi002.stderr:2023-08-25T03:33:47.012+0000 7f616c275700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T03:33:47.018 INFO:tasks.ceph.osd.1.smithi002.stderr:2023-08-25T03:33:47.017+0000 7f3d56d91700 -1 osd.1 0 waiting for initial osdmap 2023-08-25T03:33:47.032 INFO:tasks.ceph.osd.1.smithi002.stderr:2023-08-25T03:33:47.031+0000 7f3d4f16c700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T03:33:47.631 INFO:tasks.ceph.osd.5.smithi031.stderr:2023-08-25T03:33:47.630+0000 7fc733e69700 -1 osd.5 0 waiting for initial osdmap 2023-08-25T03:33:47.652 INFO:tasks.ceph.osd.5.smithi031.stderr:2023-08-25T03:33:47.651+0000 7fc72c244700 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T03:33:47.658 INFO:tasks.ceph.osd.7.smithi031.stderr:2023-08-25T03:33:47.657+0000 7f4266d4b700 -1 osd.7 0 waiting for initial osdmap 2023-08-25T03:33:47.662 INFO:tasks.ceph.osd.4.smithi031.stderr:2023-08-25T03:33:47.661+0000 7f9640afd700 -1 osd.4 0 waiting for initial osdmap 2023-08-25T03:33:47.665 INFO:tasks.ceph.osd.6.smithi031.stderr:2023-08-25T03:33:47.664+0000 7ff831edc700 -1 osd.6 0 waiting for initial osdmap 2023-08-25T03:33:47.673 INFO:tasks.ceph.osd.4.smithi031.stderr:2023-08-25T03:33:47.672+0000 7f9638ed8700 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T03:33:47.673 INFO:tasks.ceph.osd.7.smithi031.stderr:2023-08-25T03:33:47.672+0000 7f425f126700 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T03:33:47.681 INFO:tasks.ceph.osd.6.smithi031.stderr:2023-08-25T03:33:47.680+0000 7ff82a2b7700 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-08-25T03:33:48.304 DEBUG:teuthology.orchestra.run.smithi002:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-08-25T03:33:48.435 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.431+0000 7f1f78c2b700 1 Processor -- start 2023-08-25T03:33:48.436 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.431+0000 7f1f78c2b700 1 -- start start 2023-08-25T03:33:48.436 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.432+0000 7f1f78c2b700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1f7406f250 0x7f1f7406d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:48.437 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.432+0000 7f1f78c2b700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f1f74071000 0x7f1f74071460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:48.437 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.432+0000 7f1f78c2b700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f740ade30 con 0x7f1f7415a3c0 2023-08-25T03:33:48.437 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.432+0000 7f1f78c2b700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f740cc5d0 con 0x7f1f74071000 2023-08-25T03:33:48.437 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.432+0000 7f1f78c2b700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740cb650 con 0x7f1f7406f250 2023-08-25T03:33:48.437 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.432+0000 7f1f72d9d700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f1f74071000 0x7f1f74071460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:48.438 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.432+0000 7f1f7259c700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f1f7415a3c0 0x7f1f740ca560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45392/0 (socket says 172.21.15.2:45392) 2023-08-25T03:33:48.438 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.432+0000 7f1f7259c700 1 -- 172.21.15.2:0/560748452 learned_addr learned my addr 172.21.15.2:0/560748452 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:33:48.438 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.432+0000 7f1f71d9b700 1 --2- 172.21.15.2:0/560748452 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1f7406f250 0x7f1f7406d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:48.438 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.432+0000 7f1f72d9d700 1 -- 172.21.15.2:0/560748452 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1f7406f250 msgr2=0x7f1f7406d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:33:48.439 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.432+0000 7f1f72d9d700 1 --2- 172.21.15.2:0/560748452 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1f7406f250 0x7f1f7406d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:48.439 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.432+0000 7f1f72d9d700 1 -- 172.21.15.2:0/560748452 >> v1:172.21.15.2:6789/0 conn(0x7f1f7415a3c0 legacy=0x7f1f740ca560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:48.439 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.432+0000 7f1f72d9d700 1 -- 172.21.15.2:0/560748452 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f740719a0 con 0x7f1f74071000 2023-08-25T03:33:48.439 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.432+0000 7f1f7159a700 1 -- 172.21.15.2:0/560748452 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1800124540 0 0) 0x7f1f740ade30 con 0x7f1f7415a3c0 2023-08-25T03:33:48.439 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.432+0000 7f1f72d9d700 1 --2- 172.21.15.2:0/560748452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f74071000 0x7f1f74071460 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009d90 tx=0x7f1f6c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e4e73bca825bc2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:48.439 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.432+0000 7f1f7159a700 1 -- 172.21.15.2:0/560748452 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c00e070 con 0x7f1f74071000 2023-08-25T03:33:48.440 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.432+0000 7f1f7159a700 1 -- 172.21.15.2:0/560748452 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6c00bbd0 con 0x7f1f74071000 2023-08-25T03:33:48.440 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.432+0000 7f1f7159a700 1 -- 172.21.15.2:0/560748452 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c012430 con 0x7f1f74071000 2023-08-25T03:33:48.440 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.433+0000 7f1f78c2b700 1 -- 172.21.15.2:0/560748452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f74071000 msgr2=0x7f1f74071460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:48.440 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.433+0000 7f1f78c2b700 1 --2- 172.21.15.2:0/560748452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f74071000 0x7f1f74071460 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009d90 tx=0x7f1f6c00b330 comp rx=0 tx=0).stop 2023-08-25T03:33:48.440 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.433+0000 7f1f78c2b700 1 -- 172.21.15.2:0/560748452 shutdown_connections 2023-08-25T03:33:48.440 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.433+0000 7f1f78c2b700 1 --2- 172.21.15.2:0/560748452 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f74071000 0x7f1f74071460 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:48.440 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.433+0000 7f1f78c2b700 1 --2- 172.21.15.2:0/560748452 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1f7406f250 0x7f1f7406d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:48.441 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.433+0000 7f1f78c2b700 1 -- 172.21.15.2:0/560748452 >> 172.21.15.2:0/560748452 conn(0x7f1f740cff10 msgr2=0x7f1f740d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:48.441 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.433+0000 7f1f78c2b700 1 -- 172.21.15.2:0/560748452 shutdown_connections 2023-08-25T03:33:48.441 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.433+0000 7f1f78c2b700 1 -- 172.21.15.2:0/560748452 wait complete. 2023-08-25T03:33:48.443 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.434+0000 7f1f78c2b700 1 Processor -- start 2023-08-25T03:33:48.443 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.435+0000 7f1f78c2b700 1 -- start start 2023-08-25T03:33:48.443 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.435+0000 7f1f78c2b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f7406f250 0x7f1f740b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:48.443 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.435+0000 7f1f78c2b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f74071000 0x7f1f74158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:48.444 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.435+0000 7f1f78c2b700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1f7415a3c0 0x7f1f741f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:48.444 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.435+0000 7f1f78c2b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f741f36b0 con 0x7f1f74071000 2023-08-25T03:33:48.444 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.435+0000 7f1f78c2b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740c6b50 con 0x7f1f7406f250 2023-08-25T03:33:48.444 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.435+0000 7f1f78c2b700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740c5fd0 con 0x7f1f7415a3c0 2023-08-25T03:33:48.444 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.435+0000 7f1f71d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f74071000 0x7f1f74158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:48.445 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.435+0000 7f1f71d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f74071000 0x7f1f74158b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58736/0 (socket says 172.21.15.2:58736) 2023-08-25T03:33:48.445 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.435+0000 7f1f71d9b700 1 -- 172.21.15.2:0/451388597 learned_addr learned my addr 172.21.15.2:0/451388597 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:33:48.445 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.435+0000 7f1f72d9d700 1 --2- 172.21.15.2:0/451388597 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1f7415a3c0 0x7f1f741f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:48.445 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.435+0000 7f1f7259c700 1 --2- 172.21.15.2:0/451388597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f7406f250 0x7f1f740b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:48.445 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.436+0000 7f1f71d9b700 1 -- 172.21.15.2:0/451388597 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1f7415a3c0 msgr2=0x7f1f741f27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:48.445 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.436+0000 7f1f71d9b700 1 --2- 172.21.15.2:0/451388597 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1f7415a3c0 0x7f1f741f27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:48.445 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.436+0000 7f1f71d9b700 1 -- 172.21.15.2:0/451388597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f7406f250 msgr2=0x7f1f740b2200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:48.446 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.436+0000 7f1f71d9b700 1 --2- 172.21.15.2:0/451388597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f7406f250 0x7f1f740b2200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:48.446 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.436+0000 7f1f71d9b700 1 -- 172.21.15.2:0/451388597 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c009a70 con 0x7f1f74071000 2023-08-25T03:33:48.446 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.436+0000 7f1f71d9b700 1 --2- 172.21.15.2:0/451388597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f74071000 0x7f1f74158b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1f6800c7f0 tx=0x7f1f6800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:48.446 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.436+0000 7f1f62ffd700 1 -- 172.21.15.2:0/451388597 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f68011800 con 0x7f1f74071000 2023-08-25T03:33:48.446 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.436+0000 7f1f62ffd700 1 -- 172.21.15.2:0/451388597 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f68011960 con 0x7f1f74071000 2023-08-25T03:33:48.446 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.436+0000 7f1f78c2b700 1 -- 172.21.15.2:0/451388597 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f740c62b0 con 0x7f1f74071000 2023-08-25T03:33:48.447 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.436+0000 7f1f62ffd700 1 -- 172.21.15.2:0/451388597 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f68011cd0 con 0x7f1f74071000 2023-08-25T03:33:48.447 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.436+0000 7f1f78c2b700 1 -- 172.21.15.2:0/451388597 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f740c6870 con 0x7f1f74071000 2023-08-25T03:33:48.447 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.437+0000 7f1f78c2b700 1 -- 172.21.15.2:0/451388597 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f740746d0 con 0x7f1f74071000 2023-08-25T03:33:48.447 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.437+0000 7f1f62ffd700 1 -- 172.21.15.2:0/451388597 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 4) v1 ==== 88285+0+0 (secure 0 0 0) 0x7f1f68010350 con 0x7f1f74071000 2023-08-25T03:33:48.447 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.438+0000 7f1f62ffd700 1 --2- 172.21.15.2:0/451388597 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1f5406a050 0x7f1f5406c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:48.447 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.438+0000 7f1f62ffd700 1 -- 172.21.15.2:0/451388597 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 3351+0+0 (secure 0 0 0) 0x7f1f68087380 con 0x7f1f74071000 2023-08-25T03:33:48.448 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.438+0000 7f1f7259c700 1 --2- 172.21.15.2:0/451388597 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1f5406a050 0x7f1f5406c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:48.448 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.438+0000 7f1f7259c700 1 --2- 172.21.15.2:0/451388597 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1f5406a050 0x7f1f5406c510 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1f64007fd0 tx=0x7f1f64007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:48.448 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.443+0000 7f1f62ffd700 1 -- 172.21.15.2:0/451388597 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1f68053670 con 0x7f1f74071000 2023-08-25T03:33:48.645 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.643+0000 7f1f78c2b700 1 -- 172.21.15.2:0/451388597 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f740b5e90 con 0x7f1f74071000 2023-08-25T03:33:48.647 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.644+0000 7f1f62ffd700 1 -- 172.21.15.2:0/451388597 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+8904 (secure 0 0 0) 0x7f1f6801f090 con 0x7f1f74071000 2023-08-25T03:33:48.647 INFO:teuthology.misc.health.smithi002.stdout: 2023-08-25T03:33:48.647 INFO:teuthology.misc.health.smithi002.stdout:{"epoch":13,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:33:47.577528+0000","last_up_change":"2023-08-25T03:33:47.577528+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:33:48.648 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.646+0000 7f1f78c2b700 1 -- 172.21.15.2:0/451388597 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1f5406a050 msgr2=0x7f1f5406c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:48.649 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.646+0000 7f1f78c2b700 1 --2- 172.21.15.2:0/451388597 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1f5406a050 0x7f1f5406c510 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1f64007fd0 tx=0x7f1f64007de0 comp rx=0 tx=0).stop 2023-08-25T03:33:48.649 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.646+0000 7f1f78c2b700 1 -- 172.21.15.2:0/451388597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f74071000 msgr2=0x7f1f74158b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:48.649 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.646+0000 7f1f78c2b700 1 --2- 172.21.15.2:0/451388597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f74071000 0x7f1f74158b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1f6800c7f0 tx=0x7f1f6800cb00 comp rx=0 tx=0).stop 2023-08-25T03:33:48.650 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.646+0000 7f1f78c2b700 1 -- 172.21.15.2:0/451388597 shutdown_connections 2023-08-25T03:33:48.650 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.646+0000 7f1f78c2b700 1 --2- 172.21.15.2:0/451388597 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1f7415a3c0 0x7f1f741f27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:48.650 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.646+0000 7f1f78c2b700 1 --2- 172.21.15.2:0/451388597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f7406f250 0x7f1f740b2200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:48.650 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.646+0000 7f1f78c2b700 1 --2- 172.21.15.2:0/451388597 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1f5406a050 0x7f1f5406c510 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:48.650 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.646+0000 7f1f78c2b700 1 --2- 172.21.15.2:0/451388597 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f74071000 0x7f1f74158b50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:48.650 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.646+0000 7f1f78c2b700 1 -- 172.21.15.2:0/451388597 >> 172.21.15.2:0/451388597 conn(0x7f1f740cff10 msgr2=0x7f1f7406a250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:48.651 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.646+0000 7f1f78c2b700 1 -- 172.21.15.2:0/451388597 shutdown_connections 2023-08-25T03:33:48.651 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:48.646+0000 7f1f78c2b700 1 -- 172.21.15.2:0/451388597 wait complete. 2023-08-25T03:33:48.659 DEBUG:teuthology.misc:4 of 8 OSDs are up 2023-08-25T03:33:55.466 DEBUG:teuthology.orchestra.run.smithi002:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-08-25T03:33:55.669 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.667+0000 7f52d1563700 1 Processor -- start 2023-08-25T03:33:55.669 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.667+0000 7f52d1563700 1 -- start start 2023-08-25T03:33:55.670 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.668+0000 7f52d1563700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f52cc160ad0 0x7f52cc06e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:55.670 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.668+0000 7f52d1563700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f52cc06bfa0 0x7f52cc0adec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:55.671 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.668+0000 7f52d1563700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52cc06fa90 con 0x7f52cc160eb0 2023-08-25T03:33:55.671 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.668+0000 7f52d1563700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52cc06ee80 con 0x7f52cc06bfa0 2023-08-25T03:33:55.671 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.668+0000 7f52d1563700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52cc06f160 con 0x7f52cc160ad0 2023-08-25T03:33:55.671 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.668+0000 7f52cb7fe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f52cc06bfa0 0x7f52cc0adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:55.671 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.668+0000 7f52cb7fe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f52cc06bfa0 0x7f52cc0adec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58886/0 (socket says 172.21.15.2:58886) 2023-08-25T03:33:55.672 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.668+0000 7f52cb7fe700 1 -- 172.21.15.2:0/1728154962 learned_addr learned my addr 172.21.15.2:0/1728154962 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:33:55.672 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.668+0000 7f52caffd700 1 --2- 172.21.15.2:0/1728154962 >> v2:172.21.15.31:3300/0 conn(0x7f52cc160ad0 0x7f52cc06e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:55.672 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.668+0000 7f52cb7fe700 1 -- 172.21.15.2:0/1728154962 >> v2:172.21.15.31:3300/0 conn(0x7f52cc160ad0 msgr2=0x7f52cc06e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:33:55.675 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.668+0000 7f52cb7fe700 1 --2- 172.21.15.2:0/1728154962 >> v2:172.21.15.31:3300/0 conn(0x7f52cc160ad0 0x7f52cc06e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:55.675 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.669+0000 7f52cb7fe700 1 -- 172.21.15.2:0/1728154962 >> v1:172.21.15.31:6789/0 conn(0x7f52cc160eb0 legacy=0x7f52cc06b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:55.675 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.669+0000 7f52cb7fe700 1 -- 172.21.15.2:0/1728154962 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52cc06f440 con 0x7f52cc06bfa0 2023-08-25T03:33:55.675 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.669+0000 7f52cb7fe700 1 --2- 172.21.15.2:0/1728154962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52cc06bfa0 0x7f52cc0adec0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f52cc0700a0 tx=0x7f52c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545389b49f9e88c5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:55.675 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.669+0000 7f52c9ffb700 1 -- 172.21.15.2:0/1728154962 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c400e070 con 0x7f52cc06bfa0 2023-08-25T03:33:55.676 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.669+0000 7f52c9ffb700 1 -- 172.21.15.2:0/1728154962 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c400b9c0 con 0x7f52cc06bfa0 2023-08-25T03:33:55.676 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.669+0000 7f52c9ffb700 1 -- 172.21.15.2:0/1728154962 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c400bd70 con 0x7f52cc06bfa0 2023-08-25T03:33:55.676 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.669+0000 7f52d1563700 1 -- 172.21.15.2:0/1728154962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52cc06bfa0 msgr2=0x7f52cc0adec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:55.676 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.669+0000 7f52d1563700 1 --2- 172.21.15.2:0/1728154962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52cc06bfa0 0x7f52cc0adec0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f52cc0700a0 tx=0x7f52c400b330 comp rx=0 tx=0).stop 2023-08-25T03:33:55.676 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.669+0000 7f52d1563700 1 -- 172.21.15.2:0/1728154962 shutdown_connections 2023-08-25T03:33:55.676 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.669+0000 7f52d1563700 1 --2- 172.21.15.2:0/1728154962 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52cc06bfa0 0x7f52cc0adec0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:55.676 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.669+0000 7f52d1563700 1 --2- 172.21.15.2:0/1728154962 >> v2:172.21.15.31:3300/0 conn(0x7f52cc160ad0 0x7f52cc06e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:55.677 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.669+0000 7f52d1563700 1 -- 172.21.15.2:0/1728154962 >> 172.21.15.2:0/1728154962 conn(0x7f52cc0d0ff0 msgr2=0x7f52cc0d1400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:55.677 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.669+0000 7f52d1563700 1 -- 172.21.15.2:0/1728154962 shutdown_connections 2023-08-25T03:33:55.677 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.670+0000 7f52d1563700 1 -- 172.21.15.2:0/1728154962 wait complete. 2023-08-25T03:33:55.677 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.671+0000 7f52d1563700 1 Processor -- start 2023-08-25T03:33:55.677 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.671+0000 7f52d1563700 1 -- start start 2023-08-25T03:33:55.677 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.672+0000 7f52d1563700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52cc06bfa0 0x7f52cc155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:55.678 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.672+0000 7f52d1563700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f52cc160ad0 0x7f52cc0b4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:55.678 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.672+0000 7f52d1563700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52cc160eb0 0x7f52cc0b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:55.678 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.672+0000 7f52d1563700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc0b3cd0 con 0x7f52cc06bfa0 2023-08-25T03:33:55.678 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.672+0000 7f52d1563700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc0b3f70 con 0x7f52cc160eb0 2023-08-25T03:33:55.678 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.672+0000 7f52d1563700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52cc0b6340 con 0x7f52cc160ad0 2023-08-25T03:33:55.679 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.672+0000 7f52caffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52cc06bfa0 0x7f52cc155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:55.679 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.672+0000 7f52caffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52cc06bfa0 0x7f52cc155810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58888/0 (socket says 172.21.15.2:58888) 2023-08-25T03:33:55.679 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.672+0000 7f52caffd700 1 -- 172.21.15.2:0/3287530261 learned_addr learned my addr 172.21.15.2:0/3287530261 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:33:55.679 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.672+0000 7f52cb7fe700 1 --2- 172.21.15.2:0/3287530261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52cc160eb0 0x7f52cc0b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:55.680 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.672+0000 7f52ca7fc700 1 --2- 172.21.15.2:0/3287530261 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f52cc160ad0 0x7f52cc0b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:55.680 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.672+0000 7f52caffd700 1 -- 172.21.15.2:0/3287530261 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f52cc160ad0 msgr2=0x7f52cc0b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:55.680 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.672+0000 7f52caffd700 1 --2- 172.21.15.2:0/3287530261 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f52cc160ad0 0x7f52cc0b4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:55.680 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.672+0000 7f52caffd700 1 -- 172.21.15.2:0/3287530261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52cc160eb0 msgr2=0x7f52cc0b47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:55.680 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.672+0000 7f52caffd700 1 --2- 172.21.15.2:0/3287530261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52cc160eb0 0x7f52cc0b47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:55.680 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.672+0000 7f52caffd700 1 -- 172.21.15.2:0/3287530261 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c4009b30 con 0x7f52cc06bfa0 2023-08-25T03:33:55.681 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.672+0000 7f52ca7fc700 1 --2- 172.21.15.2:0/3287530261 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f52cc160ad0 0x7f52cc0b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:33:55.681 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.672+0000 7f52caffd700 1 --2- 172.21.15.2:0/3287530261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52cc06bfa0 0x7f52cc155810 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00e760 tx=0x7f52bc00ea70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:55.681 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.672+0000 7f52b37fe700 1 -- 172.21.15.2:0/3287530261 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc005c20 con 0x7f52cc06bfa0 2023-08-25T03:33:55.681 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.672+0000 7f52b37fe700 1 -- 172.21.15.2:0/3287530261 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52bc005da0 con 0x7f52cc06bfa0 2023-08-25T03:33:55.681 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.672+0000 7f52d1563700 1 -- 172.21.15.2:0/3287530261 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52cc0b6620 con 0x7f52cc06bfa0 2023-08-25T03:33:55.681 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.672+0000 7f52b37fe700 1 -- 172.21.15.2:0/3287530261 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52bc006260 con 0x7f52cc06bfa0 2023-08-25T03:33:55.682 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.673+0000 7f52d1563700 1 -- 172.21.15.2:0/3287530261 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52cc0b5070 con 0x7f52cc06bfa0 2023-08-25T03:33:55.683 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.673+0000 7f52d1563700 1 -- 172.21.15.2:0/3287530261 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52cc1e7800 con 0x7f52cc06bfa0 2023-08-25T03:33:55.683 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.674+0000 7f52b37fe700 1 -- 172.21.15.2:0/3287530261 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f52bc0063c0 con 0x7f52cc06bfa0 2023-08-25T03:33:55.683 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.674+0000 7f52b37fe700 1 --2- 172.21.15.2:0/3287530261 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f52b806a0e0 0x7f52b806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:55.683 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.674+0000 7f52b37fe700 1 -- 172.21.15.2:0/3287530261 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f52bc0878b0 con 0x7f52cc06bfa0 2023-08-25T03:33:55.684 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.674+0000 7f52ca7fc700 1 --2- 172.21.15.2:0/3287530261 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f52b806a0e0 0x7f52b806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:55.684 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.675+0000 7f52ca7fc700 1 --2- 172.21.15.2:0/3287530261 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f52b806a0e0 0x7f52b806c5a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f52cc156260 tx=0x7f52c0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:55.684 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.679+0000 7f52b37fe700 1 -- 172.21.15.2:0/3287530261 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f52bc0534e0 con 0x7f52cc06bfa0 2023-08-25T03:33:55.880 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.879+0000 7f52d1563700 1 -- 172.21.15.2:0/3287530261 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52cc0746d0 con 0x7f52cc06bfa0 2023-08-25T03:33:55.882 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.879+0000 7f52b37fe700 1 -- 172.21.15.2:0/3287530261 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12706 (secure 0 0 0) 0x7f52bc020090 con 0x7f52cc06bfa0 2023-08-25T03:33:55.882 INFO:teuthology.misc.health.smithi002.stdout: 2023-08-25T03:33:55.883 INFO:teuthology.misc.health.smithi002.stdout:{"epoch":17,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:33:51.675339+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:33:55.884 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.881+0000 7f52d1563700 1 -- 172.21.15.2:0/3287530261 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f52b806a0e0 msgr2=0x7f52b806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:55.884 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.881+0000 7f52d1563700 1 --2- 172.21.15.2:0/3287530261 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f52b806a0e0 0x7f52b806c5a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f52cc156260 tx=0x7f52c0009040 comp rx=0 tx=0).stop 2023-08-25T03:33:55.884 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.881+0000 7f52d1563700 1 -- 172.21.15.2:0/3287530261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52cc06bfa0 msgr2=0x7f52cc155810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:55.884 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.881+0000 7f52d1563700 1 --2- 172.21.15.2:0/3287530261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52cc06bfa0 0x7f52cc155810 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00e760 tx=0x7f52bc00ea70 comp rx=0 tx=0).stop 2023-08-25T03:33:55.884 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.882+0000 7f52d1563700 1 -- 172.21.15.2:0/3287530261 shutdown_connections 2023-08-25T03:33:55.885 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.882+0000 7f52d1563700 1 --2- 172.21.15.2:0/3287530261 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f52b806a0e0 0x7f52b806c5a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:55.885 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.882+0000 7f52d1563700 1 --2- 172.21.15.2:0/3287530261 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52cc06bfa0 0x7f52cc155810 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:55.885 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.882+0000 7f52d1563700 1 --2- 172.21.15.2:0/3287530261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52cc160eb0 0x7f52cc0b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:55.885 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.882+0000 7f52d1563700 1 --2- 172.21.15.2:0/3287530261 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f52cc160ad0 0x7f52cc0b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:55.885 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.882+0000 7f52d1563700 1 -- 172.21.15.2:0/3287530261 >> 172.21.15.2:0/3287530261 conn(0x7f52cc0d0ff0 msgr2=0x7f52cc0d13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:55.885 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.882+0000 7f52d1563700 1 -- 172.21.15.2:0/3287530261 shutdown_connections 2023-08-25T03:33:55.885 INFO:teuthology.misc.health.smithi002.stderr:2023-08-25T03:33:55.882+0000 7f52d1563700 1 -- 172.21.15.2:0/3287530261 wait complete. 2023-08-25T03:33:55.895 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-08-25T03:33:55.896 INFO:tasks.ceph:Creating RBD pool 2023-08-25T03:33:55.896 DEBUG:teuthology.orchestra.run.smithi002:> sudo ceph --cluster ceph osd pool create rbd 8 2023-08-25T03:33:56.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.020+0000 7f011caaf700 1 Processor -- start 2023-08-25T03:33:56.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.020+0000 7f011caaf700 1 -- start start 2023-08-25T03:33:56.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.021+0000 7f011caaf700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f011815b170 0x7f01180cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:56.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.021+0000 7f011caaf700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f01180cc640 0x7f01180ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:56.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.021+0000 7f011caaf700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01180cf6f0 con 0x7f01180cd7f0 2023-08-25T03:33:56.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.021+0000 7f011caaf700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01180d1d50 con 0x7f011815b170 2023-08-25T03:33:56.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.021+0000 7f011caaf700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f011815b550 con 0x7f01180cc640 2023-08-25T03:33:56.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.021+0000 7f011659c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f011815b170 0x7f01180cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:56.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.021+0000 7f011659c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f011815b170 0x7f01180cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58896/0 (socket says 172.21.15.2:58896) 2023-08-25T03:33:56.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.021+0000 7f011659c700 1 -- 172.21.15.2:0/1750227683 learned_addr learned my addr 172.21.15.2:0/1750227683 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:33:56.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.021+0000 7f0115d9b700 1 --2- 172.21.15.2:0/1750227683 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f01180cc640 0x7f01180ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:56.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.021+0000 7f011659c700 1 -- 172.21.15.2:0/1750227683 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f01180cc640 msgr2=0x7f01180ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:33:56.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.021+0000 7f011659c700 1 --2- 172.21.15.2:0/1750227683 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f01180cc640 0x7f01180ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:56.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.021+0000 7f011659c700 1 -- 172.21.15.2:0/1750227683 >> v1:172.21.15.31:6789/0 conn(0x7f01180cd7f0 legacy=0x7f011806d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:56.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.021+0000 7f011659c700 1 -- 172.21.15.2:0/1750227683 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01180ceae0 con 0x7f011815b170 2023-08-25T03:33:56.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.021+0000 7f011659c700 1 --2- 172.21.15.2:0/1750227683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f011815b170 0x7f01180cfb80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0108006fd0 tx=0x7f0108009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66dff36b564dcfb2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:56.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.021+0000 7f011559a700 1 -- 172.21.15.2:0/1750227683 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0108005e30 con 0x7f011815b170 2023-08-25T03:33:56.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.022+0000 7f011559a700 1 -- 172.21.15.2:0/1750227683 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0108004030 con 0x7f011815b170 2023-08-25T03:33:56.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.022+0000 7f011559a700 1 -- 172.21.15.2:0/1750227683 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01080043e0 con 0x7f011815b170 2023-08-25T03:33:56.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.022+0000 7f011caaf700 1 -- 172.21.15.2:0/1750227683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f011815b170 msgr2=0x7f01180cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:56.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.022+0000 7f011caaf700 1 --2- 172.21.15.2:0/1750227683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f011815b170 0x7f01180cfb80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0108006fd0 tx=0x7f0108009bd0 comp rx=0 tx=0).stop 2023-08-25T03:33:56.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.022+0000 7f011caaf700 1 -- 172.21.15.2:0/1750227683 shutdown_connections 2023-08-25T03:33:56.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.022+0000 7f011caaf700 1 --2- 172.21.15.2:0/1750227683 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f011815b170 0x7f01180cfb80 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:56.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.022+0000 7f011caaf700 1 --2- 172.21.15.2:0/1750227683 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f01180cc640 0x7f01180ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:56.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.022+0000 7f011caaf700 1 -- 172.21.15.2:0/1750227683 >> 172.21.15.2:0/1750227683 conn(0x7f011809a200 msgr2=0x7f01180d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:56.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.022+0000 7f011caaf700 1 -- 172.21.15.2:0/1750227683 shutdown_connections 2023-08-25T03:33:56.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.022+0000 7f011caaf700 1 -- 172.21.15.2:0/1750227683 wait complete. 2023-08-25T03:33:56.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.024+0000 7f011caaf700 1 Processor -- start 2023-08-25T03:33:56.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.024+0000 7f011caaf700 1 -- start start 2023-08-25T03:33:56.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.024+0000 7f011caaf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01180cc640 0x7f0118157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:56.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.024+0000 7f011caaf700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f01180cd7f0 0x7f011815c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:56.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.024+0000 7f011caaf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f011815b170 0x7f01180b5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:56.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.024+0000 7f011caaf700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01180cb2b0 con 0x7f011815b170 2023-08-25T03:33:56.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.024+0000 7f011caaf700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01180b3a90 con 0x7f01180cc640 2023-08-25T03:33:56.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.024+0000 7f011caaf700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01180b6520 con 0x7f01180cd7f0 2023-08-25T03:33:56.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.024+0000 7f0116d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f011815b170 0x7f01180b5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:56.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.024+0000 7f0116d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f011815b170 0x7f01180b5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58906/0 (socket says 172.21.15.2:58906) 2023-08-25T03:33:56.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.024+0000 7f0116d9d700 1 -- 172.21.15.2:0/2230471492 learned_addr learned my addr 172.21.15.2:0/2230471492 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:33:56.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.025+0000 7f011659c700 1 --2- 172.21.15.2:0/2230471492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01180cc640 0x7f0118157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:56.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.025+0000 7f0115d9b700 1 --2- 172.21.15.2:0/2230471492 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f01180cd7f0 0x7f011815c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:56.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.025+0000 7f0116d9d700 1 -- 172.21.15.2:0/2230471492 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f01180cd7f0 msgr2=0x7f011815c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:56.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.025+0000 7f0116d9d700 1 --2- 172.21.15.2:0/2230471492 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f01180cd7f0 0x7f011815c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:56.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.025+0000 7f0116d9d700 1 -- 172.21.15.2:0/2230471492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01180cc640 msgr2=0x7f0118157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:56.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.025+0000 7f0116d9d700 1 --2- 172.21.15.2:0/2230471492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01180cc640 0x7f0118157d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:56.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.025+0000 7f0116d9d700 1 -- 172.21.15.2:0/2230471492 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0108009500 con 0x7f011815b170 2023-08-25T03:33:56.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.025+0000 7f0116d9d700 1 --2- 172.21.15.2:0/2230471492 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f011815b170 0x7f01180b5fe0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f011000cc50 tx=0x7f011000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:56.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.025+0000 7f0106ffd700 1 -- 172.21.15.2:0/2230471492 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0110003c00 con 0x7f011815b170 2023-08-25T03:33:56.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.025+0000 7f0106ffd700 1 -- 172.21.15.2:0/2230471492 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0110003d60 con 0x7f011815b170 2023-08-25T03:33:56.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.025+0000 7f011caaf700 1 -- 172.21.15.2:0/2230471492 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01180b3d30 con 0x7f011815b170 2023-08-25T03:33:56.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.025+0000 7f0106ffd700 1 -- 172.21.15.2:0/2230471492 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f011000aab0 con 0x7f011815b170 2023-08-25T03:33:56.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.025+0000 7f011caaf700 1 -- 172.21.15.2:0/2230471492 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01180b42f0 con 0x7f011815b170 2023-08-25T03:33:56.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.026+0000 7f011caaf700 1 -- 172.21.15.2:0/2230471492 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01180746d0 con 0x7f011815b170 2023-08-25T03:33:56.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.026+0000 7f0106ffd700 1 -- 172.21.15.2:0/2230471492 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f011000ac10 con 0x7f011815b170 2023-08-25T03:33:56.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.027+0000 7f0106ffd700 1 --2- 172.21.15.2:0/2230471492 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f00f806a0e0 0x7f00f806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:56.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.027+0000 7f0106ffd700 1 -- 172.21.15.2:0/2230471492 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0110088040 con 0x7f011815b170 2023-08-25T03:33:56.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.027+0000 7f011659c700 1 --2- 172.21.15.2:0/2230471492 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f00f806a0e0 0x7f00f806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:56.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.027+0000 7f011659c700 1 --2- 172.21.15.2:0/2230471492 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f00f806a0e0 0x7f00f806c5a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0108009c10 tx=0x7f0108008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:56.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.031+0000 7f0106ffd700 1 -- 172.21.15.2:0/2230471492 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0110053be0 con 0x7f011815b170 2023-08-25T03:33:56.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.233+0000 7f011caaf700 1 -- 172.21.15.2:0/2230471492 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f01180ce620 con 0x7f011815b170 2023-08-25T03:33:56.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.558+0000 7f0106ffd700 1 -- 172.21.15.2:0/2230471492 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v18) v1 ==== 109+0+0 (secure 0 0 0) 0x7f0110057200 con 0x7f011815b170 2023-08-25T03:33:56.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.561+0000 7f011caaf700 1 -- 172.21.15.2:0/2230471492 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f00f806a0e0 msgr2=0x7f00f806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:56.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.561+0000 7f011caaf700 1 --2- 172.21.15.2:0/2230471492 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f00f806a0e0 0x7f00f806c5a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0108009c10 tx=0x7f0108008070 comp rx=0 tx=0).stop 2023-08-25T03:33:56.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.561+0000 7f011caaf700 1 -- 172.21.15.2:0/2230471492 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f011815b170 msgr2=0x7f01180b5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:56.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.561+0000 7f011caaf700 1 --2- 172.21.15.2:0/2230471492 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f011815b170 0x7f01180b5fe0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f011000cc50 tx=0x7f011000cf60 comp rx=0 tx=0).stop 2023-08-25T03:33:56.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.561+0000 7f011caaf700 1 -- 172.21.15.2:0/2230471492 shutdown_connections 2023-08-25T03:33:56.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.561+0000 7f011caaf700 1 --2- 172.21.15.2:0/2230471492 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f00f806a0e0 0x7f00f806c5a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:56.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.561+0000 7f011caaf700 1 --2- 172.21.15.2:0/2230471492 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f011815b170 0x7f01180b5fe0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:56.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.561+0000 7f011caaf700 1 --2- 172.21.15.2:0/2230471492 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f01180cc640 0x7f0118157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:56.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.561+0000 7f011caaf700 1 --2- 172.21.15.2:0/2230471492 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f01180cd7f0 0x7f011815c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:56.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.561+0000 7f011caaf700 1 -- 172.21.15.2:0/2230471492 >> 172.21.15.2:0/2230471492 conn(0x7f011809a200 msgr2=0x7f011806dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:56.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.562+0000 7f011caaf700 1 -- 172.21.15.2:0/2230471492 shutdown_connections 2023-08-25T03:33:56.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.562+0000 7f011caaf700 1 -- 172.21.15.2:0/2230471492 wait complete. 2023-08-25T03:33:56.565 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'rbd' created 2023-08-25T03:33:56.577 DEBUG:teuthology.orchestra.run.smithi002:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-08-25T03:33:56.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.704+0000 7f31fd1aa700 1 Processor -- start 2023-08-25T03:33:56.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.705+0000 7f31fd1aa700 1 -- start start 2023-08-25T03:33:56.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.705+0000 7f31fd1aa700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f31f80cadf0 0x7f31f80ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:56.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.705+0000 7f31fd1aa700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f31f80cb1d0 0x7f31f8160f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:56.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.705+0000 7f31fd1aa700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f31f81614d0 0x7f31f806a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:56.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.705+0000 7f31fd1aa700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f80cddc0 con 0x7f31f80cb1d0 2023-08-25T03:33:56.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.705+0000 7f31fd1aa700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f8161a50 con 0x7f31f80cadf0 2023-08-25T03:33:56.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.705+0000 7f31fd1aa700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f8161d30 con 0x7f31f81614d0 2023-08-25T03:33:56.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.705+0000 7f31f659c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f31f80cb1d0 0x7f31f8160f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:56.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.705+0000 7f31f659c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f31f80cb1d0 0x7f31f8160f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58910/0 (socket says 172.21.15.2:58910) 2023-08-25T03:33:56.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.705+0000 7f31f659c700 1 -- 172.21.15.2:0/2103424858 learned_addr learned my addr 172.21.15.2:0/2103424858 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:33:56.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.706+0000 7f31f759e700 1 --2- 172.21.15.2:0/2103424858 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f31f81614d0 0x7f31f806a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:56.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.706+0000 7f31f6d9d700 1 --2- 172.21.15.2:0/2103424858 >> v2:172.21.15.31:3300/0 conn(0x7f31f80cadf0 0x7f31f80ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:56.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.706+0000 7f31f659c700 1 -- 172.21.15.2:0/2103424858 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f31f81614d0 msgr2=0x7f31f806a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:33:56.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.706+0000 7f31f659c700 1 --2- 172.21.15.2:0/2103424858 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f31f81614d0 0x7f31f806a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:56.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.706+0000 7f31f659c700 1 -- 172.21.15.2:0/2103424858 >> v2:172.21.15.31:3300/0 conn(0x7f31f80cadf0 msgr2=0x7f31f80ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:33:56.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.706+0000 7f31f659c700 1 --2- 172.21.15.2:0/2103424858 >> v2:172.21.15.31:3300/0 conn(0x7f31f80cadf0 0x7f31f80ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:56.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.706+0000 7f31f659c700 1 -- 172.21.15.2:0/2103424858 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f80ce080 con 0x7f31f80cb1d0 2023-08-25T03:33:56.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.706+0000 7f31f659c700 1 --2- 172.21.15.2:0/2103424858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31f80cb1d0 0x7f31f8160f90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f31f806c810 tx=0x7f31e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b52387178a6c2fc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:56.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.706+0000 7f31f5d9b700 1 -- 172.21.15.2:0/2103424858 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e800b870 con 0x7f31f80cb1d0 2023-08-25T03:33:56.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.706+0000 7f31f5d9b700 1 -- 172.21.15.2:0/2103424858 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e800b9d0 con 0x7f31f80cb1d0 2023-08-25T03:33:56.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.706+0000 7f31f5d9b700 1 -- 172.21.15.2:0/2103424858 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e800bd60 con 0x7f31f80cb1d0 2023-08-25T03:33:56.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.707+0000 7f31fd1aa700 1 -- 172.21.15.2:0/2103424858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31f80cb1d0 msgr2=0x7f31f8160f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:56.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.707+0000 7f31fd1aa700 1 --2- 172.21.15.2:0/2103424858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31f80cb1d0 0x7f31f8160f90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f31f806c810 tx=0x7f31e8009bd0 comp rx=0 tx=0).stop 2023-08-25T03:33:56.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.707+0000 7f31fd1aa700 1 -- 172.21.15.2:0/2103424858 shutdown_connections 2023-08-25T03:33:56.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.707+0000 7f31fd1aa700 1 --2- 172.21.15.2:0/2103424858 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f31f81614d0 0x7f31f806a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:56.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.707+0000 7f31fd1aa700 1 --2- 172.21.15.2:0/2103424858 >> v2:172.21.15.31:3300/0 conn(0x7f31f80cadf0 0x7f31f80ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:56.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.707+0000 7f31fd1aa700 1 --2- 172.21.15.2:0/2103424858 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31f80cb1d0 0x7f31f8160f90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:56.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.707+0000 7f31fd1aa700 1 -- 172.21.15.2:0/2103424858 >> 172.21.15.2:0/2103424858 conn(0x7f31f80b1990 msgr2=0x7f31f80b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:56.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.707+0000 7f31fd1aa700 1 -- 172.21.15.2:0/2103424858 shutdown_connections 2023-08-25T03:33:56.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.707+0000 7f31fd1aa700 1 -- 172.21.15.2:0/2103424858 wait complete. 2023-08-25T03:33:56.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.708+0000 7f31fd1aa700 1 Processor -- start 2023-08-25T03:33:56.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.708+0000 7f31fd1aa700 1 -- start start 2023-08-25T03:33:56.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.709+0000 7f31fd1aa700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31f80cadf0 0x7f31f81475d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:56.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.709+0000 7f31fd1aa700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31f80cb1d0 0x7f31f814a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:56.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.709+0000 7f31fd1aa700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f31f81614d0 0x7f31f80c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:56.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.709+0000 7f31fd1aa700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f814a890 con 0x7f31f80cb1d0 2023-08-25T03:33:56.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.709+0000 7f31fd1aa700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f80aca10 con 0x7f31f80cadf0 2023-08-25T03:33:56.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.709+0000 7f31fd1aa700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f80c3d00 con 0x7f31f81614d0 2023-08-25T03:33:56.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.709+0000 7f31f659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31f80cb1d0 0x7f31f814a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:56.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.709+0000 7f31f659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31f80cb1d0 0x7f31f814a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58918/0 (socket says 172.21.15.2:58918) 2023-08-25T03:33:56.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.709+0000 7f31f659c700 1 -- 172.21.15.2:0/3230118010 learned_addr learned my addr 172.21.15.2:0/3230118010 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:33:56.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.709+0000 7f31f6d9d700 1 --2- 172.21.15.2:0/3230118010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31f80cadf0 0x7f31f81475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:56.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.709+0000 7f31f759e700 1 --2- 172.21.15.2:0/3230118010 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f31f81614d0 0x7f31f80c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:56.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.709+0000 7f31f659c700 1 -- 172.21.15.2:0/3230118010 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f31f81614d0 msgr2=0x7f31f80c37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:56.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.709+0000 7f31f659c700 1 --2- 172.21.15.2:0/3230118010 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f31f81614d0 0x7f31f80c37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:56.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.709+0000 7f31f659c700 1 -- 172.21.15.2:0/3230118010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31f80cadf0 msgr2=0x7f31f81475d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:56.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.709+0000 7f31f659c700 1 --2- 172.21.15.2:0/3230118010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31f80cadf0 0x7f31f81475d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:56.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.709+0000 7f31f659c700 1 -- 172.21.15.2:0/3230118010 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e8009550 con 0x7f31f80cb1d0 2023-08-25T03:33:56.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.710+0000 7f31f659c700 1 --2- 172.21.15.2:0/3230118010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31f80cb1d0 0x7f31f814a350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f31e8000c00 tx=0x7f31e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:56.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.710+0000 7f31df7fe700 1 -- 172.21.15.2:0/3230118010 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e8004480 con 0x7f31f80cb1d0 2023-08-25T03:33:56.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.710+0000 7f31df7fe700 1 -- 172.21.15.2:0/3230118010 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e80045e0 con 0x7f31f80cb1d0 2023-08-25T03:33:56.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.710+0000 7f31fd1aa700 1 -- 172.21.15.2:0/3230118010 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f80c60a0 con 0x7f31f80cb1d0 2023-08-25T03:33:56.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.710+0000 7f31df7fe700 1 -- 172.21.15.2:0/3230118010 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e80038e0 con 0x7f31f80cb1d0 2023-08-25T03:33:56.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.710+0000 7f31fd1aa700 1 -- 172.21.15.2:0/3230118010 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f80c6660 con 0x7f31f80cb1d0 2023-08-25T03:33:56.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.711+0000 7f31fd1aa700 1 -- 172.21.15.2:0/3230118010 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f8066f70 con 0x7f31f80cb1d0 2023-08-25T03:33:56.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.711+0000 7f31df7fe700 1 -- 172.21.15.2:0/3230118010 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f31e8021070 con 0x7f31f80cb1d0 2023-08-25T03:33:56.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.712+0000 7f31df7fe700 1 --2- 172.21.15.2:0/3230118010 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f31e406a0e0 0x7f31e406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:56.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.712+0000 7f31df7fe700 1 -- 172.21.15.2:0/3230118010 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5199+0+0 (secure 0 0 0) 0x7f31e8089730 con 0x7f31f80cb1d0 2023-08-25T03:33:56.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.712+0000 7f31f6d9d700 1 --2- 172.21.15.2:0/3230118010 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f31e406a0e0 0x7f31e406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:56.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.712+0000 7f31f6d9d700 1 --2- 172.21.15.2:0/3230118010 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f31e406a0e0 0x7f31e406c5a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f31ec009950 tx=0x7f31ec008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:56.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.717+0000 7f31df7fe700 1 -- 172.21.15.2:0/3230118010 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f31e8055330 con 0x7f31f80cb1d0 2023-08-25T03:33:56.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:56.919+0000 7f31fd1aa700 1 -- 172.21.15.2:0/3230118010 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f31f80ced70 con 0x7f31f80cb1d0 2023-08-25T03:33:57.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.563+0000 7f31df7fe700 1 -- 172.21.15.2:0/3230118010 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v19) v1 ==== 173+0+0 (secure 0 0 0) 0x7f31e80150c0 con 0x7f31f80cb1d0 2023-08-25T03:33:57.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.567+0000 7f31fd1aa700 1 -- 172.21.15.2:0/3230118010 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f31e406a0e0 msgr2=0x7f31e406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:57.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.567+0000 7f31fd1aa700 1 --2- 172.21.15.2:0/3230118010 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f31e406a0e0 0x7f31e406c5a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f31ec009950 tx=0x7f31ec008040 comp rx=0 tx=0).stop 2023-08-25T03:33:57.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.567+0000 7f31fd1aa700 1 -- 172.21.15.2:0/3230118010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31f80cb1d0 msgr2=0x7f31f814a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:57.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.567+0000 7f31fd1aa700 1 --2- 172.21.15.2:0/3230118010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31f80cb1d0 0x7f31f814a350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f31e8000c00 tx=0x7f31e8004750 comp rx=0 tx=0).stop 2023-08-25T03:33:57.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.567+0000 7f31fd1aa700 1 -- 172.21.15.2:0/3230118010 shutdown_connections 2023-08-25T03:33:57.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.567+0000 7f31fd1aa700 1 --2- 172.21.15.2:0/3230118010 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f31f81614d0 0x7f31f80c37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:57.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.567+0000 7f31fd1aa700 1 --2- 172.21.15.2:0/3230118010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f31f80cadf0 0x7f31f81475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:57.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.567+0000 7f31fd1aa700 1 --2- 172.21.15.2:0/3230118010 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f31e406a0e0 0x7f31e406c5a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:57.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.567+0000 7f31fd1aa700 1 --2- 172.21.15.2:0/3230118010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f31f80cb1d0 0x7f31f814a350 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:57.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.567+0000 7f31fd1aa700 1 -- 172.21.15.2:0/3230118010 >> 172.21.15.2:0/3230118010 conn(0x7f31f80b1990 msgr2=0x7f31f806db20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:57.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.568+0000 7f31fd1aa700 1 -- 172.21.15.2:0/3230118010 shutdown_connections 2023-08-25T03:33:57.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.568+0000 7f31fd1aa700 1 -- 172.21.15.2:0/3230118010 wait complete. 2023-08-25T03:33:57.571 INFO:teuthology.orchestra.run.smithi002.stderr:enabled application 'rbd' on pool 'rbd' 2023-08-25T03:33:57.586 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-08-25T03:33:57.586 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-25T03:33:57.587 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-25T03:33:57.590 INFO:tasks.ceph.mds.a:Started 2023-08-25T03:33:57.591 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-25T03:33:57.591 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-25T03:33:57.594 INFO:tasks.ceph.mds.c:Started 2023-08-25T03:33:57.594 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-25T03:33:57.594 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-25T03:33:57.597 INFO:tasks.ceph.mds.b:Started 2023-08-25T03:33:57.597 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-25T03:33:57.597 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-25T03:33:57.600 INFO:tasks.ceph.mds.d:Started 2023-08-25T03:33:57.600 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-08-25T03:33:57.601 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:33:57.677 INFO:tasks.ceph.mds.a.smithi002.stdout:starting mds.a at 2023-08-25T03:33:57.681 INFO:tasks.ceph.mds.c.smithi002.stdout:starting mds.c at 2023-08-25T03:33:57.690 INFO:tasks.ceph.mds.b.smithi031.stdout:starting mds.b at 2023-08-25T03:33:57.691 INFO:tasks.ceph.mds.d.smithi031.stdout:starting mds.d at 2023-08-25T03:33:57.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.736+0000 7f5e17912700 1 Processor -- start 2023-08-25T03:33:57.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.737+0000 7f5e17912700 1 -- start start 2023-08-25T03:33:57.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.737+0000 7f5e17912700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5e1015b060 0x7f5e1015b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:57.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.737+0000 7f5e17912700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5e1006dff0 0x7f5e1006e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:57.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.737+0000 7f5e17912700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e1006e980 con 0x7f5e100cb5a0 2023-08-25T03:33:57.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.737+0000 7f5e17912700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e100b6b30 con 0x7f5e1015b060 2023-08-25T03:33:57.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.737+0000 7f5e17912700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e100b36a0 con 0x7f5e1006dff0 2023-08-25T03:33:57.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.737+0000 7f5e156ae700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5e1015b060 0x7f5e1015b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:57.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.738+0000 7f5e156ae700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5e1015b060 0x7f5e1015b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58950/0 (socket says 172.21.15.2:58950) 2023-08-25T03:33:57.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.738+0000 7f5e156ae700 1 -- 172.21.15.2:0/452381695 learned_addr learned my addr 172.21.15.2:0/452381695 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:33:57.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.738+0000 7f5e15eaf700 1 --2- 172.21.15.2:0/452381695 >> v2:172.21.15.31:3300/0 conn(0x7f5e1006dff0 0x7f5e1006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:57.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.738+0000 7f5e156ae700 1 -- 172.21.15.2:0/452381695 >> v2:172.21.15.31:3300/0 conn(0x7f5e1006dff0 msgr2=0x7f5e1006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:33:57.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.738+0000 7f5e156ae700 1 --2- 172.21.15.2:0/452381695 >> v2:172.21.15.31:3300/0 conn(0x7f5e1006dff0 0x7f5e1006e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:57.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.738+0000 7f5e156ae700 1 -- 172.21.15.2:0/452381695 >> v1:172.21.15.31:6789/0 conn(0x7f5e100cb5a0 legacy=0x7f5e100ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:33:57.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.738+0000 7f5e156ae700 1 -- 172.21.15.2:0/452381695 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1015b980 con 0x7f5e1015b060 2023-08-25T03:33:57.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.738+0000 7f5e156ae700 1 --2- 172.21.15.2:0/452381695 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e1015b060 0x7f5e1015b440 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5e100700f0 tx=0x7f5e00009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c827ac0e28993de server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:57.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.738+0000 7f5e07fff700 1 -- 172.21.15.2:0/452381695 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e00004230 con 0x7f5e1015b060 2023-08-25T03:33:57.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.738+0000 7f5e07fff700 1 -- 172.21.15.2:0/452381695 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e00004390 con 0x7f5e1015b060 2023-08-25T03:33:57.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.738+0000 7f5e07fff700 1 -- 172.21.15.2:0/452381695 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e000036a0 con 0x7f5e1015b060 2023-08-25T03:33:57.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.739+0000 7f5e17912700 1 -- 172.21.15.2:0/452381695 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e1015b060 msgr2=0x7f5e1015b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:57.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.739+0000 7f5e17912700 1 --2- 172.21.15.2:0/452381695 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e1015b060 0x7f5e1015b440 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5e100700f0 tx=0x7f5e00009bd0 comp rx=0 tx=0).stop 2023-08-25T03:33:57.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.739+0000 7f5e17912700 1 -- 172.21.15.2:0/452381695 shutdown_connections 2023-08-25T03:33:57.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.739+0000 7f5e17912700 1 --2- 172.21.15.2:0/452381695 >> v2:172.21.15.31:3300/0 conn(0x7f5e1006dff0 0x7f5e1006e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:57.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.739+0000 7f5e17912700 1 --2- 172.21.15.2:0/452381695 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e1015b060 0x7f5e1015b440 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:57.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.739+0000 7f5e17912700 1 -- 172.21.15.2:0/452381695 >> 172.21.15.2:0/452381695 conn(0x7f5e100d0f70 msgr2=0x7f5e100d1350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:57.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.739+0000 7f5e17912700 1 -- 172.21.15.2:0/452381695 shutdown_connections 2023-08-25T03:33:57.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.739+0000 7f5e17912700 1 -- 172.21.15.2:0/452381695 wait complete. 2023-08-25T03:33:57.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.740+0000 7f5e17912700 1 Processor -- start 2023-08-25T03:33:57.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.740+0000 7f5e17912700 1 -- start start 2023-08-25T03:33:57.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.741+0000 7f5e17912700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5e1006dff0 0x7f5e100afcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:57.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.741+0000 7f5e17912700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e100cb5a0 0x7f5e100cf530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:57.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.741+0000 7f5e17912700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e1015b060 0x7f5e100cfa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:57.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.741+0000 7f5e17912700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e101f6920 con 0x7f5e1015b060 2023-08-25T03:33:57.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.741+0000 7f5e17912700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e100b58b0 con 0x7f5e100cb5a0 2023-08-25T03:33:57.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.741+0000 7f5e17912700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e101f5ff0 con 0x7f5e1006dff0 2023-08-25T03:33:57.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.741+0000 7f5e15eaf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e1015b060 0x7f5e100cfa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:57.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.741+0000 7f5e15eaf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e1015b060 0x7f5e100cfa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58960/0 (socket says 172.21.15.2:58960) 2023-08-25T03:33:57.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.741+0000 7f5e15eaf700 1 -- 172.21.15.2:0/1251426910 learned_addr learned my addr 172.21.15.2:0/1251426910 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:33:57.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.741+0000 7f5e156ae700 1 --2- 172.21.15.2:0/1251426910 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5e1006dff0 0x7f5e100afcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:57.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.741+0000 7f5e14ead700 1 --2- 172.21.15.2:0/1251426910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e100cb5a0 0x7f5e100cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:57.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.741+0000 7f5e15eaf700 1 -- 172.21.15.2:0/1251426910 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5e1006dff0 msgr2=0x7f5e100afcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:57.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.741+0000 7f5e15eaf700 1 --2- 172.21.15.2:0/1251426910 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5e1006dff0 0x7f5e100afcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:57.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.741+0000 7f5e15eaf700 1 -- 172.21.15.2:0/1251426910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e100cb5a0 msgr2=0x7f5e100cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:57.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.741+0000 7f5e15eaf700 1 --2- 172.21.15.2:0/1251426910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e100cb5a0 0x7f5e100cf530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:57.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.741+0000 7f5e15eaf700 1 -- 172.21.15.2:0/1251426910 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e00009550 con 0x7f5e1015b060 2023-08-25T03:33:57.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.741+0000 7f5e156ae700 1 --2- 172.21.15.2:0/1251426910 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5e1006dff0 0x7f5e100afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:33:57.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.741+0000 7f5e14ead700 1 --2- 172.21.15.2:0/1251426910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e100cb5a0 0x7f5e100cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:33:57.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.741+0000 7f5e15eaf700 1 --2- 172.21.15.2:0/1251426910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e1015b060 0x7f5e100cfa70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5e08007e40 tx=0x7f5e0800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:57.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.741+0000 7f5e05ffb700 1 -- 172.21.15.2:0/1251426910 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e0800eea0 con 0x7f5e1015b060 2023-08-25T03:33:57.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.741+0000 7f5e05ffb700 1 -- 172.21.15.2:0/1251426910 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e08014420 con 0x7f5e1015b060 2023-08-25T03:33:57.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.741+0000 7f5e17912700 1 -- 172.21.15.2:0/1251426910 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e101f62d0 con 0x7f5e1015b060 2023-08-25T03:33:57.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.741+0000 7f5e05ffb700 1 -- 172.21.15.2:0/1251426910 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e080147b0 con 0x7f5e1015b060 2023-08-25T03:33:57.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.742+0000 7f5e17912700 1 -- 172.21.15.2:0/1251426910 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e101f6db0 con 0x7f5e1015b060 2023-08-25T03:33:57.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.743+0000 7f5e17912700 1 -- 172.21.15.2:0/1251426910 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e101f6ad0 con 0x7f5e1015b060 2023-08-25T03:33:57.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.743+0000 7f5e05ffb700 1 -- 172.21.15.2:0/1251426910 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5e08014910 con 0x7f5e1015b060 2023-08-25T03:33:57.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.743+0000 7f5e05ffb700 1 --2- 172.21.15.2:0/1251426910 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5df406a0e0 0x7f5df406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:57.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.744+0000 7f5e05ffb700 1 -- 172.21.15.2:0/1251426910 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5e08088af0 con 0x7f5e1015b060 2023-08-25T03:33:57.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.744+0000 7f5e156ae700 1 --2- 172.21.15.2:0/1251426910 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5df406a0e0 0x7f5df406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:57.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.744+0000 7f5e156ae700 1 --2- 172.21.15.2:0/1251426910 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5df406a0e0 0x7f5df406c5a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5e1015d310 tx=0x7f5e00008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:57.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.748+0000 7f5e05ffb700 1 -- 172.21.15.2:0/1251426910 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5e101f6ad0 con 0x7f5e1015b060 2023-08-25T03:33:57.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.950+0000 7f5e17912700 1 -- 172.21.15.2:0/1251426910 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e100746d0 con 0x7f5e1015b060 2023-08-25T03:33:57.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.951+0000 7f5e05ffb700 1 -- 172.21.15.2:0/1251426910 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+14505 (secure 0 0 0) 0x7f5e0805d020 con 0x7f5e1015b060 2023-08-25T03:33:57.954 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:57.954 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":19,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:33:57.559145+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:33:57.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.953+0000 7f5e17912700 1 -- 172.21.15.2:0/1251426910 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5df406a0e0 msgr2=0x7f5df406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:57.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.953+0000 7f5e17912700 1 --2- 172.21.15.2:0/1251426910 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5df406a0e0 0x7f5df406c5a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5e1015d310 tx=0x7f5e00008030 comp rx=0 tx=0).stop 2023-08-25T03:33:57.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.953+0000 7f5e17912700 1 -- 172.21.15.2:0/1251426910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e1015b060 msgr2=0x7f5e100cfa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:57.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.953+0000 7f5e17912700 1 --2- 172.21.15.2:0/1251426910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e1015b060 0x7f5e100cfa70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5e08007e40 tx=0x7f5e0800e3c0 comp rx=0 tx=0).stop 2023-08-25T03:33:57.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.954+0000 7f5e17912700 1 -- 172.21.15.2:0/1251426910 shutdown_connections 2023-08-25T03:33:57.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.954+0000 7f5e17912700 1 --2- 172.21.15.2:0/1251426910 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5df406a0e0 0x7f5df406c5a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:57.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.954+0000 7f5e17912700 1 --2- 172.21.15.2:0/1251426910 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5e1015b060 0x7f5e100cfa70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:57.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.954+0000 7f5e17912700 1 --2- 172.21.15.2:0/1251426910 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5e1006dff0 0x7f5e100afcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:57.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.954+0000 7f5e17912700 1 --2- 172.21.15.2:0/1251426910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5e100cb5a0 0x7f5e100cf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:57.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.954+0000 7f5e17912700 1 -- 172.21.15.2:0/1251426910 >> 172.21.15.2:0/1251426910 conn(0x7f5e100d0f70 msgr2=0x7f5e100ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:57.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.954+0000 7f5e17912700 1 -- 172.21.15.2:0/1251426910 shutdown_connections 2023-08-25T03:33:57.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:57.954+0000 7f5e17912700 1 -- 172.21.15.2:0/1251426910 wait complete. 2023-08-25T03:33:57.967 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T03:33:49.560643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T03:33:56.265184+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T03:33:57.968 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T03:33:58.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.094+0000 7f9eaf309700 1 Processor -- start 2023-08-25T03:33:58.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.094+0000 7f9eaf309700 1 -- start start 2023-08-25T03:33:58.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.095+0000 7f9eaf309700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f9ea815cfb0 0x7f9ea80b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:58.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.095+0000 7f9eaf309700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9ea806b120 0x7f9ea806b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:58.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.095+0000 7f9eaf309700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea815d390 con 0x7f9ea8071960 2023-08-25T03:33:58.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.095+0000 7f9eaf309700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea80cefe0 con 0x7f9ea815cfb0 2023-08-25T03:33:58.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.095+0000 7f9eaf309700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea80d0410 con 0x7f9ea806b120 2023-08-25T03:33:58.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.095+0000 7f9ead0a5700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f9ea815cfb0 0x7f9ea80b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:58.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.095+0000 7f9ead8a6700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f9ea8071960 0x7f9ea80b41b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45634/0 (socket says 172.21.15.2:45634) 2023-08-25T03:33:58.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.095+0000 7f9ead0a5700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f9ea815cfb0 0x7f9ea80b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58964/0 (socket says 172.21.15.2:58964) 2023-08-25T03:33:58.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.095+0000 7f9ead8a6700 1 -- 172.21.15.2:0/1121966395 learned_addr learned my addr 172.21.15.2:0/1121966395 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:33:58.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.095+0000 7f9eac8a4700 1 --2- 172.21.15.2:0/1121966395 >> v2:172.21.15.31:3300/0 conn(0x7f9ea806b120 0x7f9ea806b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:58.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.096+0000 7f9ead0a5700 1 -- 172.21.15.2:0/1121966395 >> v2:172.21.15.31:3300/0 conn(0x7f9ea806b120 msgr2=0x7f9ea806b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:33:58.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.096+0000 7f9ead0a5700 1 --2- 172.21.15.2:0/1121966395 >> v2:172.21.15.31:3300/0 conn(0x7f9ea806b120 0x7f9ea806b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:58.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.096+0000 7f9ead0a5700 1 -- 172.21.15.2:0/1121966395 >> v1:172.21.15.2:6789/0 conn(0x7f9ea8071960 legacy=0x7f9ea80b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:58.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.096+0000 7f9ead0a5700 1 -- 172.21.15.2:0/1121966395 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea80ce740 con 0x7f9ea815cfb0 2023-08-25T03:33:58.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.096+0000 7f9e9ffff700 1 -- 172.21.15.2:0/1121966395 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 425901692 0 0) 0x7f9ea815d390 con 0x7f9ea8071960 2023-08-25T03:33:58.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.096+0000 7f9ead0a5700 1 --2- 172.21.15.2:0/1121966395 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ea815cfb0 0x7f9ea80b1990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8073cc0 tx=0x7f9e98009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b604d05c94065822 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:58.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.096+0000 7f9e9ffff700 1 -- 172.21.15.2:0/1121966395 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e98005e30 con 0x7f9ea815cfb0 2023-08-25T03:33:58.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.096+0000 7f9e9ffff700 1 -- 172.21.15.2:0/1121966395 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e98006030 con 0x7f9ea815cfb0 2023-08-25T03:33:58.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.096+0000 7f9e9ffff700 1 -- 172.21.15.2:0/1121966395 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e980063c0 con 0x7f9ea815cfb0 2023-08-25T03:33:58.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.096+0000 7f9eaf309700 1 -- 172.21.15.2:0/1121966395 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ea815cfb0 msgr2=0x7f9ea80b1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:58.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.096+0000 7f9eaf309700 1 --2- 172.21.15.2:0/1121966395 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ea815cfb0 0x7f9ea80b1990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9ea8073cc0 tx=0x7f9e98009b30 comp rx=0 tx=0).stop 2023-08-25T03:33:58.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.096+0000 7f9eaf309700 1 -- 172.21.15.2:0/1121966395 shutdown_connections 2023-08-25T03:33:58.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.096+0000 7f9eaf309700 1 --2- 172.21.15.2:0/1121966395 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ea815cfb0 0x7f9ea80b1990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:58.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.096+0000 7f9eaf309700 1 --2- 172.21.15.2:0/1121966395 >> v2:172.21.15.31:3300/0 conn(0x7f9ea806b120 0x7f9ea806b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:58.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.096+0000 7f9eaf309700 1 -- 172.21.15.2:0/1121966395 >> 172.21.15.2:0/1121966395 conn(0x7f9ea80cf500 msgr2=0x7f9ea80cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:58.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.097+0000 7f9eaf309700 1 -- 172.21.15.2:0/1121966395 shutdown_connections 2023-08-25T03:33:58.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.097+0000 7f9eaf309700 1 -- 172.21.15.2:0/1121966395 wait complete. 2023-08-25T03:33:58.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.098+0000 7f9eaf309700 1 Processor -- start 2023-08-25T03:33:58.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.098+0000 7f9eaf309700 1 -- start start 2023-08-25T03:33:58.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.099+0000 7f9eaf309700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ea806b120 0x7f9ea8098580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:58.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.099+0000 7f9eaf309700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9ea8071960 0x7f9ea8098ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:58.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.099+0000 7f9eaf309700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ea815cfb0 0x7f9ea8153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:58.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.099+0000 7f9eaf309700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea80735a0 con 0x7f9ea806b120 2023-08-25T03:33:58.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.099+0000 7f9eaf309700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea80ad2e0 con 0x7f9ea815cfb0 2023-08-25T03:33:58.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.099+0000 7f9eaf309700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea80ad5c0 con 0x7f9ea8071960 2023-08-25T03:33:58.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.099+0000 7f9ead0a5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ea806b120 0x7f9ea8098580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:58.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.099+0000 7f9ead0a5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ea806b120 0x7f9ea8098580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58970/0 (socket says 172.21.15.2:58970) 2023-08-25T03:33:58.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.099+0000 7f9ead0a5700 1 -- 172.21.15.2:0/2403375547 learned_addr learned my addr 172.21.15.2:0/2403375547 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:33:58.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.099+0000 7f9ead8a6700 1 --2- 172.21.15.2:0/2403375547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ea815cfb0 0x7f9ea8153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:58.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.099+0000 7f9ead0a5700 1 -- 172.21.15.2:0/2403375547 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9ea8071960 msgr2=0x7f9ea8098ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:58.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.099+0000 7f9ead0a5700 1 --2- 172.21.15.2:0/2403375547 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9ea8071960 0x7f9ea8098ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:58.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.099+0000 7f9ead0a5700 1 -- 172.21.15.2:0/2403375547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ea815cfb0 msgr2=0x7f9ea8153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:58.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.099+0000 7f9ead0a5700 1 --2- 172.21.15.2:0/2403375547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ea815cfb0 0x7f9ea8153670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:58.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.099+0000 7f9ead0a5700 1 -- 172.21.15.2:0/2403375547 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e98009430 con 0x7f9ea806b120 2023-08-25T03:33:58.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.099+0000 7f9ead0a5700 1 --2- 172.21.15.2:0/2403375547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ea806b120 0x7f9ea8098580 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9e98000c00 tx=0x7f9e98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:58.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.099+0000 7f9e9dffb700 1 -- 172.21.15.2:0/2403375547 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e98004240 con 0x7f9ea806b120 2023-08-25T03:33:58.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.099+0000 7f9e9dffb700 1 -- 172.21.15.2:0/2403375547 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e980043a0 con 0x7f9ea806b120 2023-08-25T03:33:58.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.099+0000 7f9eaf309700 1 -- 172.21.15.2:0/2403375547 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea80ad8a0 con 0x7f9ea806b120 2023-08-25T03:33:58.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.099+0000 7f9e9dffb700 1 -- 172.21.15.2:0/2403375547 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e980036a0 con 0x7f9ea806b120 2023-08-25T03:33:58.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.100+0000 7f9eaf309700 1 -- 172.21.15.2:0/2403375547 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea80ade60 con 0x7f9ea806b120 2023-08-25T03:33:58.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.101+0000 7f9eaf309700 1 -- 172.21.15.2:0/2403375547 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea80746d0 con 0x7f9ea806b120 2023-08-25T03:33:58.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.101+0000 7f9e9dffb700 1 -- 172.21.15.2:0/2403375547 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9e98022070 con 0x7f9ea806b120 2023-08-25T03:33:58.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.101+0000 7f9e9dffb700 1 --2- 172.21.15.2:0/2403375547 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9e8c06a0e0 0x7f9e8c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:58.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.101+0000 7f9e9dffb700 1 -- 172.21.15.2:0/2403375547 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9e98089aa0 con 0x7f9ea806b120 2023-08-25T03:33:58.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.101+0000 7f9eac8a4700 1 --2- 172.21.15.2:0/2403375547 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9e8c06a0e0 0x7f9e8c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:58.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.102+0000 7f9eac8a4700 1 --2- 172.21.15.2:0/2403375547 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9e8c06a0e0 0x7f9e8c06c5a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4006b20 tx=0x7f9ea4006ab0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:58.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.106+0000 7f9e9dffb700 1 -- 172.21.15.2:0/2403375547 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9e98055570 con 0x7f9ea806b120 2023-08-25T03:33:58.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.305+0000 7f9eaf309700 1 -- 172.21.15.2:0/2403375547 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9ea80cd9c0 con 0x7f9ea806b120 2023-08-25T03:33:58.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.306+0000 7f9e9dffb700 1 -- 172.21.15.2:0/2403375547 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9e98058b90 con 0x7f9ea806b120 2023-08-25T03:33:58.308 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-08-25T03:33:58.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.308+0000 7f9eaf309700 1 -- 172.21.15.2:0/2403375547 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9e8c06a0e0 msgr2=0x7f9e8c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:58.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.308+0000 7f9eaf309700 1 --2- 172.21.15.2:0/2403375547 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9e8c06a0e0 0x7f9e8c06c5a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9ea4006b20 tx=0x7f9ea4006ab0 comp rx=0 tx=0).stop 2023-08-25T03:33:58.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.308+0000 7f9eaf309700 1 -- 172.21.15.2:0/2403375547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ea806b120 msgr2=0x7f9ea8098580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:58.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.308+0000 7f9eaf309700 1 --2- 172.21.15.2:0/2403375547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ea806b120 0x7f9ea8098580 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9e98000c00 tx=0x7f9e98004750 comp rx=0 tx=0).stop 2023-08-25T03:33:58.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.308+0000 7f9eaf309700 1 -- 172.21.15.2:0/2403375547 shutdown_connections 2023-08-25T03:33:58.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.308+0000 7f9eaf309700 1 --2- 172.21.15.2:0/2403375547 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9e8c06a0e0 0x7f9e8c06c5a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:58.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.308+0000 7f9eaf309700 1 --2- 172.21.15.2:0/2403375547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ea806b120 0x7f9ea8098580 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:58.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.308+0000 7f9eaf309700 1 --2- 172.21.15.2:0/2403375547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ea815cfb0 0x7f9ea8153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:58.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.308+0000 7f9eaf309700 1 --2- 172.21.15.2:0/2403375547 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9ea8071960 0x7f9ea8098ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:58.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.308+0000 7f9eaf309700 1 -- 172.21.15.2:0/2403375547 >> 172.21.15.2:0/2403375547 conn(0x7f9ea80cf500 msgr2=0x7f9ea8157980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:58.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.308+0000 7f9eaf309700 1 -- 172.21.15.2:0/2403375547 shutdown_connections 2023-08-25T03:33:58.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.308+0000 7f9eaf309700 1 -- 172.21.15.2:0/2403375547 wait complete. 2023-08-25T03:33:58.322 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T03:33:58.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.448+0000 7f093a6c6700 1 Processor -- start 2023-08-25T03:33:58.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.448+0000 7f093a6c6700 1 -- start start 2023-08-25T03:33:58.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.449+0000 7f093a6c6700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f09340af180 0x7f0934070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:58.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.449+0000 7f093a6c6700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0934160bd0 con 0x7f09341607f0 2023-08-25T03:33:58.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.449+0000 7f093a6c6700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09340cdd00 con 0x7f093406a1e0 2023-08-25T03:33:58.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.449+0000 7f093a6c6700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09340713a0 con 0x7f09340af180 2023-08-25T03:33:58.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.449+0000 7f09337fe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f09340af180 0x7f0934070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:58.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.449+0000 7f0933fff700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f09341607f0 0x7f09340ac8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45644/0 (socket says 172.21.15.2:45644) 2023-08-25T03:33:58.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.449+0000 7f0933fff700 1 -- 172.21.15.2:0/1907504767 learned_addr learned my addr 172.21.15.2:0/1907504767 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:33:58.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.449+0000 7f09337fe700 1 -- 172.21.15.2:0/1907504767 >> v1:172.21.15.31:6789/0 conn(0x7f093406a1e0 legacy=0x7f093406ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:58.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.450+0000 7f09337fe700 1 -- 172.21.15.2:0/1907504767 >> v1:172.21.15.2:6789/0 conn(0x7f09341607f0 legacy=0x7f09340ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:58.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.450+0000 7f09337fe700 1 -- 172.21.15.2:0/1907504767 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09340cda00 con 0x7f09340af180 2023-08-25T03:33:58.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.450+0000 7f0932ffd700 1 -- 172.21.15.2:0/1907504767 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1366735109 0 0) 0x7f0934160bd0 con 0x7f09341607f0 2023-08-25T03:33:58.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.450+0000 7f09337fe700 1 --2- 172.21.15.2:0/1907504767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09340af180 0x7f0934070e60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f091c009840 tx=0x7f091c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38dee5fd001fafc4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:58.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.450+0000 7f0932ffd700 1 -- 172.21.15.2:0/1907504767 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c011830 con 0x7f09340af180 2023-08-25T03:33:58.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.450+0000 7f0932ffd700 1 -- 172.21.15.2:0/1907504767 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091c011990 con 0x7f09340af180 2023-08-25T03:33:58.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.450+0000 7f0932ffd700 1 -- 172.21.15.2:0/1907504767 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f091c011d20 con 0x7f09340af180 2023-08-25T03:33:58.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.450+0000 7f093a6c6700 1 -- 172.21.15.2:0/1907504767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09340af180 msgr2=0x7f0934070e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:58.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.450+0000 7f093a6c6700 1 --2- 172.21.15.2:0/1907504767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09340af180 0x7f0934070e60 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f091c009840 tx=0x7f091c009b50 comp rx=0 tx=0).stop 2023-08-25T03:33:58.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.450+0000 7f093a6c6700 1 -- 172.21.15.2:0/1907504767 shutdown_connections 2023-08-25T03:33:58.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.450+0000 7f093a6c6700 1 --2- 172.21.15.2:0/1907504767 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09340af180 0x7f0934070e60 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:58.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.450+0000 7f093a6c6700 1 -- 172.21.15.2:0/1907504767 >> 172.21.15.2:0/1907504767 conn(0x7f09340cfd90 msgr2=0x7f09340d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:58.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.450+0000 7f093a6c6700 1 -- 172.21.15.2:0/1907504767 shutdown_connections 2023-08-25T03:33:58.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.450+0000 7f093a6c6700 1 -- 172.21.15.2:0/1907504767 wait complete. 2023-08-25T03:33:58.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.452+0000 7f093a6c6700 1 Processor -- start 2023-08-25T03:33:58.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.452+0000 7f093a6c6700 1 -- start start 2023-08-25T03:33:58.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.453+0000 7f093a6c6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f093406a1e0 0x7f09340c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:58.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.453+0000 7f093a6c6700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f09340af180 0x7f09340c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:58.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.453+0000 7f093a6c6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09341607f0 0x7f09341f6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:58.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.453+0000 7f093a6c6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09340c6280 con 0x7f09341607f0 2023-08-25T03:33:58.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.453+0000 7f093a6c6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09340c4a90 con 0x7f093406a1e0 2023-08-25T03:33:58.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.453+0000 7f093a6c6700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09340c41a0 con 0x7f09340af180 2023-08-25T03:33:58.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.453+0000 7f0938c63700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09341607f0 0x7f09341f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:58.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.453+0000 7f0938c63700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09341607f0 0x7f09341f6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58986/0 (socket says 172.21.15.2:58986) 2023-08-25T03:33:58.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.453+0000 7f0938c63700 1 -- 172.21.15.2:0/3797523178 learned_addr learned my addr 172.21.15.2:0/3797523178 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:33:58.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.453+0000 7f0933fff700 1 --2- 172.21.15.2:0/3797523178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f093406a1e0 0x7f09340c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:58.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.453+0000 7f09337fe700 1 --2- 172.21.15.2:0/3797523178 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f09340af180 0x7f09340c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:58.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.453+0000 7f0938c63700 1 -- 172.21.15.2:0/3797523178 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f09340af180 msgr2=0x7f09340c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:58.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.453+0000 7f0938c63700 1 --2- 172.21.15.2:0/3797523178 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f09340af180 0x7f09340c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:58.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.453+0000 7f0938c63700 1 -- 172.21.15.2:0/3797523178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f093406a1e0 msgr2=0x7f09340c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:58.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.453+0000 7f0938c63700 1 --2- 172.21.15.2:0/3797523178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f093406a1e0 0x7f09340c99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:58.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.453+0000 7f0938c63700 1 -- 172.21.15.2:0/3797523178 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f091c0094a0 con 0x7f09341607f0 2023-08-25T03:33:58.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.453+0000 7f0938c63700 1 --2- 172.21.15.2:0/3797523178 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09341607f0 0x7f09341f6070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f092c00cb90 tx=0x7f092c00cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:58.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.453+0000 7f0930ff9700 1 -- 172.21.15.2:0/3797523178 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c003c50 con 0x7f09341607f0 2023-08-25T03:33:58.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.453+0000 7f0930ff9700 1 -- 172.21.15.2:0/3797523178 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f092c003dd0 con 0x7f09341607f0 2023-08-25T03:33:58.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.453+0000 7f093a6c6700 1 -- 172.21.15.2:0/3797523178 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09340c4440 con 0x7f09341607f0 2023-08-25T03:33:58.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.453+0000 7f0930ff9700 1 -- 172.21.15.2:0/3797523178 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092c00aae0 con 0x7f09341607f0 2023-08-25T03:33:58.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.453+0000 7f093a6c6700 1 -- 172.21.15.2:0/3797523178 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09340c6470 con 0x7f09341607f0 2023-08-25T03:33:58.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.454+0000 7f093a6c6700 1 -- 172.21.15.2:0/3797523178 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09340c4c20 con 0x7f09341607f0 2023-08-25T03:33:58.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.455+0000 7f0930ff9700 1 -- 172.21.15.2:0/3797523178 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f092c00ac40 con 0x7f09341607f0 2023-08-25T03:33:58.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.455+0000 7f0930ff9700 1 --2- 172.21.15.2:0/3797523178 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f091406a0e0 0x7f091406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:58.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.455+0000 7f0930ff9700 1 -- 172.21.15.2:0/3797523178 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f092c088130 con 0x7f09341607f0 2023-08-25T03:33:58.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.455+0000 7f0933fff700 1 --2- 172.21.15.2:0/3797523178 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f091406a0e0 0x7f091406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:58.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.456+0000 7f0933fff700 1 --2- 172.21.15.2:0/3797523178 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f091406a0e0 0x7f091406c5a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0924000f60 tx=0x7f0924004a50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:58.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.460+0000 7f0930ff9700 1 -- 172.21.15.2:0/3797523178 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f092c053c00 con 0x7f09341607f0 2023-08-25T03:33:58.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.661+0000 7f093a6c6700 1 -- 172.21.15.2:0/3797523178 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f09340746d0 con 0x7f09341607f0 2023-08-25T03:33:58.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.661+0000 7f0930ff9700 1 -- 172.21.15.2:0/3797523178 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f092c057220 con 0x7f09341607f0 2023-08-25T03:33:58.663 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 8 2023-08-25T03:33:58.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.663+0000 7f093a6c6700 1 -- 172.21.15.2:0/3797523178 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f091406a0e0 msgr2=0x7f091406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:58.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.663+0000 7f093a6c6700 1 --2- 172.21.15.2:0/3797523178 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f091406a0e0 0x7f091406c5a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0924000f60 tx=0x7f0924004a50 comp rx=0 tx=0).stop 2023-08-25T03:33:58.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.663+0000 7f093a6c6700 1 -- 172.21.15.2:0/3797523178 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09341607f0 msgr2=0x7f09341f6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:58.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.663+0000 7f093a6c6700 1 --2- 172.21.15.2:0/3797523178 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09341607f0 0x7f09341f6070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f092c00cb90 tx=0x7f092c00cea0 comp rx=0 tx=0).stop 2023-08-25T03:33:58.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.663+0000 7f093a6c6700 1 -- 172.21.15.2:0/3797523178 shutdown_connections 2023-08-25T03:33:58.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.663+0000 7f093a6c6700 1 --2- 172.21.15.2:0/3797523178 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f091406a0e0 0x7f091406c5a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:58.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.663+0000 7f093a6c6700 1 --2- 172.21.15.2:0/3797523178 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09341607f0 0x7f09341f6070 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:58.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.663+0000 7f093a6c6700 1 --2- 172.21.15.2:0/3797523178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f093406a1e0 0x7f09340c99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:58.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.663+0000 7f093a6c6700 1 --2- 172.21.15.2:0/3797523178 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f09340af180 0x7f09340c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:58.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.663+0000 7f093a6c6700 1 -- 172.21.15.2:0/3797523178 >> 172.21.15.2:0/3797523178 conn(0x7f09340cfd90 msgr2=0x7f093406eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:58.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.663+0000 7f093a6c6700 1 -- 172.21.15.2:0/3797523178 shutdown_connections 2023-08-25T03:33:58.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.663+0000 7f093a6c6700 1 -- 172.21.15.2:0/3797523178 wait complete. 2023-08-25T03:33:58.677 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T03:33:58.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.801+0000 7f3509f05700 1 Processor -- start 2023-08-25T03:33:58.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.801+0000 7f3509f05700 1 -- start start 2023-08-25T03:33:58.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.802+0000 7f3509f05700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f3504153ea0 0x7f35040cbf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:58.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.802+0000 7f3509f05700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3504159cf0 0x7f35040b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:58.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.802+0000 7f3509f05700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f350406b1f0 0x7f350406b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:58.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.802+0000 7f3509f05700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f350406bbc0 con 0x7f3504153ea0 2023-08-25T03:33:58.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.802+0000 7f3509f05700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35040cc4b0 con 0x7f350406b1f0 2023-08-25T03:33:58.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.802+0000 7f3509f05700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f350415a1f0 con 0x7f3504159cf0 2023-08-25T03:33:58.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.802+0000 7f35037fe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f3504153ea0 0x7f35040cbf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:58.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.802+0000 7f35037fe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f3504153ea0 0x7f35040cbf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58992/0 (socket says 172.21.15.2:58992) 2023-08-25T03:33:58.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.802+0000 7f35037fe700 1 -- 172.21.15.2:0/1538818494 learned_addr learned my addr 172.21.15.2:0/1538818494 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:33:58.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.802+0000 7f3503fff700 1 --2- 172.21.15.2:0/1538818494 >> v2:172.21.15.31:3300/0 conn(0x7f350406b1f0 0x7f350406b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:58.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.802+0000 7f3502ffd700 1 --2- 172.21.15.2:0/1538818494 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3504159cf0 0x7f35040b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:58.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.802+0000 7f35037fe700 1 -- 172.21.15.2:0/1538818494 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3504159cf0 msgr2=0x7f35040b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:33:58.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.803+0000 7f35037fe700 1 --2- 172.21.15.2:0/1538818494 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3504159cf0 0x7f35040b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:58.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.803+0000 7f35037fe700 1 -- 172.21.15.2:0/1538818494 >> v2:172.21.15.31:3300/0 conn(0x7f350406b1f0 msgr2=0x7f350406b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:33:58.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.803+0000 7f35037fe700 1 --2- 172.21.15.2:0/1538818494 >> v2:172.21.15.31:3300/0 conn(0x7f350406b1f0 0x7f350406b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:58.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.803+0000 7f35037fe700 1 -- 172.21.15.2:0/1538818494 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f350415a4d0 con 0x7f3504153ea0 2023-08-25T03:33:58.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.803+0000 7f3503fff700 1 --2- 172.21.15.2:0/1538818494 >> v2:172.21.15.31:3300/0 conn(0x7f350406b1f0 0x7f350406b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:33:58.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.803+0000 7f3502ffd700 1 --2- 172.21.15.2:0/1538818494 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3504159cf0 0x7f35040b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:33:58.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.803+0000 7f35037fe700 1 --2- 172.21.15.2:0/1538818494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3504153ea0 0x7f35040cbf70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3504070620 tx=0x7f34f400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe2169588eccfe9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:58.808 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.803+0000 7f35027fc700 1 -- 172.21.15.2:0/1538818494 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f400dc20 con 0x7f3504153ea0 2023-08-25T03:33:58.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.803+0000 7f35027fc700 1 -- 172.21.15.2:0/1538818494 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f400dd80 con 0x7f3504153ea0 2023-08-25T03:33:58.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.803+0000 7f35027fc700 1 -- 172.21.15.2:0/1538818494 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f4017630 con 0x7f3504153ea0 2023-08-25T03:33:58.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.803+0000 7f3509f05700 1 -- 172.21.15.2:0/1538818494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3504153ea0 msgr2=0x7f35040cbf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:58.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.803+0000 7f3509f05700 1 --2- 172.21.15.2:0/1538818494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3504153ea0 0x7f35040cbf70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3504070620 tx=0x7f34f400be70 comp rx=0 tx=0).stop 2023-08-25T03:33:58.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.803+0000 7f3509f05700 1 -- 172.21.15.2:0/1538818494 shutdown_connections 2023-08-25T03:33:58.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.803+0000 7f3509f05700 1 --2- 172.21.15.2:0/1538818494 >> v2:172.21.15.31:3300/0 conn(0x7f350406b1f0 0x7f350406b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:58.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.803+0000 7f3509f05700 1 --2- 172.21.15.2:0/1538818494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3504153ea0 0x7f35040cbf70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:58.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.803+0000 7f3509f05700 1 --2- 172.21.15.2:0/1538818494 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3504159cf0 0x7f35040b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:58.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.803+0000 7f3509f05700 1 -- 172.21.15.2:0/1538818494 >> 172.21.15.2:0/1538818494 conn(0x7f35040cfd10 msgr2=0x7f350406ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:58.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.803+0000 7f3509f05700 1 -- 172.21.15.2:0/1538818494 shutdown_connections 2023-08-25T03:33:58.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.804+0000 7f3509f05700 1 -- 172.21.15.2:0/1538818494 wait complete. 2023-08-25T03:33:58.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.805+0000 7f3509f05700 1 Processor -- start 2023-08-25T03:33:58.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.805+0000 7f3509f05700 1 -- start start 2023-08-25T03:33:58.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.806+0000 7f3509f05700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f350406b1f0 0x7f3504140fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:58.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.806+0000 7f3509f05700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3504153ea0 0x7f350415acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:58.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.806+0000 7f3509f05700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3504159cf0 0x7f3504158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:58.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.806+0000 7f3509f05700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35041f7260 con 0x7f350406b1f0 2023-08-25T03:33:58.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.806+0000 7f3509f05700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35041448d0 con 0x7f3504159cf0 2023-08-25T03:33:58.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.806+0000 7f3509f05700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35041f66e0 con 0x7f3504153ea0 2023-08-25T03:33:58.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.806+0000 7f35037fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f350406b1f0 0x7f3504140fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:58.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.806+0000 7f35037fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f350406b1f0 0x7f3504140fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:58998/0 (socket says 172.21.15.2:58998) 2023-08-25T03:33:58.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.806+0000 7f35037fe700 1 -- 172.21.15.2:0/3584812822 learned_addr learned my addr 172.21.15.2:0/3584812822 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:33:58.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.806+0000 7f3503fff700 1 --2- 172.21.15.2:0/3584812822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3504159cf0 0x7f3504158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:58.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.806+0000 7f3502ffd700 1 --2- 172.21.15.2:0/3584812822 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3504153ea0 0x7f350415acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:58.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.806+0000 7f35037fe700 1 -- 172.21.15.2:0/3584812822 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3504153ea0 msgr2=0x7f350415acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:58.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.806+0000 7f35037fe700 1 --2- 172.21.15.2:0/3584812822 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3504153ea0 0x7f350415acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:58.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.806+0000 7f35037fe700 1 -- 172.21.15.2:0/3584812822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3504159cf0 msgr2=0x7f3504158690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:58.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.806+0000 7f35037fe700 1 --2- 172.21.15.2:0/3584812822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3504159cf0 0x7f3504158690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:58.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.806+0000 7f35037fe700 1 -- 172.21.15.2:0/3584812822 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f400b7f0 con 0x7f350406b1f0 2023-08-25T03:33:58.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.806+0000 7f35037fe700 1 --2- 172.21.15.2:0/3584812822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f350406b1f0 0x7f3504140fa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3504072940 tx=0x7f34f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:58.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.806+0000 7f34ebfff700 1 -- 172.21.15.2:0/3584812822 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f4009050 con 0x7f350406b1f0 2023-08-25T03:33:58.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.806+0000 7f34ebfff700 1 -- 172.21.15.2:0/3584812822 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f400a040 con 0x7f350406b1f0 2023-08-25T03:33:58.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.806+0000 7f3509f05700 1 -- 172.21.15.2:0/3584812822 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35041f69c0 con 0x7f350406b1f0 2023-08-25T03:33:58.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.806+0000 7f34ebfff700 1 -- 172.21.15.2:0/3584812822 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f4021870 con 0x7f350406b1f0 2023-08-25T03:33:58.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.807+0000 7f3509f05700 1 -- 172.21.15.2:0/3584812822 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35041f6f80 con 0x7f350406b1f0 2023-08-25T03:33:58.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.808+0000 7f3509f05700 1 -- 172.21.15.2:0/3584812822 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35040746d0 con 0x7f350406b1f0 2023-08-25T03:33:58.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.808+0000 7f34ebfff700 1 -- 172.21.15.2:0/3584812822 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f34f4015070 con 0x7f350406b1f0 2023-08-25T03:33:58.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.808+0000 7f34ebfff700 1 --2- 172.21.15.2:0/3584812822 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f34f006a0e0 0x7f34f006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:58.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.808+0000 7f34ebfff700 1 -- 172.21.15.2:0/3584812822 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f34f4089a00 con 0x7f350406b1f0 2023-08-25T03:33:58.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.808+0000 7f3502ffd700 1 --2- 172.21.15.2:0/3584812822 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f34f006a0e0 0x7f34f006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:58.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.809+0000 7f3502ffd700 1 --2- 172.21.15.2:0/3584812822 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f34f006a0e0 0x7f34f006c5a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f34f8006fd0 tx=0x7f34f8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:58.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:58.813+0000 7f34ebfff700 1 -- 172.21.15.2:0/3584812822 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f34f40555f0 con 0x7f350406b1f0 2023-08-25T03:33:59.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.062+0000 7f3509f05700 1 -- 172.21.15.2:0/3584812822 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35040b3aa0 con 0x7f350406b1f0 2023-08-25T03:33:59.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.063+0000 7f34ebfff700 1 -- 172.21.15.2:0/3584812822 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3682 (secure 0 0 0) 0x7f34f4006d20 con 0x7f350406b1f0 2023-08-25T03:33:59.065 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:59.065 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2074312983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2074312983},{"type":"v1","addr":"172.21.15.2:6837","nonce":2074312983}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4059180922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4059180922},{"type":"v1","addr":"172.21.15.31:6835","nonce":4059180922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/922384230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":922384230},{"type":"v1","addr":"172.21.15.31:6834","nonce":922384230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4299,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6835/193990892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":193990892},{"type":"v1","addr":"172.21.15.2:6835","nonce":193990892}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-08-25T03:33:59.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.065+0000 7f3509f05700 1 -- 172.21.15.2:0/3584812822 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f34f006a0e0 msgr2=0x7f34f006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:59.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.065+0000 7f3509f05700 1 --2- 172.21.15.2:0/3584812822 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f34f006a0e0 0x7f34f006c5a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f34f8006fd0 tx=0x7f34f8008040 comp rx=0 tx=0).stop 2023-08-25T03:33:59.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.065+0000 7f3509f05700 1 -- 172.21.15.2:0/3584812822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f350406b1f0 msgr2=0x7f3504140fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:59.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.065+0000 7f3509f05700 1 --2- 172.21.15.2:0/3584812822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f350406b1f0 0x7f3504140fa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3504072940 tx=0x7f34f4004750 comp rx=0 tx=0).stop 2023-08-25T03:33:59.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.065+0000 7f3509f05700 1 -- 172.21.15.2:0/3584812822 shutdown_connections 2023-08-25T03:33:59.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.065+0000 7f3509f05700 1 --2- 172.21.15.2:0/3584812822 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f34f006a0e0 0x7f34f006c5a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:59.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.065+0000 7f3509f05700 1 --2- 172.21.15.2:0/3584812822 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f350406b1f0 0x7f3504140fa0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:59.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.065+0000 7f3509f05700 1 --2- 172.21.15.2:0/3584812822 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3504159cf0 0x7f3504158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:59.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.065+0000 7f3509f05700 1 --2- 172.21.15.2:0/3584812822 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3504153ea0 0x7f350415acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:59.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.065+0000 7f3509f05700 1 -- 172.21.15.2:0/3584812822 >> 172.21.15.2:0/3584812822 conn(0x7f35040cfd10 msgr2=0x7f35040cc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:59.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.065+0000 7f3509f05700 1 -- 172.21.15.2:0/3584812822 shutdown_connections 2023-08-25T03:33:59.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.065+0000 7f3509f05700 1 -- 172.21.15.2:0/3584812822 wait complete. 2023-08-25T03:33:59.070 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 3 2023-08-25T03:33:59.079 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:33:59.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.201+0000 7f8d7c80c700 1 Processor -- start 2023-08-25T03:33:59.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.201+0000 7f8d7c80c700 1 -- start start 2023-08-25T03:33:59.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.202+0000 7f8d7c80c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8d780cdb70 0x7f8d7806b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:59.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.202+0000 7f8d7c80c700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8d780ce000 0x7f8d7806be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:59.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.202+0000 7f8d7c80c700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d78161620 con 0x7f8d780b1fb0 2023-08-25T03:33:59.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.202+0000 7f8d7c80c700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d7806d450 con 0x7f8d780cdb70 2023-08-25T03:33:59.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.202+0000 7f8d7c80c700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d7806d730 con 0x7f8d780ce000 2023-08-25T03:33:59.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.202+0000 7f8d75d9b700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8d780cdb70 0x7f8d7806b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:59.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.202+0000 7f8d75d9b700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8d780cdb70 0x7f8d7806b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59008/0 (socket says 172.21.15.2:59008) 2023-08-25T03:33:59.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.202+0000 7f8d75d9b700 1 -- 172.21.15.2:0/3651846250 learned_addr learned my addr 172.21.15.2:0/3651846250 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:33:59.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.202+0000 7f8d76d9d700 1 --2- 172.21.15.2:0/3651846250 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8d780ce000 0x7f8d7806be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:59.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.202+0000 7f8d75d9b700 1 -- 172.21.15.2:0/3651846250 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8d780ce000 msgr2=0x7f8d7806be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:33:59.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.203+0000 7f8d75d9b700 1 --2- 172.21.15.2:0/3651846250 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8d780ce000 0x7f8d7806be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:59.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.203+0000 7f8d75d9b700 1 -- 172.21.15.2:0/3651846250 >> v1:172.21.15.31:6789/0 conn(0x7f8d780b1fb0 legacy=0x7f8d78160f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:33:59.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.203+0000 7f8d75d9b700 1 -- 172.21.15.2:0/3651846250 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d7806da10 con 0x7f8d780cdb70 2023-08-25T03:33:59.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.203+0000 7f8d75d9b700 1 --2- 172.21.15.2:0/3651846250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d780cdb70 0x7f8d7806b8c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8d680098f0 tx=0x7f8d68009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978546acffb35bf9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:59.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.203+0000 7f8d7559a700 1 -- 172.21.15.2:0/3651846250 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d68004230 con 0x7f8d780cdb70 2023-08-25T03:33:59.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.203+0000 7f8d7559a700 1 -- 172.21.15.2:0/3651846250 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d68004390 con 0x7f8d780cdb70 2023-08-25T03:33:59.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.203+0000 7f8d7559a700 1 -- 172.21.15.2:0/3651846250 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d680036a0 con 0x7f8d780cdb70 2023-08-25T03:33:59.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.203+0000 7f8d7c80c700 1 -- 172.21.15.2:0/3651846250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d780cdb70 msgr2=0x7f8d7806b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:59.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.203+0000 7f8d7c80c700 1 --2- 172.21.15.2:0/3651846250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d780cdb70 0x7f8d7806b8c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8d680098f0 tx=0x7f8d68009c00 comp rx=0 tx=0).stop 2023-08-25T03:33:59.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.204+0000 7f8d7c80c700 1 -- 172.21.15.2:0/3651846250 shutdown_connections 2023-08-25T03:33:59.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.204+0000 7f8d7c80c700 1 --2- 172.21.15.2:0/3651846250 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8d780ce000 0x7f8d7806be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:59.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.204+0000 7f8d7c80c700 1 --2- 172.21.15.2:0/3651846250 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d780cdb70 0x7f8d7806b8c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:59.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.204+0000 7f8d7c80c700 1 -- 172.21.15.2:0/3651846250 >> 172.21.15.2:0/3651846250 conn(0x7f8d780cfea0 msgr2=0x7f8d780d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:59.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.204+0000 7f8d7c80c700 1 -- 172.21.15.2:0/3651846250 shutdown_connections 2023-08-25T03:33:59.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.204+0000 7f8d7c80c700 1 -- 172.21.15.2:0/3651846250 wait complete. 2023-08-25T03:33:59.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.205+0000 7f8d7c80c700 1 Processor -- start 2023-08-25T03:33:59.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.205+0000 7f8d7c80c700 1 -- start start 2023-08-25T03:33:59.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.206+0000 7f8d7c80c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d780b1fb0 0x7f8d78151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:59.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.206+0000 7f8d7c80c700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8d780cdb70 0x7f8d78156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:59.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.206+0000 7f8d7c80c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d780ce000 0x7f8d78156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:59.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.206+0000 7f8d7c80c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d780b39a0 con 0x7f8d780ce000 2023-08-25T03:33:59.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.206+0000 7f8d7c80c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d780c9310 con 0x7f8d780b1fb0 2023-08-25T03:33:59.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.206+0000 7f8d7c80c700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d780c8e80 con 0x7f8d780cdb70 2023-08-25T03:33:59.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.206+0000 7f8d76d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d780ce000 0x7f8d78156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:59.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.206+0000 7f8d76d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d780ce000 0x7f8d78156f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59016/0 (socket says 172.21.15.2:59016) 2023-08-25T03:33:59.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.206+0000 7f8d76d9d700 1 -- 172.21.15.2:0/1727332720 learned_addr learned my addr 172.21.15.2:0/1727332720 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:33:59.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.206+0000 7f8d7659c700 1 --2- 172.21.15.2:0/1727332720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d780b1fb0 0x7f8d78151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:59.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.206+0000 7f8d75d9b700 1 --2- 172.21.15.2:0/1727332720 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8d780cdb70 0x7f8d78156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:59.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.206+0000 7f8d76d9d700 1 -- 172.21.15.2:0/1727332720 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8d780cdb70 msgr2=0x7f8d78156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:59.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.206+0000 7f8d76d9d700 1 --2- 172.21.15.2:0/1727332720 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8d780cdb70 0x7f8d78156a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:59.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.206+0000 7f8d76d9d700 1 -- 172.21.15.2:0/1727332720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d780b1fb0 msgr2=0x7f8d78151910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:59.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.206+0000 7f8d76d9d700 1 --2- 172.21.15.2:0/1727332720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d780b1fb0 0x7f8d78151910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:59.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.206+0000 7f8d76d9d700 1 -- 172.21.15.2:0/1727332720 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d68009550 con 0x7f8d780ce000 2023-08-25T03:33:59.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.206+0000 7f8d76d9d700 1 --2- 172.21.15.2:0/1727332720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d780ce000 0x7f8d78156f60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000cda0 tx=0x7f8d7000a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:59.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.206+0000 7f8d66ffd700 1 -- 172.21.15.2:0/1727332720 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d70003a20 con 0x7f8d780ce000 2023-08-25T03:33:59.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.206+0000 7f8d66ffd700 1 -- 172.21.15.2:0/1727332720 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d70003b80 con 0x7f8d780ce000 2023-08-25T03:33:59.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.206+0000 7f8d66ffd700 1 -- 172.21.15.2:0/1727332720 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d70004020 con 0x7f8d780ce000 2023-08-25T03:33:59.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.206+0000 7f8d7c80c700 1 -- 172.21.15.2:0/1727332720 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d780c8550 con 0x7f8d780ce000 2023-08-25T03:33:59.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.207+0000 7f8d7c80c700 1 -- 172.21.15.2:0/1727332720 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d780c8b10 con 0x7f8d780ce000 2023-08-25T03:33:59.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.207+0000 7f8d7c80c700 1 -- 172.21.15.2:0/1727332720 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d78066f70 con 0x7f8d780ce000 2023-08-25T03:33:59.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.208+0000 7f8d66ffd700 1 -- 172.21.15.2:0/1727332720 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8d70004180 con 0x7f8d780ce000 2023-08-25T03:33:59.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.208+0000 7f8d66ffd700 1 --2- 172.21.15.2:0/1727332720 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8d5806a0e0 0x7f8d5806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:59.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.208+0000 7f8d66ffd700 1 -- 172.21.15.2:0/1727332720 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f8d70012070 con 0x7f8d780ce000 2023-08-25T03:33:59.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.208+0000 7f8d7659c700 1 --2- 172.21.15.2:0/1727332720 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8d5806a0e0 0x7f8d5806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:59.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.209+0000 7f8d7659c700 1 --2- 172.21.15.2:0/1727332720 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8d5806a0e0 0x7f8d5806c5a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009870 tx=0x7f8d6c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:59.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.213+0000 7f8d66ffd700 1 -- 172.21.15.2:0/1727332720 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8d70054470 con 0x7f8d780ce000 2023-08-25T03:33:59.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.415+0000 7f8d7c80c700 1 -- 172.21.15.2:0/1727332720 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d780c8ff0 con 0x7f8d780ce000 2023-08-25T03:33:59.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.416+0000 7f8d66ffd700 1 -- 172.21.15.2:0/1727332720 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+14492 (secure 0 0 0) 0x7f8d70057a90 con 0x7f8d780ce000 2023-08-25T03:33:59.418 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:33:59.418 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":20,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:33:58.563549+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:33:59.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.418+0000 7f8d7c80c700 1 -- 172.21.15.2:0/1727332720 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8d5806a0e0 msgr2=0x7f8d5806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:59.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.418+0000 7f8d7c80c700 1 --2- 172.21.15.2:0/1727332720 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8d5806a0e0 0x7f8d5806c5a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c009870 tx=0x7f8d6c008040 comp rx=0 tx=0).stop 2023-08-25T03:33:59.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.418+0000 7f8d7c80c700 1 -- 172.21.15.2:0/1727332720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d780ce000 msgr2=0x7f8d78156f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:59.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.418+0000 7f8d7c80c700 1 --2- 172.21.15.2:0/1727332720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d780ce000 0x7f8d78156f60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000cda0 tx=0x7f8d7000a570 comp rx=0 tx=0).stop 2023-08-25T03:33:59.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.418+0000 7f8d7c80c700 1 -- 172.21.15.2:0/1727332720 shutdown_connections 2023-08-25T03:33:59.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.418+0000 7f8d7c80c700 1 --2- 172.21.15.2:0/1727332720 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8d5806a0e0 0x7f8d5806c5a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:59.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.418+0000 7f8d7c80c700 1 --2- 172.21.15.2:0/1727332720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8d780ce000 0x7f8d78156f60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:59.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.418+0000 7f8d7c80c700 1 --2- 172.21.15.2:0/1727332720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8d780b1fb0 0x7f8d78151910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:59.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.418+0000 7f8d7c80c700 1 --2- 172.21.15.2:0/1727332720 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8d780cdb70 0x7f8d78156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:59.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.418+0000 7f8d7c80c700 1 -- 172.21.15.2:0/1727332720 >> 172.21.15.2:0/1727332720 conn(0x7f8d780cfea0 msgr2=0x7f8d780caf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:59.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.418+0000 7f8d7c80c700 1 -- 172.21.15.2:0/1727332720 shutdown_connections 2023-08-25T03:33:59.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.418+0000 7f8d7c80c700 1 -- 172.21.15.2:0/1727332720 wait complete. 2023-08-25T03:33:59.433 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T03:33:49.560643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T03:33:56.265184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T03:33:59.435 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T03:33:59.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.562+0000 7f047773b700 1 Processor -- start 2023-08-25T03:33:59.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.562+0000 7f047773b700 1 -- start start 2023-08-25T03:33:59.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.563+0000 7f047773b700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0470072370 0x7f04700ce760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:59.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.563+0000 7f047773b700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f04700b4590 0x7f04700b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:59.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.563+0000 7f047773b700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f047006dac0 con 0x7f0470159d10 2023-08-25T03:33:59.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.563+0000 7f047773b700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f047006e2e0 con 0x7f04700b4590 2023-08-25T03:33:59.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.563+0000 7f047773b700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f047006a1e0 con 0x7f0470072370 2023-08-25T03:33:59.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.563+0000 7f0475cd8700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f04700b4590 0x7f04700b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:59.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.563+0000 7f0475cd8700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f04700b4590 0x7f04700b49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59024/0 (socket says 172.21.15.2:59024) 2023-08-25T03:33:59.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.563+0000 7f0475cd8700 1 -- 172.21.15.2:0/2660453652 learned_addr learned my addr 172.21.15.2:0/2660453652 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:33:59.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.563+0000 7f0474cd6700 1 --2- 172.21.15.2:0/2660453652 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0470072370 0x7f04700ce760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:59.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.564+0000 7f0475cd8700 1 -- 172.21.15.2:0/2660453652 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0470072370 msgr2=0x7f04700ce760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:33:59.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.564+0000 7f0475cd8700 1 --2- 172.21.15.2:0/2660453652 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0470072370 0x7f04700ce760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:59.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.564+0000 7f0475cd8700 1 -- 172.21.15.2:0/2660453652 >> v1:172.21.15.31:6789/0 conn(0x7f0470159d10 legacy=0x7f04700ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:59.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.564+0000 7f0475cd8700 1 -- 172.21.15.2:0/2660453652 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f047006a4a0 con 0x7f04700b4590 2023-08-25T03:33:59.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.564+0000 7f0475cd8700 1 --2- 172.21.15.2:0/2660453652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04700b4590 0x7f04700b49c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0468009e50 tx=0x7f046800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b948624b9cd227 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:59.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.564+0000 7f0467fff700 1 -- 172.21.15.2:0/2660453652 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046800e070 con 0x7f04700b4590 2023-08-25T03:33:59.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.564+0000 7f0467fff700 1 -- 172.21.15.2:0/2660453652 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046800b910 con 0x7f04700b4590 2023-08-25T03:33:59.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.564+0000 7f0467fff700 1 -- 172.21.15.2:0/2660453652 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046800bcc0 con 0x7f04700b4590 2023-08-25T03:33:59.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.564+0000 7f047773b700 1 -- 172.21.15.2:0/2660453652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04700b4590 msgr2=0x7f04700b49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:59.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.564+0000 7f047773b700 1 --2- 172.21.15.2:0/2660453652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04700b4590 0x7f04700b49c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0468009e50 tx=0x7f046800b330 comp rx=0 tx=0).stop 2023-08-25T03:33:59.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.565+0000 7f047773b700 1 -- 172.21.15.2:0/2660453652 shutdown_connections 2023-08-25T03:33:59.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.565+0000 7f047773b700 1 --2- 172.21.15.2:0/2660453652 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04700b4590 0x7f04700b49c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:59.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.565+0000 7f047773b700 1 --2- 172.21.15.2:0/2660453652 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0470072370 0x7f04700ce760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:59.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.565+0000 7f047773b700 1 -- 172.21.15.2:0/2660453652 >> 172.21.15.2:0/2660453652 conn(0x7f04700cf500 msgr2=0x7f04700cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:59.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.565+0000 7f047773b700 1 -- 172.21.15.2:0/2660453652 shutdown_connections 2023-08-25T03:33:59.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.565+0000 7f047773b700 1 -- 172.21.15.2:0/2660453652 wait complete. 2023-08-25T03:33:59.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.566+0000 7f047773b700 1 Processor -- start 2023-08-25T03:33:59.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.566+0000 7f047773b700 1 -- start start 2023-08-25T03:33:59.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.567+0000 7f047773b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0470072370 0x7f04701f24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:59.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.567+0000 7f047773b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04700b4590 0x7f047015c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:59.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.567+0000 7f047773b700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0470159d10 0x7f047015c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:59.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.567+0000 7f047773b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04700b13b0 con 0x7f04700b4590 2023-08-25T03:33:59.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.567+0000 7f047773b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0470151760 con 0x7f0470072370 2023-08-25T03:33:59.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.567+0000 7f047773b700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0470151a40 con 0x7f0470159d10 2023-08-25T03:33:59.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.567+0000 7f0474cd6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04700b4590 0x7f047015c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:59.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.567+0000 7f0474cd6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04700b4590 0x7f047015c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59028/0 (socket says 172.21.15.2:59028) 2023-08-25T03:33:59.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.567+0000 7f0474cd6700 1 -- 172.21.15.2:0/1369228795 learned_addr learned my addr 172.21.15.2:0/1369228795 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:33:59.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.567+0000 7f04754d7700 1 --2- 172.21.15.2:0/1369228795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0470072370 0x7f04701f24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:59.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.567+0000 7f0475cd8700 1 --2- 172.21.15.2:0/1369228795 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0470159d10 0x7f047015c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:59.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.567+0000 7f0474cd6700 1 -- 172.21.15.2:0/1369228795 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0470159d10 msgr2=0x7f047015c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:59.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.567+0000 7f0474cd6700 1 --2- 172.21.15.2:0/1369228795 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0470159d10 0x7f047015c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:59.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.567+0000 7f0474cd6700 1 -- 172.21.15.2:0/1369228795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0470072370 msgr2=0x7f04701f24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:59.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.567+0000 7f0474cd6700 1 --2- 172.21.15.2:0/1369228795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0470072370 0x7f04701f24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:59.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.567+0000 7f0474cd6700 1 -- 172.21.15.2:0/1369228795 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0468009b30 con 0x7f04700b4590 2023-08-25T03:33:59.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.568+0000 7f0474cd6700 1 --2- 172.21.15.2:0/1369228795 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04700b4590 0x7f047015c120 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f046c00e870 tx=0x7f046c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:59.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.569+0000 7f0465ffb700 1 -- 172.21.15.2:0/1369228795 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c009590 con 0x7f04700b4590 2023-08-25T03:33:59.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.569+0000 7f047773b700 1 -- 172.21.15.2:0/1369228795 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0470151d20 con 0x7f04700b4590 2023-08-25T03:33:59.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.569+0000 7f047773b700 1 -- 172.21.15.2:0/1369228795 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04701522e0 con 0x7f04700b4590 2023-08-25T03:33:59.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.569+0000 7f0465ffb700 1 -- 172.21.15.2:0/1369228795 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f046c013680 con 0x7f04700b4590 2023-08-25T03:33:59.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.569+0000 7f0465ffb700 1 -- 172.21.15.2:0/1369228795 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f046c0139f0 con 0x7f04700b4590 2023-08-25T03:33:59.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.570+0000 7f047773b700 1 -- 172.21.15.2:0/1369228795 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0470152650 con 0x7f04700b4590 2023-08-25T03:33:59.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.570+0000 7f0465ffb700 1 -- 172.21.15.2:0/1369228795 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f046c013b50 con 0x7f04700b4590 2023-08-25T03:33:59.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.571+0000 7f0465ffb700 1 --2- 172.21.15.2:0/1369228795 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f045406a0e0 0x7f045406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:59.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.571+0000 7f0465ffb700 1 -- 172.21.15.2:0/1369228795 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f046c088420 con 0x7f04700b4590 2023-08-25T03:33:59.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.571+0000 7f04754d7700 1 --2- 172.21.15.2:0/1369228795 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f045406a0e0 0x7f045406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:59.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.571+0000 7f04754d7700 1 --2- 172.21.15.2:0/1369228795 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f045406a0e0 0x7f045406c5a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0460000fb0 tx=0x7f0460009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:59.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.575+0000 7f0465ffb700 1 -- 172.21.15.2:0/1369228795 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f046c0578f0 con 0x7f04700b4590 2023-08-25T03:33:59.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.773+0000 7f047773b700 1 -- 172.21.15.2:0/1369228795 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f04700746d0 con 0x7f04700b4590 2023-08-25T03:33:59.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.774+0000 7f0465ffb700 1 -- 172.21.15.2:0/1369228795 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f046c057480 con 0x7f04700b4590 2023-08-25T03:33:59.776 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-08-25T03:33:59.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.776+0000 7f047773b700 1 -- 172.21.15.2:0/1369228795 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f045406a0e0 msgr2=0x7f045406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:59.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.776+0000 7f047773b700 1 --2- 172.21.15.2:0/1369228795 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f045406a0e0 0x7f045406c5a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0460000fb0 tx=0x7f0460009040 comp rx=0 tx=0).stop 2023-08-25T03:33:59.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.776+0000 7f047773b700 1 -- 172.21.15.2:0/1369228795 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04700b4590 msgr2=0x7f047015c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:59.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.776+0000 7f047773b700 1 --2- 172.21.15.2:0/1369228795 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04700b4590 0x7f047015c120 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f046c00e870 tx=0x7f046c00eb80 comp rx=0 tx=0).stop 2023-08-25T03:33:59.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.776+0000 7f047773b700 1 -- 172.21.15.2:0/1369228795 shutdown_connections 2023-08-25T03:33:59.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.776+0000 7f047773b700 1 --2- 172.21.15.2:0/1369228795 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0470159d10 0x7f047015c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:59.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.776+0000 7f047773b700 1 --2- 172.21.15.2:0/1369228795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0470072370 0x7f04701f24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:59.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.776+0000 7f047773b700 1 --2- 172.21.15.2:0/1369228795 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f045406a0e0 0x7f045406c5a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:59.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.776+0000 7f047773b700 1 --2- 172.21.15.2:0/1369228795 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f04700b4590 0x7f047015c120 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:59.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.776+0000 7f047773b700 1 -- 172.21.15.2:0/1369228795 >> 172.21.15.2:0/1369228795 conn(0x7f04700cf500 msgr2=0x7f04700cd970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:59.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.776+0000 7f047773b700 1 -- 172.21.15.2:0/1369228795 shutdown_connections 2023-08-25T03:33:59.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.776+0000 7f047773b700 1 -- 172.21.15.2:0/1369228795 wait complete. 2023-08-25T03:33:59.790 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T03:33:59.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.915+0000 7f69761cc700 1 Processor -- start 2023-08-25T03:33:59.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.915+0000 7f69761cc700 1 -- start start 2023-08-25T03:33:59.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.916+0000 7f69761cc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f69700af180 0x7f6970070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:59.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.916+0000 7f69761cc700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6970160bd0 con 0x7f69701607f0 2023-08-25T03:33:59.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.916+0000 7f69761cc700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69700cdd00 con 0x7f697006a1e0 2023-08-25T03:33:59.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.916+0000 7f69761cc700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69700713a0 con 0x7f69700af180 2023-08-25T03:33:59.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.916+0000 7f696f7fe700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f69701607f0 0x7f69700ac8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45700/0 (socket says 172.21.15.2:45700) 2023-08-25T03:33:59.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.916+0000 7f696f7fe700 1 -- 172.21.15.2:0/1146827917 learned_addr learned my addr 172.21.15.2:0/1146827917 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:33:59.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.916+0000 7f696effd700 1 --2- 172.21.15.2:0/1146827917 >> v2:172.21.15.31:3300/0 conn(0x7f69700af180 0x7f6970070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:59.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.916+0000 7f696e7fc700 1 -- 172.21.15.2:0/1146827917 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3290567209 0 0) 0x7f6970160bd0 con 0x7f69701607f0 2023-08-25T03:33:59.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.916+0000 7f696e7fc700 1 -- 172.21.15.2:0/1146827917 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f695c0035b0 con 0x7f69701607f0 2023-08-25T03:33:59.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.916+0000 7f696effd700 1 -- 172.21.15.2:0/1146827917 >> v1:172.21.15.31:6789/0 conn(0x7f697006a1e0 legacy=0x7f697006ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:59.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.916+0000 7f696effd700 1 -- 172.21.15.2:0/1146827917 >> v1:172.21.15.2:6789/0 conn(0x7f69701607f0 legacy=0x7f69700ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:59.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.916+0000 7f696effd700 1 -- 172.21.15.2:0/1146827917 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69700cda00 con 0x7f69700af180 2023-08-25T03:33:59.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.917+0000 7f696effd700 1 --2- 172.21.15.2:0/1146827917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69700af180 0x7f6970070e60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6958009840 tx=0x7f6958009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1c91fd6cf72034c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:59.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.917+0000 7f696e7fc700 1 -- 172.21.15.2:0/1146827917 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6958011830 con 0x7f69700af180 2023-08-25T03:33:59.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.917+0000 7f696e7fc700 1 -- 172.21.15.2:0/1146827917 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6958011990 con 0x7f69700af180 2023-08-25T03:33:59.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.917+0000 7f696e7fc700 1 -- 172.21.15.2:0/1146827917 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6958011d20 con 0x7f69700af180 2023-08-25T03:33:59.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.917+0000 7f69761cc700 1 -- 172.21.15.2:0/1146827917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69700af180 msgr2=0x7f6970070e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:59.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.917+0000 7f69761cc700 1 --2- 172.21.15.2:0/1146827917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69700af180 0x7f6970070e60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6958009840 tx=0x7f6958009b50 comp rx=0 tx=0).stop 2023-08-25T03:33:59.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.917+0000 7f69761cc700 1 -- 172.21.15.2:0/1146827917 shutdown_connections 2023-08-25T03:33:59.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.917+0000 7f69761cc700 1 --2- 172.21.15.2:0/1146827917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69700af180 0x7f6970070e60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:59.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.917+0000 7f69761cc700 1 -- 172.21.15.2:0/1146827917 >> 172.21.15.2:0/1146827917 conn(0x7f69700cfd90 msgr2=0x7f69700d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:33:59.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.917+0000 7f69761cc700 1 -- 172.21.15.2:0/1146827917 shutdown_connections 2023-08-25T03:33:59.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.917+0000 7f69761cc700 1 -- 172.21.15.2:0/1146827917 wait complete. 2023-08-25T03:33:59.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.919+0000 7f69761cc700 1 Processor -- start 2023-08-25T03:33:59.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.919+0000 7f69761cc700 1 -- start start 2023-08-25T03:33:59.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.919+0000 7f69761cc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f697006a1e0 0x7f69700c40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:59.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.919+0000 7f69761cc700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f69700af180 0x7f69701f20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:59.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.919+0000 7f69761cc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69701607f0 0x7f69700c93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:59.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.919+0000 7f69761cc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69700c48e0 con 0x7f69701607f0 2023-08-25T03:33:59.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.919+0000 7f69761cc700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69700c4bc0 con 0x7f697006a1e0 2023-08-25T03:33:59.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.919+0000 7f69761cc700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69700c4ea0 con 0x7f69700af180 2023-08-25T03:33:59.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.920+0000 7f696ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69701607f0 0x7f69700c93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:59.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.920+0000 7f696ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69701607f0 0x7f69700c93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59044/0 (socket says 172.21.15.2:59044) 2023-08-25T03:33:59.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.920+0000 7f696ffff700 1 -- 172.21.15.2:0/2514683171 learned_addr learned my addr 172.21.15.2:0/2514683171 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:33:59.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.920+0000 7f696f7fe700 1 --2- 172.21.15.2:0/2514683171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f697006a1e0 0x7f69700c40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:59.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.920+0000 7f696effd700 1 --2- 172.21.15.2:0/2514683171 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f69700af180 0x7f69701f20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:59.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.920+0000 7f696ffff700 1 -- 172.21.15.2:0/2514683171 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f69700af180 msgr2=0x7f69701f20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:59.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.920+0000 7f696ffff700 1 --2- 172.21.15.2:0/2514683171 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f69700af180 0x7f69701f20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:59.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.920+0000 7f696ffff700 1 -- 172.21.15.2:0/2514683171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f697006a1e0 msgr2=0x7f69700c40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:33:59.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.920+0000 7f696ffff700 1 --2- 172.21.15.2:0/2514683171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f697006a1e0 0x7f69700c40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:33:59.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.920+0000 7f696ffff700 1 -- 172.21.15.2:0/2514683171 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69580094a0 con 0x7f69701607f0 2023-08-25T03:33:59.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.920+0000 7f696effd700 1 --2- 172.21.15.2:0/2514683171 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f69700af180 0x7f69701f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:33:59.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.920+0000 7f696f7fe700 1 --2- 172.21.15.2:0/2514683171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f697006a1e0 0x7f69700c40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:33:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.920+0000 7f696ffff700 1 --2- 172.21.15.2:0/2514683171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69701607f0 0x7f69700c93a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f696800cb90 tx=0x7f696800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.920+0000 7f6967fff700 1 -- 172.21.15.2:0/2514683171 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6968003c50 con 0x7f69701607f0 2023-08-25T03:33:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.920+0000 7f6967fff700 1 -- 172.21.15.2:0/2514683171 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6968003dd0 con 0x7f69701607f0 2023-08-25T03:33:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.920+0000 7f69761cc700 1 -- 172.21.15.2:0/2514683171 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69700c7000 con 0x7f69701607f0 2023-08-25T03:33:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.920+0000 7f6967fff700 1 -- 172.21.15.2:0/2514683171 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f696800abb0 con 0x7f69701607f0 2023-08-25T03:33:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.920+0000 7f69761cc700 1 -- 172.21.15.2:0/2514683171 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69701f2610 con 0x7f69701607f0 2023-08-25T03:33:59.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.921+0000 7f69761cc700 1 -- 172.21.15.2:0/2514683171 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69700c6e60 con 0x7f69701607f0 2023-08-25T03:33:59.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.922+0000 7f6967fff700 1 -- 172.21.15.2:0/2514683171 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f696800ad10 con 0x7f69701607f0 2023-08-25T03:33:59.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.922+0000 7f6967fff700 1 --2- 172.21.15.2:0/2514683171 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f695006a0e0 0x7f695006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:33:59.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.922+0000 7f6967fff700 1 -- 172.21.15.2:0/2514683171 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f69680882c0 con 0x7f69701607f0 2023-08-25T03:33:59.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.922+0000 7f696f7fe700 1 --2- 172.21.15.2:0/2514683171 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f695006a0e0 0x7f695006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:33:59.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.923+0000 7f696f7fe700 1 --2- 172.21.15.2:0/2514683171 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f695006a0e0 0x7f695006c5a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f69600011e0 tx=0x7f6960009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:33:59.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:33:59.926+0000 7f6967fff700 1 -- 172.21.15.2:0/2514683171 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f69700c6e60 con 0x7f69701607f0 2023-08-25T03:34:00.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.127+0000 7f69761cc700 1 -- 172.21.15.2:0/2514683171 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f69700c6e60 con 0x7f69701607f0 2023-08-25T03:34:00.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.127+0000 7f6967fff700 1 -- 172.21.15.2:0/2514683171 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v20) v1 ==== 92+0+10 (secure 0 0 0) 0x7f69700c6e60 con 0x7f69701607f0 2023-08-25T03:34:00.129 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 8 2023-08-25T03:34:00.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.129+0000 7f69761cc700 1 -- 172.21.15.2:0/2514683171 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f695006a0e0 msgr2=0x7f695006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:00.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.129+0000 7f69761cc700 1 --2- 172.21.15.2:0/2514683171 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f695006a0e0 0x7f695006c5a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f69600011e0 tx=0x7f6960009040 comp rx=0 tx=0).stop 2023-08-25T03:34:00.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.129+0000 7f69761cc700 1 -- 172.21.15.2:0/2514683171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69701607f0 msgr2=0x7f69700c93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:00.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.129+0000 7f69761cc700 1 --2- 172.21.15.2:0/2514683171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69701607f0 0x7f69700c93a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f696800cb90 tx=0x7f696800cf50 comp rx=0 tx=0).stop 2023-08-25T03:34:00.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.129+0000 7f69761cc700 1 -- 172.21.15.2:0/2514683171 shutdown_connections 2023-08-25T03:34:00.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.129+0000 7f69761cc700 1 --2- 172.21.15.2:0/2514683171 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f695006a0e0 0x7f695006c5a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:00.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.129+0000 7f69761cc700 1 --2- 172.21.15.2:0/2514683171 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f69701607f0 0x7f69700c93a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:00.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.129+0000 7f69761cc700 1 --2- 172.21.15.2:0/2514683171 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f697006a1e0 0x7f69700c40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:00.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.129+0000 7f69761cc700 1 --2- 172.21.15.2:0/2514683171 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f69700af180 0x7f69701f20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:00.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.129+0000 7f69761cc700 1 -- 172.21.15.2:0/2514683171 >> 172.21.15.2:0/2514683171 conn(0x7f69700cfd90 msgr2=0x7f69700d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:00.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.129+0000 7f69761cc700 1 -- 172.21.15.2:0/2514683171 shutdown_connections 2023-08-25T03:34:00.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.129+0000 7f69761cc700 1 -- 172.21.15.2:0/2514683171 wait complete. 2023-08-25T03:34:00.143 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-25T03:34:00.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.265+0000 7fe9bd0a2700 1 Processor -- start 2023-08-25T03:34:00.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.265+0000 7fe9bd0a2700 1 -- start start 2023-08-25T03:34:00.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.266+0000 7fe9bd0a2700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe9b8158f60 0x7fe9b80ca950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:00.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.266+0000 7fe9bd0a2700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9b80ccc00 0x7fe9b80cd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:00.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.266+0000 7fe9bd0a2700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9b8071f70 con 0x7fe9b8158b80 2023-08-25T03:34:00.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.266+0000 7fe9bd0a2700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9b80cd5b0 con 0x7fe9b8158f60 2023-08-25T03:34:00.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.266+0000 7fe9bd0a2700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b806d140 con 0x7fe9b80ccc00 2023-08-25T03:34:00.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.266+0000 7fe9b659c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe9b8158f60 0x7fe9b80ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:00.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.266+0000 7fe9b6d9d700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fe9b8158b80 0x7fe9b8071760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45714/0 (socket says 172.21.15.2:45714) 2023-08-25T03:34:00.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.266+0000 7fe9b659c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe9b8158f60 0x7fe9b80ca950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59050/0 (socket says 172.21.15.2:59050) 2023-08-25T03:34:00.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.266+0000 7fe9b6d9d700 1 -- 172.21.15.2:0/1069165136 learned_addr learned my addr 172.21.15.2:0/1069165136 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:00.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.266+0000 7fe9b759e700 1 --2- 172.21.15.2:0/1069165136 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9b80ccc00 0x7fe9b80cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:00.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.266+0000 7fe9b659c700 1 -- 172.21.15.2:0/1069165136 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9b80ccc00 msgr2=0x7fe9b80cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:00.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.266+0000 7fe9b659c700 1 --2- 172.21.15.2:0/1069165136 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9b80ccc00 0x7fe9b80cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:00.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.267+0000 7fe9b659c700 1 -- 172.21.15.2:0/1069165136 >> v1:172.21.15.2:6789/0 conn(0x7fe9b8158b80 legacy=0x7fe9b8071760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:00.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.267+0000 7fe9b659c700 1 -- 172.21.15.2:0/1069165136 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b806d420 con 0x7fe9b8158f60 2023-08-25T03:34:00.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.267+0000 7fe9b5d9b700 1 -- 172.21.15.2:0/1069165136 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3532358575 0 0) 0x7fe9b8071f70 con 0x7fe9b8158b80 2023-08-25T03:34:00.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.267+0000 7fe9b659c700 1 --2- 172.21.15.2:0/1069165136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9b8158f60 0x7fe9b80ca950 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac009840 tx=0x7fe9ac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989d167effede5cc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:00.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.267+0000 7fe9b5d9b700 1 -- 172.21.15.2:0/1069165136 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac011830 con 0x7fe9b8158f60 2023-08-25T03:34:00.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.267+0000 7fe9b5d9b700 1 -- 172.21.15.2:0/1069165136 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9ac011990 con 0x7fe9b8158f60 2023-08-25T03:34:00.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.267+0000 7fe9b5d9b700 1 -- 172.21.15.2:0/1069165136 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9ac011d20 con 0x7fe9b8158f60 2023-08-25T03:34:00.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.267+0000 7fe9bd0a2700 1 -- 172.21.15.2:0/1069165136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9b8158f60 msgr2=0x7fe9b80ca950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:00.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.267+0000 7fe9bd0a2700 1 --2- 172.21.15.2:0/1069165136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9b8158f60 0x7fe9b80ca950 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac009840 tx=0x7fe9ac009b50 comp rx=0 tx=0).stop 2023-08-25T03:34:00.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.267+0000 7fe9bd0a2700 1 -- 172.21.15.2:0/1069165136 shutdown_connections 2023-08-25T03:34:00.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.267+0000 7fe9bd0a2700 1 --2- 172.21.15.2:0/1069165136 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9b80ccc00 0x7fe9b80cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:00.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.267+0000 7fe9bd0a2700 1 --2- 172.21.15.2:0/1069165136 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9b8158f60 0x7fe9b80ca950 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:00.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.267+0000 7fe9bd0a2700 1 -- 172.21.15.2:0/1069165136 >> 172.21.15.2:0/1069165136 conn(0x7fe9b80b1cf0 msgr2=0x7fe9b80b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:00.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.267+0000 7fe9bd0a2700 1 -- 172.21.15.2:0/1069165136 shutdown_connections 2023-08-25T03:34:00.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.267+0000 7fe9bd0a2700 1 -- 172.21.15.2:0/1069165136 wait complete. 2023-08-25T03:34:00.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.269+0000 7fe9bd0a2700 1 Processor -- start 2023-08-25T03:34:00.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.269+0000 7fe9bd0a2700 1 -- start start 2023-08-25T03:34:00.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.270+0000 7fe9bd0a2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b80ccc00 0x7fe9b81f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:00.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.270+0000 7fe9bd0a2700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9b8158b80 0x7fe9b8156080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:00.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.270+0000 7fe9bd0a2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9b8158f60 0x7fe9b806a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:00.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.270+0000 7fe9bd0a2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b80c3a70 con 0x7fe9b8158f60 2023-08-25T03:34:00.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.270+0000 7fe9bd0a2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b80c3d10 con 0x7fe9b80ccc00 2023-08-25T03:34:00.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.270+0000 7fe9bd0a2700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b81f4130 con 0x7fe9b8158b80 2023-08-25T03:34:00.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.270+0000 7fe9b759e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9b8158f60 0x7fe9b806a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:00.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.270+0000 7fe9b759e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9b8158f60 0x7fe9b806a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59058/0 (socket says 172.21.15.2:59058) 2023-08-25T03:34:00.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.270+0000 7fe9b759e700 1 -- 172.21.15.2:0/3292832434 learned_addr learned my addr 172.21.15.2:0/3292832434 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:00.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.270+0000 7fe9b659c700 1 --2- 172.21.15.2:0/3292832434 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9b8158b80 0x7fe9b8156080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:00.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.270+0000 7fe9b6d9d700 1 --2- 172.21.15.2:0/3292832434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b80ccc00 0x7fe9b81f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:00.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.270+0000 7fe9b759e700 1 -- 172.21.15.2:0/3292832434 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9b8158b80 msgr2=0x7fe9b8156080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:00.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.270+0000 7fe9b759e700 1 --2- 172.21.15.2:0/3292832434 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9b8158b80 0x7fe9b8156080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:00.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.270+0000 7fe9b759e700 1 -- 172.21.15.2:0/3292832434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b80ccc00 msgr2=0x7fe9b81f3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:00.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.270+0000 7fe9b759e700 1 --2- 172.21.15.2:0/3292832434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b80ccc00 0x7fe9b81f3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:00.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.270+0000 7fe9b759e700 1 -- 172.21.15.2:0/3292832434 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ac0094a0 con 0x7fe9b8158f60 2023-08-25T03:34:00.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.270+0000 7fe9b6d9d700 1 --2- 172.21.15.2:0/3292832434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b80ccc00 0x7fe9b81f3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:00.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.270+0000 7fe9b759e700 1 --2- 172.21.15.2:0/3292832434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9b8158f60 0x7fe9b806a1e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800d600 tx=0x7fe9a800d910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:00.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.270+0000 7fe99f7fe700 1 -- 172.21.15.2:0/3292832434 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a80125f0 con 0x7fe9b8158f60 2023-08-25T03:34:00.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.271+0000 7fe99f7fe700 1 -- 172.21.15.2:0/3292832434 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a8012750 con 0x7fe9b8158f60 2023-08-25T03:34:00.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.271+0000 7fe9bd0a2700 1 -- 172.21.15.2:0/3292832434 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b81f43d0 con 0x7fe9b8158f60 2023-08-25T03:34:00.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.271+0000 7fe99f7fe700 1 -- 172.21.15.2:0/3292832434 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a8012b00 con 0x7fe9b8158f60 2023-08-25T03:34:00.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.271+0000 7fe9bd0a2700 1 -- 172.21.15.2:0/3292832434 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b81f4990 con 0x7fe9b8158f60 2023-08-25T03:34:00.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.272+0000 7fe9bd0a2700 1 -- 172.21.15.2:0/3292832434 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b8066f70 con 0x7fe9b8158f60 2023-08-25T03:34:00.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.272+0000 7fe99f7fe700 1 -- 172.21.15.2:0/3292832434 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe9a8012c60 con 0x7fe9b8158f60 2023-08-25T03:34:00.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.272+0000 7fe99f7fe700 1 --2- 172.21.15.2:0/3292832434 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe9a406a0e0 0x7fe9a406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:00.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.272+0000 7fe99f7fe700 1 -- 172.21.15.2:0/3292832434 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fe9a8088290 con 0x7fe9b8158f60 2023-08-25T03:34:00.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.272+0000 7fe9b6d9d700 1 --2- 172.21.15.2:0/3292832434 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe9a406a0e0 0x7fe9a406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:00.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.273+0000 7fe9b6d9d700 1 --2- 172.21.15.2:0/3292832434 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe9a406a0e0 0x7fe9a406c5a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00012e0 tx=0x7fe9b0001a30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:00.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.277+0000 7fe99f7fe700 1 -- 172.21.15.2:0/3292832434 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe9a8053d60 con 0x7fe9b8158f60 2023-08-25T03:34:00.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.480+0000 7fe9bd0a2700 1 -- 172.21.15.2:0/3292832434 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe9b815b7c0 con 0x7fe9b8158f60 2023-08-25T03:34:00.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.480+0000 7fe99f7fe700 1 -- 172.21.15.2:0/3292832434 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v20) v1 ==== 84+0+126 (secure 0 0 0) 0x7fe9a8057380 con 0x7fe9b8158f60 2023-08-25T03:34:00.482 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:00.482 INFO:teuthology.orchestra.run.smithi002.stdout:[ 2023-08-25T03:34:00.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:34:00.483 INFO:teuthology.orchestra.run.smithi002.stdout: "poolnum": 1, 2023-08-25T03:34:00.483 INFO:teuthology.orchestra.run.smithi002.stdout: "poolname": ".mgr" 2023-08-25T03:34:00.483 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-08-25T03:34:00.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:34:00.483 INFO:teuthology.orchestra.run.smithi002.stdout: "poolnum": 2, 2023-08-25T03:34:00.483 INFO:teuthology.orchestra.run.smithi002.stdout: "poolname": "rbd" 2023-08-25T03:34:00.484 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:34:00.484 INFO:teuthology.orchestra.run.smithi002.stdout:] 2023-08-25T03:34:00.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.482+0000 7fe9bd0a2700 1 -- 172.21.15.2:0/3292832434 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe9a406a0e0 msgr2=0x7fe9a406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:00.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.482+0000 7fe9bd0a2700 1 --2- 172.21.15.2:0/3292832434 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe9a406a0e0 0x7fe9a406c5a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe9b00012e0 tx=0x7fe9b0001a30 comp rx=0 tx=0).stop 2023-08-25T03:34:00.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.482+0000 7fe9bd0a2700 1 -- 172.21.15.2:0/3292832434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9b8158f60 msgr2=0x7fe9b806a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:00.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.482+0000 7fe9bd0a2700 1 --2- 172.21.15.2:0/3292832434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9b8158f60 0x7fe9b806a1e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe9a800d600 tx=0x7fe9a800d910 comp rx=0 tx=0).stop 2023-08-25T03:34:00.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.482+0000 7fe9bd0a2700 1 -- 172.21.15.2:0/3292832434 shutdown_connections 2023-08-25T03:34:00.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.482+0000 7fe9bd0a2700 1 --2- 172.21.15.2:0/3292832434 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe9a406a0e0 0x7fe9a406c5a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:00.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.482+0000 7fe9bd0a2700 1 --2- 172.21.15.2:0/3292832434 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9b8158f60 0x7fe9b806a1e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:00.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.482+0000 7fe9bd0a2700 1 --2- 172.21.15.2:0/3292832434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9b80ccc00 0x7fe9b81f3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:00.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.482+0000 7fe9bd0a2700 1 --2- 172.21.15.2:0/3292832434 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9b8158b80 0x7fe9b8156080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:00.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.482+0000 7fe9bd0a2700 1 -- 172.21.15.2:0/3292832434 >> 172.21.15.2:0/3292832434 conn(0x7fe9b80b1cf0 msgr2=0x7fe9b8158370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:00.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.483+0000 7fe9bd0a2700 1 -- 172.21.15.2:0/3292832434 shutdown_connections 2023-08-25T03:34:00.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.483+0000 7fe9bd0a2700 1 -- 172.21.15.2:0/3292832434 wait complete. 2023-08-25T03:34:00.496 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-25T03:34:00.496 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-25T03:34:00.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.625+0000 7f9e4197f700 1 Processor -- start 2023-08-25T03:34:00.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.625+0000 7f9e4197f700 1 -- start start 2023-08-25T03:34:00.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.626+0000 7f9e4197f700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9e3c15b2b0 0x7f9e3c0cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:00.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.626+0000 7f9e4197f700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e3c0b3ef0 0x7f9e3c06e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:00.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.626+0000 7f9e4197f700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e3c0cda60 con 0x7f9e3c15b690 2023-08-25T03:34:00.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.626+0000 7f9e4197f700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e3c0cdd40 con 0x7f9e3c15b2b0 2023-08-25T03:34:00.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.626+0000 7f9e4197f700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c0ce020 con 0x7f9e3c0b3ef0 2023-08-25T03:34:00.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.626+0000 7f9e3a7fc700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e3c0b3ef0 0x7f9e3c06e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:00.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.626+0000 7f9e3affd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9e3c15b2b0 0x7f9e3c0cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:00.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.626+0000 7f9e3a7fc700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e3c0b3ef0 0x7f9e3c06e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.2:59306/0 (socket says 172.21.15.2:59306) 2023-08-25T03:34:00.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.626+0000 7f9e3a7fc700 1 -- 172.21.15.2:0/2363361026 learned_addr learned my addr 172.21.15.2:0/2363361026 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:00.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.627+0000 7f9e3a7fc700 1 -- 172.21.15.2:0/2363361026 >> v2:172.21.15.31:3300/0 conn(0x7f9e3c15b2b0 msgr2=0x7f9e3c0cd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:00.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.627+0000 7f9e3a7fc700 1 --2- 172.21.15.2:0/2363361026 >> v2:172.21.15.31:3300/0 conn(0x7f9e3c15b2b0 0x7f9e3c0cd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:00.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.627+0000 7f9e3a7fc700 1 -- 172.21.15.2:0/2363361026 >> v1:172.21.15.31:6789/0 conn(0x7f9e3c15b690 legacy=0x7f9e3c0b2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:00.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.627+0000 7f9e3a7fc700 1 -- 172.21.15.2:0/2363361026 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c0ce300 con 0x7f9e3c0b3ef0 2023-08-25T03:34:00.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.627+0000 7f9e39ffb700 1 -- 172.21.15.2:0/2363361026 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1825412996 0 0) 0x7f9e3c0cda60 con 0x7f9e3c15b690 2023-08-25T03:34:00.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.627+0000 7f9e3a7fc700 1 --2- 172.21.15.2:0/2363361026 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e3c0b3ef0 0x7f9e3c06e750 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c0098f0 tx=0x7f9e2c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=984e5aef9f318126 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:00.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.627+0000 7f9e39ffb700 1 -- 172.21.15.2:0/2363361026 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c011830 con 0x7f9e3c0b3ef0 2023-08-25T03:34:00.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.627+0000 7f9e39ffb700 1 -- 172.21.15.2:0/2363361026 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2c011990 con 0x7f9e3c0b3ef0 2023-08-25T03:34:00.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.627+0000 7f9e39ffb700 1 -- 172.21.15.2:0/2363361026 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c011d20 con 0x7f9e3c0b3ef0 2023-08-25T03:34:00.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.628+0000 7f9e4197f700 1 -- 172.21.15.2:0/2363361026 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e3c0b3ef0 msgr2=0x7f9e3c06e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:00.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.628+0000 7f9e4197f700 1 --2- 172.21.15.2:0/2363361026 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e3c0b3ef0 0x7f9e3c06e750 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c0098f0 tx=0x7f9e2c009c00 comp rx=0 tx=0).stop 2023-08-25T03:34:00.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.628+0000 7f9e4197f700 1 -- 172.21.15.2:0/2363361026 shutdown_connections 2023-08-25T03:34:00.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.628+0000 7f9e4197f700 1 --2- 172.21.15.2:0/2363361026 >> v2:172.21.15.31:3300/0 conn(0x7f9e3c15b2b0 0x7f9e3c0cd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:00.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.628+0000 7f9e4197f700 1 --2- 172.21.15.2:0/2363361026 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e3c0b3ef0 0x7f9e3c06e750 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:00.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.628+0000 7f9e4197f700 1 -- 172.21.15.2:0/2363361026 >> 172.21.15.2:0/2363361026 conn(0x7f9e3c065fe0 msgr2=0x7f9e3c0663c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:00.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.628+0000 7f9e4197f700 1 -- 172.21.15.2:0/2363361026 shutdown_connections 2023-08-25T03:34:00.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.628+0000 7f9e4197f700 1 -- 172.21.15.2:0/2363361026 wait complete. 2023-08-25T03:34:00.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.629+0000 7f9e4197f700 1 Processor -- start 2023-08-25T03:34:00.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.629+0000 7f9e4197f700 1 -- start start 2023-08-25T03:34:00.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.630+0000 7f9e4197f700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e3c0b3ef0 0x7f9e3c06b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:00.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.630+0000 7f9e4197f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e3c15b2b0 0x7f9e3c15c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:00.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.630+0000 7f9e4197f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e3c15b690 0x7f9e3c0ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:00.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.630+0000 7f9e4197f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c159750 con 0x7f9e3c15b2b0 2023-08-25T03:34:00.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.630+0000 7f9e4197f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c0c92f0 con 0x7f9e3c15b690 2023-08-25T03:34:00.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.630+0000 7f9e4197f700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c0c95d0 con 0x7f9e3c0b3ef0 2023-08-25T03:34:00.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.630+0000 7f9e3a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e3c15b2b0 0x7f9e3c15c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:00.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.630+0000 7f9e3a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e3c15b2b0 0x7f9e3c15c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59070/0 (socket says 172.21.15.2:59070) 2023-08-25T03:34:00.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.630+0000 7f9e3a7fc700 1 -- 172.21.15.2:0/2581824578 learned_addr learned my addr 172.21.15.2:0/2581824578 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:00.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.630+0000 7f9e3affd700 1 --2- 172.21.15.2:0/2581824578 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e3c0b3ef0 0x7f9e3c06b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:00.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.630+0000 7f9e3b7fe700 1 --2- 172.21.15.2:0/2581824578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e3c15b690 0x7f9e3c0ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:00.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.630+0000 7f9e3a7fc700 1 -- 172.21.15.2:0/2581824578 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e3c0b3ef0 msgr2=0x7f9e3c06b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:00.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.630+0000 7f9e3a7fc700 1 --2- 172.21.15.2:0/2581824578 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e3c0b3ef0 0x7f9e3c06b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:00.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.630+0000 7f9e3a7fc700 1 -- 172.21.15.2:0/2581824578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e3c15b690 msgr2=0x7f9e3c0ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:00.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.630+0000 7f9e3a7fc700 1 --2- 172.21.15.2:0/2581824578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e3c15b690 0x7f9e3c0ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:00.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.630+0000 7f9e3a7fc700 1 -- 172.21.15.2:0/2581824578 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e2c009550 con 0x7f9e3c15b2b0 2023-08-25T03:34:00.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.630+0000 7f9e3affd700 1 --2- 172.21.15.2:0/2581824578 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e3c0b3ef0 0x7f9e3c06b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:00.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.630+0000 7f9e3b7fe700 1 --2- 172.21.15.2:0/2581824578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e3c15b690 0x7f9e3c0ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:00.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.630+0000 7f9e3a7fc700 1 --2- 172.21.15.2:0/2581824578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e3c15b2b0 0x7f9e3c15c9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c009c40 tx=0x7f9e2c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:00.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.631+0000 7f9e23fff700 1 -- 172.21.15.2:0/2581824578 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c004480 con 0x7f9e3c15b2b0 2023-08-25T03:34:00.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.631+0000 7f9e23fff700 1 -- 172.21.15.2:0/2581824578 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2c01f070 con 0x7f9e3c15b2b0 2023-08-25T03:34:00.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.631+0000 7f9e4197f700 1 -- 172.21.15.2:0/2581824578 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e3c0c98b0 con 0x7f9e3c15b2b0 2023-08-25T03:34:00.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.631+0000 7f9e23fff700 1 -- 172.21.15.2:0/2581824578 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2c0038c0 con 0x7f9e3c15b2b0 2023-08-25T03:34:00.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.631+0000 7f9e4197f700 1 -- 172.21.15.2:0/2581824578 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3c159b90 con 0x7f9e3c15b2b0 2023-08-25T03:34:00.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.632+0000 7f9e4197f700 1 -- 172.21.15.2:0/2581824578 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e3c0c9dd0 con 0x7f9e3c15b2b0 2023-08-25T03:34:00.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.632+0000 7f9e23fff700 1 -- 172.21.15.2:0/2581824578 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9e2c021030 con 0x7f9e3c15b2b0 2023-08-25T03:34:00.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.632+0000 7f9e23fff700 1 --2- 172.21.15.2:0/2581824578 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9e2806a0e0 0x7f9e2806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:00.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.632+0000 7f9e23fff700 1 -- 172.21.15.2:0/2581824578 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9e2c089670 con 0x7f9e3c15b2b0 2023-08-25T03:34:00.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.632+0000 7f9e3affd700 1 --2- 172.21.15.2:0/2581824578 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9e2806a0e0 0x7f9e2806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:00.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.633+0000 7f9e3affd700 1 --2- 172.21.15.2:0/2581824578 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9e2806a0e0 0x7f9e2806c5a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9e30009c30 tx=0x7f9e30008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:00.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.637+0000 7f9e23fff700 1 -- 172.21.15.2:0/2581824578 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9e3c0c9dd0 con 0x7f9e3c15b2b0 2023-08-25T03:34:00.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:00.837+0000 7f9e4197f700 1 -- 172.21.15.2:0/2581824578 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9e3c0c9dd0 con 0x7f9e3c15b2b0 2023-08-25T03:34:01.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.579+0000 7f9e23fff700 1 -- 172.21.15.2:0/2581824578 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v21) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9e3c0c9dd0 con 0x7f9e3c15b2b0 2023-08-25T03:34:01.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.585+0000 7f9e4197f700 1 -- 172.21.15.2:0/2581824578 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9e2806a0e0 msgr2=0x7f9e2806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:01.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.585+0000 7f9e4197f700 1 --2- 172.21.15.2:0/2581824578 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9e2806a0e0 0x7f9e2806c5a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9e30009c30 tx=0x7f9e30008040 comp rx=0 tx=0).stop 2023-08-25T03:34:01.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.585+0000 7f9e4197f700 1 -- 172.21.15.2:0/2581824578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e3c15b2b0 msgr2=0x7f9e3c15c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:01.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.585+0000 7f9e4197f700 1 --2- 172.21.15.2:0/2581824578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e3c15b2b0 0x7f9e3c15c9a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9e2c009c40 tx=0x7f9e2c004750 comp rx=0 tx=0).stop 2023-08-25T03:34:01.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.585+0000 7f9e4197f700 1 -- 172.21.15.2:0/2581824578 shutdown_connections 2023-08-25T03:34:01.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.586+0000 7f9e4197f700 1 --2- 172.21.15.2:0/2581824578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e3c15b690 0x7f9e3c0ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:01.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.586+0000 7f9e4197f700 1 --2- 172.21.15.2:0/2581824578 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e3c0b3ef0 0x7f9e3c06b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:01.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.586+0000 7f9e4197f700 1 --2- 172.21.15.2:0/2581824578 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9e2806a0e0 0x7f9e2806c5a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:01.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.586+0000 7f9e4197f700 1 --2- 172.21.15.2:0/2581824578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e3c15b2b0 0x7f9e3c15c9a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:01.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.586+0000 7f9e4197f700 1 -- 172.21.15.2:0/2581824578 >> 172.21.15.2:0/2581824578 conn(0x7f9e3c065fe0 msgr2=0x7f9e3c15a250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:01.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.586+0000 7f9e4197f700 1 -- 172.21.15.2:0/2581824578 shutdown_connections 2023-08-25T03:34:01.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.586+0000 7f9e4197f700 1 -- 172.21.15.2:0/2581824578 wait complete. 2023-08-25T03:34:01.590 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_metadata' created 2023-08-25T03:34:01.604 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-25T03:34:01.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.744+0000 7f5f072de700 1 Processor -- start 2023-08-25T03:34:01.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.744+0000 7f5f072de700 1 -- start start 2023-08-25T03:34:01.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.744+0000 7f5f072de700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f0015a180 0x7f5f0006cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:01.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.744+0000 7f5f072de700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5f0015e6e0 0x7f5f00071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:01.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.744+0000 7f5f072de700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f000719a0 con 0x7f5f00071540 2023-08-25T03:34:01.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.744+0000 7f5f072de700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f000ce290 con 0x7f5f0015e6e0 2023-08-25T03:34:01.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.744+0000 7f5f072de700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000cdfb0 con 0x7f5f0015a180 2023-08-25T03:34:01.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.744+0000 7f5f04879700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5f0015e6e0 0x7f5f00071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:01.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.744+0000 7f5f04879700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5f0015e6e0 0x7f5f00071000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59080/0 (socket says 172.21.15.2:59080) 2023-08-25T03:34:01.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.744+0000 7f5f04879700 1 -- 172.21.15.2:0/607857995 learned_addr learned my addr 172.21.15.2:0/607857995 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:01.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.744+0000 7f5f0587b700 1 --1- 172.21.15.2:0/607857995 >> v1:172.21.15.2:6789/0 conn(0x7f5f00071540 0x7f5f000cbb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45744/0 (socket says 172.21.15.2:45744) 2023-08-25T03:34:01.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.744+0000 7f5f0507a700 1 --2- 172.21.15.2:0/607857995 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f0015a180 0x7f5f0006cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:01.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.744+0000 7f5f04879700 1 -- 172.21.15.2:0/607857995 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f0015a180 msgr2=0x7f5f0006cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:01.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.744+0000 7f5f04879700 1 --2- 172.21.15.2:0/607857995 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f0015a180 0x7f5f0006cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:01.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.744+0000 7f5f04879700 1 -- 172.21.15.2:0/607857995 >> v1:172.21.15.2:6789/0 conn(0x7f5f00071540 legacy=0x7f5f000cbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:01.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.744+0000 7f5f04879700 1 -- 172.21.15.2:0/607857995 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f000cb090 con 0x7f5f0015e6e0 2023-08-25T03:34:01.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.744+0000 7f5ef7fff700 1 -- 172.21.15.2:0/607857995 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1476495493 0 0) 0x7f5f000719a0 con 0x7f5f00071540 2023-08-25T03:34:01.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.744+0000 7f5f0507a700 1 --2- 172.21.15.2:0/607857995 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f0015a180 0x7f5f0006cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:01.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.744+0000 7f5f04879700 1 --2- 172.21.15.2:0/607857995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f0015e6e0 0x7f5f00071000 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000bb90 tx=0x7f5ef000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9ebb9a33e5cf6a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:01.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.745+0000 7f5ef7fff700 1 -- 172.21.15.2:0/607857995 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef000dd50 con 0x7f5f0015e6e0 2023-08-25T03:34:01.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.745+0000 7f5ef7fff700 1 -- 172.21.15.2:0/607857995 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef0010470 con 0x7f5f0015e6e0 2023-08-25T03:34:01.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.745+0000 7f5ef7fff700 1 -- 172.21.15.2:0/607857995 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0010800 con 0x7f5f0015e6e0 2023-08-25T03:34:01.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.745+0000 7f5f072de700 1 -- 172.21.15.2:0/607857995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f0015e6e0 msgr2=0x7f5f00071000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:01.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.745+0000 7f5f072de700 1 --2- 172.21.15.2:0/607857995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f0015e6e0 0x7f5f00071000 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5ef000bb90 tx=0x7f5ef000bea0 comp rx=0 tx=0).stop 2023-08-25T03:34:01.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.745+0000 7f5f072de700 1 -- 172.21.15.2:0/607857995 shutdown_connections 2023-08-25T03:34:01.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.745+0000 7f5f072de700 1 --2- 172.21.15.2:0/607857995 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f0015a180 0x7f5f0006cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:01.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.745+0000 7f5f072de700 1 --2- 172.21.15.2:0/607857995 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f0015e6e0 0x7f5f00071000 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:01.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.745+0000 7f5f072de700 1 -- 172.21.15.2:0/607857995 >> 172.21.15.2:0/607857995 conn(0x7f5f000d0000 msgr2=0x7f5f000d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:01.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.745+0000 7f5f072de700 1 -- 172.21.15.2:0/607857995 shutdown_connections 2023-08-25T03:34:01.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.745+0000 7f5f072de700 1 -- 172.21.15.2:0/607857995 wait complete. 2023-08-25T03:34:01.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.746+0000 7f5f072de700 1 Processor -- start 2023-08-25T03:34:01.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.746+0000 7f5f072de700 1 -- start start 2023-08-25T03:34:01.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.747+0000 7f5f072de700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f00071540 0x7f5f000c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:01.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.747+0000 7f5f072de700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f0015a180 0x7f5f000cc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:01.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.747+0000 7f5f072de700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f0015e6e0 0x7f5f000ccc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:01.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.747+0000 7f5f072de700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000c8210 con 0x7f5f00071540 2023-08-25T03:34:01.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.747+0000 7f5f072de700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000c7690 con 0x7f5f0015e6e0 2023-08-25T03:34:01.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.747+0000 7f5f072de700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f000c7970 con 0x7f5f0015a180 2023-08-25T03:34:01.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.747+0000 7f5f0507a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f00071540 0x7f5f000c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:01.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.747+0000 7f5f0507a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f00071540 0x7f5f000c63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59082/0 (socket says 172.21.15.2:59082) 2023-08-25T03:34:01.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.747+0000 7f5f0507a700 1 -- 172.21.15.2:0/4067303824 learned_addr learned my addr 172.21.15.2:0/4067303824 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:01.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.747+0000 7f5f0587b700 1 --2- 172.21.15.2:0/4067303824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f0015e6e0 0x7f5f000ccc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:01.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.747+0000 7f5f04879700 1 --2- 172.21.15.2:0/4067303824 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f0015a180 0x7f5f000cc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:01.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.747+0000 7f5f0507a700 1 -- 172.21.15.2:0/4067303824 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f0015a180 msgr2=0x7f5f000cc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:01.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.748+0000 7f5f0507a700 1 --2- 172.21.15.2:0/4067303824 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f0015a180 0x7f5f000cc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:01.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.748+0000 7f5f0507a700 1 -- 172.21.15.2:0/4067303824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f0015e6e0 msgr2=0x7f5f000ccc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:01.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.748+0000 7f5f0507a700 1 --2- 172.21.15.2:0/4067303824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f0015e6e0 0x7f5f000ccc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:01.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.748+0000 7f5f0507a700 1 -- 172.21.15.2:0/4067303824 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef000b7f0 con 0x7f5f00071540 2023-08-25T03:34:01.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.748+0000 7f5f04879700 1 --2- 172.21.15.2:0/4067303824 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f0015a180 0x7f5f000cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:01.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.748+0000 7f5f0507a700 1 --2- 172.21.15.2:0/4067303824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f00071540 0x7f5f000c63e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5efc00c750 tx=0x7f5efc00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:01.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.748+0000 7f5ef5ffb700 1 -- 172.21.15.2:0/4067303824 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc011800 con 0x7f5f00071540 2023-08-25T03:34:01.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.748+0000 7f5ef5ffb700 1 -- 172.21.15.2:0/4067303824 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5efc011960 con 0x7f5f00071540 2023-08-25T03:34:01.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.748+0000 7f5f072de700 1 -- 172.21.15.2:0/4067303824 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f000c7c50 con 0x7f5f00071540 2023-08-25T03:34:01.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.748+0000 7f5ef5ffb700 1 -- 172.21.15.2:0/4067303824 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5efc011cd0 con 0x7f5f00071540 2023-08-25T03:34:01.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.748+0000 7f5f072de700 1 -- 172.21.15.2:0/4067303824 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f000c6920 con 0x7f5f00071540 2023-08-25T03:34:01.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.749+0000 7f5f072de700 1 -- 172.21.15.2:0/4067303824 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f00066f70 con 0x7f5f00071540 2023-08-25T03:34:01.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.749+0000 7f5ef5ffb700 1 -- 172.21.15.2:0/4067303824 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5efc021350 con 0x7f5f00071540 2023-08-25T03:34:01.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.750+0000 7f5ef5ffb700 1 --2- 172.21.15.2:0/4067303824 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5ee406a0e0 0x7f5ee406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:01.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.750+0000 7f5ef5ffb700 1 -- 172.21.15.2:0/4067303824 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f5efc0883f0 con 0x7f5f00071540 2023-08-25T03:34:01.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.750+0000 7f5f04879700 1 --2- 172.21.15.2:0/4067303824 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5ee406a0e0 0x7f5ee406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:01.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.750+0000 7f5f04879700 1 --2- 172.21.15.2:0/4067303824 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5ee406a0e0 0x7f5ee406c5a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5f000cd2a0 tx=0x7f5ef0013000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:01.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.755+0000 7f5ef5ffb700 1 -- 172.21.15.2:0/4067303824 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5efc053c30 con 0x7f5f00071540 2023-08-25T03:34:01.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:01.955+0000 7f5f072de700 1 -- 172.21.15.2:0/4067303824 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5f0009e190 con 0x7f5f00071540 2023-08-25T03:34:02.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.581+0000 7f5ef5ffb700 1 -- 172.21.15.2:0/4067303824 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v22) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5efc057250 con 0x7f5f00071540 2023-08-25T03:34:02.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.587+0000 7f5f072de700 1 -- 172.21.15.2:0/4067303824 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5ee406a0e0 msgr2=0x7f5ee406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:02.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.587+0000 7f5f072de700 1 --2- 172.21.15.2:0/4067303824 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5ee406a0e0 0x7f5ee406c5a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5f000cd2a0 tx=0x7f5ef0013000 comp rx=0 tx=0).stop 2023-08-25T03:34:02.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.587+0000 7f5f072de700 1 -- 172.21.15.2:0/4067303824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f00071540 msgr2=0x7f5f000c63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:02.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.587+0000 7f5f072de700 1 --2- 172.21.15.2:0/4067303824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f00071540 0x7f5f000c63e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5efc00c750 tx=0x7f5efc00cb10 comp rx=0 tx=0).stop 2023-08-25T03:34:02.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.587+0000 7f5f072de700 1 -- 172.21.15.2:0/4067303824 shutdown_connections 2023-08-25T03:34:02.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.587+0000 7f5f072de700 1 --2- 172.21.15.2:0/4067303824 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5ee406a0e0 0x7f5ee406c5a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:02.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.587+0000 7f5f072de700 1 --2- 172.21.15.2:0/4067303824 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f00071540 0x7f5f000c63e0 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5efc00c750 tx=0x7f5efc00cb10 comp rx=0 tx=0).stop 2023-08-25T03:34:02.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.587+0000 7f5f072de700 1 --2- 172.21.15.2:0/4067303824 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f0015e6e0 0x7f5f000ccc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:02.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.587+0000 7f5f072de700 1 --2- 172.21.15.2:0/4067303824 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f0015a180 0x7f5f000cc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:02.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.587+0000 7f5f072de700 1 -- 172.21.15.2:0/4067303824 >> 172.21.15.2:0/4067303824 conn(0x7f5f000d0000 msgr2=0x7f5f0006e690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:02.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.590+0000 7f5f072de700 1 -- 172.21.15.2:0/4067303824 shutdown_connections 2023-08-25T03:34:02.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.593+0000 7f5f072de700 1 -- 172.21.15.2:0/4067303824 wait complete. 2023-08-25T03:34:02.597 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_data' created 2023-08-25T03:34:02.615 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-25T03:34:02.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.746+0000 7f8b18028700 1 Processor -- start 2023-08-25T03:34:02.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.746+0000 7f8b18028700 1 -- start start 2023-08-25T03:34:02.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.747+0000 7f8b18028700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8b10159d10 0x7f8b10072370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:02.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.747+0000 7f8b18028700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8b100ce050 0x7f8b100ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:02.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.747+0000 7f8b18028700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8b100cea20 0x7f8b100b4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:02.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.747+0000 7f8b18028700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b1006e2e0 con 0x7f8b10159d10 2023-08-25T03:34:02.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.747+0000 7f8b18028700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b100728b0 con 0x7f8b100cea20 2023-08-25T03:34:02.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.747+0000 7f8b18028700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1006dac0 con 0x7f8b100ce050 2023-08-25T03:34:02.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.747+0000 7f8b15dc4700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8b10159d10 0x7f8b10072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:02.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.747+0000 7f8b15dc4700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8b10159d10 0x7f8b10072370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59090/0 (socket says 172.21.15.2:59090) 2023-08-25T03:34:02.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.747+0000 7f8b15dc4700 1 -- 172.21.15.2:0/3634885233 learned_addr learned my addr 172.21.15.2:0/3634885233 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:02.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.748+0000 7f8b165c5700 1 --2- 172.21.15.2:0/3634885233 >> v2:172.21.15.31:3300/0 conn(0x7f8b100cea20 0x7f8b100b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:02.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.748+0000 7f8b155c3700 1 --2- 172.21.15.2:0/3634885233 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8b100ce050 0x7f8b100ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:02.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.748+0000 7f8b15dc4700 1 -- 172.21.15.2:0/3634885233 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8b100ce050 msgr2=0x7f8b100ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:02.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.748+0000 7f8b15dc4700 1 --2- 172.21.15.2:0/3634885233 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8b100ce050 0x7f8b100ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:02.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.748+0000 7f8b15dc4700 1 -- 172.21.15.2:0/3634885233 >> v2:172.21.15.31:3300/0 conn(0x7f8b100cea20 msgr2=0x7f8b100b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:02.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.748+0000 7f8b15dc4700 1 --2- 172.21.15.2:0/3634885233 >> v2:172.21.15.31:3300/0 conn(0x7f8b100cea20 0x7f8b100b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:02.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.748+0000 7f8b15dc4700 1 -- 172.21.15.2:0/3634885233 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b1006a1e0 con 0x7f8b10159d10 2023-08-25T03:34:02.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.748+0000 7f8b165c5700 1 --2- 172.21.15.2:0/3634885233 >> v2:172.21.15.31:3300/0 conn(0x7f8b100cea20 0x7f8b100b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:02.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.748+0000 7f8b155c3700 1 --2- 172.21.15.2:0/3634885233 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8b100ce050 0x7f8b100ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:02.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.748+0000 7f8b15dc4700 1 --2- 172.21.15.2:0/3634885233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b10159d10 0x7f8b10072370 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8b1015dba0 tx=0x7f8b00009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2d6078d17d24c4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:02.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.748+0000 7f8b14dc2700 1 -- 172.21.15.2:0/3634885233 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b00005e30 con 0x7f8b10159d10 2023-08-25T03:34:02.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.748+0000 7f8b14dc2700 1 -- 172.21.15.2:0/3634885233 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b00004030 con 0x7f8b10159d10 2023-08-25T03:34:02.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.748+0000 7f8b14dc2700 1 -- 172.21.15.2:0/3634885233 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b000043e0 con 0x7f8b10159d10 2023-08-25T03:34:02.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.749+0000 7f8b18028700 1 -- 172.21.15.2:0/3634885233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b10159d10 msgr2=0x7f8b10072370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:02.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.749+0000 7f8b18028700 1 --2- 172.21.15.2:0/3634885233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b10159d10 0x7f8b10072370 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8b1015dba0 tx=0x7f8b00009bd0 comp rx=0 tx=0).stop 2023-08-25T03:34:02.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.749+0000 7f8b18028700 1 -- 172.21.15.2:0/3634885233 shutdown_connections 2023-08-25T03:34:02.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.749+0000 7f8b18028700 1 --2- 172.21.15.2:0/3634885233 >> v2:172.21.15.31:3300/0 conn(0x7f8b100cea20 0x7f8b100b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:02.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.749+0000 7f8b18028700 1 --2- 172.21.15.2:0/3634885233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b10159d10 0x7f8b10072370 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:02.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.749+0000 7f8b18028700 1 --2- 172.21.15.2:0/3634885233 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8b100ce050 0x7f8b100ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:02.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.749+0000 7f8b18028700 1 -- 172.21.15.2:0/3634885233 >> 172.21.15.2:0/3634885233 conn(0x7f8b100cf500 msgr2=0x7f8b100cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:02.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.749+0000 7f8b18028700 1 -- 172.21.15.2:0/3634885233 shutdown_connections 2023-08-25T03:34:02.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.749+0000 7f8b18028700 1 -- 172.21.15.2:0/3634885233 wait complete. 2023-08-25T03:34:02.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.750+0000 7f8b18028700 1 Processor -- start 2023-08-25T03:34:02.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.750+0000 7f8b18028700 1 -- start start 2023-08-25T03:34:02.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.751+0000 7f8b18028700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8b100ce050 0x7f8b10097b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:02.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.751+0000 7f8b18028700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b100cea20 0x7f8b101f4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:02.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.751+0000 7f8b18028700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b10159d10 0x7f8b101f4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:02.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.751+0000 7f8b18028700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b100c97a0 con 0x7f8b100cea20 2023-08-25T03:34:02.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.751+0000 7f8b18028700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b100c94c0 con 0x7f8b10159d10 2023-08-25T03:34:02.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.751+0000 7f8b18028700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b100ac290 con 0x7f8b100ce050 2023-08-25T03:34:02.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.751+0000 7f8b155c3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b100cea20 0x7f8b101f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:02.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.751+0000 7f8b155c3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b100cea20 0x7f8b101f4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59098/0 (socket says 172.21.15.2:59098) 2023-08-25T03:34:02.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.751+0000 7f8b155c3700 1 -- 172.21.15.2:0/2815941375 learned_addr learned my addr 172.21.15.2:0/2815941375 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:02.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.751+0000 7f8b15dc4700 1 --2- 172.21.15.2:0/2815941375 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8b100ce050 0x7f8b10097b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:02.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.751+0000 7f8b165c5700 1 --2- 172.21.15.2:0/2815941375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b10159d10 0x7f8b101f4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:02.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.751+0000 7f8b155c3700 1 -- 172.21.15.2:0/2815941375 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8b100ce050 msgr2=0x7f8b10097b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:02.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.751+0000 7f8b155c3700 1 --2- 172.21.15.2:0/2815941375 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8b100ce050 0x7f8b10097b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:02.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.751+0000 7f8b155c3700 1 -- 172.21.15.2:0/2815941375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b10159d10 msgr2=0x7f8b101f4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:02.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.751+0000 7f8b155c3700 1 --2- 172.21.15.2:0/2815941375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b10159d10 0x7f8b101f4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:02.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.751+0000 7f8b155c3700 1 -- 172.21.15.2:0/2815941375 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b00009500 con 0x7f8b100cea20 2023-08-25T03:34:02.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.751+0000 7f8b165c5700 1 --2- 172.21.15.2:0/2815941375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b10159d10 0x7f8b101f4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:02.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.751+0000 7f8b155c3700 1 --2- 172.21.15.2:0/2815941375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b100cea20 0x7f8b101f4640 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c00c7f0 tx=0x7f8b0c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:02.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.751+0000 7f8b067fc700 1 -- 172.21.15.2:0/2815941375 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c011800 con 0x7f8b100cea20 2023-08-25T03:34:02.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.751+0000 7f8b067fc700 1 -- 172.21.15.2:0/2815941375 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b0c011960 con 0x7f8b100cea20 2023-08-25T03:34:02.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.751+0000 7f8b18028700 1 -- 172.21.15.2:0/2815941375 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b100ac570 con 0x7f8b100cea20 2023-08-25T03:34:02.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.751+0000 7f8b067fc700 1 -- 172.21.15.2:0/2815941375 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b0c011d10 con 0x7f8b100cea20 2023-08-25T03:34:02.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.751+0000 7f8b18028700 1 -- 172.21.15.2:0/2815941375 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b100cb290 con 0x7f8b100cea20 2023-08-25T03:34:02.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.752+0000 7f8b18028700 1 -- 172.21.15.2:0/2815941375 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b100ac0f0 con 0x7f8b100cea20 2023-08-25T03:34:02.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.753+0000 7f8b067fc700 1 -- 172.21.15.2:0/2815941375 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8b0c018360 con 0x7f8b100cea20 2023-08-25T03:34:02.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.753+0000 7f8b067fc700 1 --2- 172.21.15.2:0/2815941375 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8af406a0e0 0x7f8af406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:02.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.753+0000 7f8b067fc700 1 -- 172.21.15.2:0/2815941375 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f8b0c0883f0 con 0x7f8b100cea20 2023-08-25T03:34:02.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.753+0000 7f8b15dc4700 1 --2- 172.21.15.2:0/2815941375 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8af406a0e0 0x7f8af406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:02.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.754+0000 7f8b15dc4700 1 --2- 172.21.15.2:0/2815941375 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8af406a0e0 0x7f8af406c5a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8b1015dba0 tx=0x7f8b00008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:02.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:02.758+0000 7f8b067fc700 1 -- 172.21.15.2:0/2815941375 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8b100ac0f0 con 0x7f8b100cea20 2023-08-25T03:34:02.851 DEBUG:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T03:34:02.854 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T03:34:03.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.007+0000 7f8b18028700 1 -- 172.21.15.2:0/2815941375 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8b100ac0f0 con 0x7f8b100cea20 2023-08-25T03:34:03.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.618+0000 7f8b067fc700 1 -- 172.21.15.2:0/2815941375 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f8b100ac0f0 con 0x7f8b100cea20 2023-08-25T03:34:03.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.622+0000 7f8b18028700 1 -- 172.21.15.2:0/2815941375 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8af406a0e0 msgr2=0x7f8af406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:03.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.622+0000 7f8b18028700 1 --2- 172.21.15.2:0/2815941375 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8af406a0e0 0x7f8af406c5a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8b1015dba0 tx=0x7f8b00008070 comp rx=0 tx=0).stop 2023-08-25T03:34:03.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.622+0000 7f8b18028700 1 -- 172.21.15.2:0/2815941375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b100cea20 msgr2=0x7f8b101f4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:03.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.622+0000 7f8b18028700 1 --2- 172.21.15.2:0/2815941375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b100cea20 0x7f8b101f4640 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8b0c00c7f0 tx=0x7f8b0c00cb00 comp rx=0 tx=0).stop 2023-08-25T03:34:03.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.623+0000 7f8b18028700 1 -- 172.21.15.2:0/2815941375 shutdown_connections 2023-08-25T03:34:03.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.623+0000 7f8b18028700 1 --2- 172.21.15.2:0/2815941375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b10159d10 0x7f8b101f4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:03.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.623+0000 7f8b18028700 1 --2- 172.21.15.2:0/2815941375 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8b100ce050 0x7f8b10097b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:03.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.623+0000 7f8b18028700 1 --2- 172.21.15.2:0/2815941375 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8af406a0e0 0x7f8af406c5a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:03.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.623+0000 7f8b18028700 1 --2- 172.21.15.2:0/2815941375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b100cea20 0x7f8b101f4640 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:03.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.623+0000 7f8b18028700 1 -- 172.21.15.2:0/2815941375 >> 172.21.15.2:0/2815941375 conn(0x7f8b100cf500 msgr2=0x7f8b100cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:03.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.623+0000 7f8b18028700 1 -- 172.21.15.2:0/2815941375 shutdown_connections 2023-08-25T03:34:03.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.623+0000 7f8b18028700 1 -- 172.21.15.2:0/2815941375 wait complete. 2023-08-25T03:34:03.629 INFO:teuthology.orchestra.run.smithi002.stderr:new fs with metadata pool 3 and data pool 4 2023-08-25T03:34:03.643 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:34:03.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.772+0000 7fa68cfc7700 1 Processor -- start 2023-08-25T03:34:03.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.772+0000 7fa68cfc7700 1 -- start start 2023-08-25T03:34:03.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.773+0000 7fa68cfc7700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa6880b1fb0 0x7fa688160f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:03.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.773+0000 7fa68cfc7700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa688161450 0x7fa6880cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:03.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.773+0000 7fa68cfc7700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa68806d450 con 0x7fa6880ce0b0 2023-08-25T03:34:03.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.773+0000 7fa68cfc7700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa68806d730 con 0x7fa688161450 2023-08-25T03:34:03.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.773+0000 7fa68cfc7700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa68806da10 con 0x7fa6880b1fb0 2023-08-25T03:34:03.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.773+0000 7fa685d9b700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa688161450 0x7fa6880cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:03.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.773+0000 7fa685d9b700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa688161450 0x7fa6880cdb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59116/0 (socket says 172.21.15.2:59116) 2023-08-25T03:34:03.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.773+0000 7fa685d9b700 1 -- 172.21.15.2:0/2123074689 learned_addr learned my addr 172.21.15.2:0/2123074689 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:03.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.773+0000 7fa68659c700 1 --2- 172.21.15.2:0/2123074689 >> v2:172.21.15.31:3300/0 conn(0x7fa6880b1fb0 0x7fa688160f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:03.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.773+0000 7fa685d9b700 1 -- 172.21.15.2:0/2123074689 >> v2:172.21.15.31:3300/0 conn(0x7fa6880b1fb0 msgr2=0x7fa688160f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:03.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.773+0000 7fa685d9b700 1 --2- 172.21.15.2:0/2123074689 >> v2:172.21.15.31:3300/0 conn(0x7fa6880b1fb0 0x7fa688160f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:03.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.773+0000 7fa685d9b700 1 -- 172.21.15.2:0/2123074689 >> v1:172.21.15.2:6789/0 conn(0x7fa6880ce0b0 legacy=0x7fa68806b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:03.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.773+0000 7fa68559a700 1 -- 172.21.15.2:0/2123074689 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3409496395 0 0) 0x7fa68806d450 con 0x7fa6880ce0b0 2023-08-25T03:34:03.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.773+0000 7fa685d9b700 1 -- 172.21.15.2:0/2123074689 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa68806dcf0 con 0x7fa688161450 2023-08-25T03:34:03.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.773+0000 7fa68659c700 1 --2- 172.21.15.2:0/2123074689 >> v2:172.21.15.31:3300/0 conn(0x7fa6880b1fb0 0x7fa688160f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:03.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.773+0000 7fa685d9b700 1 --2- 172.21.15.2:0/2123074689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa688161450 0x7fa6880cdb70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa68806e0d0 tx=0x7fa678009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad57aaa2f7ab7b1d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:03.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.774+0000 7fa68559a700 1 -- 172.21.15.2:0/2123074689 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67800b870 con 0x7fa688161450 2023-08-25T03:34:03.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.774+0000 7fa68559a700 1 -- 172.21.15.2:0/2123074689 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa67800b9d0 con 0x7fa688161450 2023-08-25T03:34:03.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.774+0000 7fa68559a700 1 -- 172.21.15.2:0/2123074689 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa67800bd60 con 0x7fa688161450 2023-08-25T03:34:03.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.774+0000 7fa68cfc7700 1 -- 172.21.15.2:0/2123074689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa688161450 msgr2=0x7fa6880cdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:03.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.774+0000 7fa68cfc7700 1 --2- 172.21.15.2:0/2123074689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa688161450 0x7fa6880cdb70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa68806e0d0 tx=0x7fa678009bd0 comp rx=0 tx=0).stop 2023-08-25T03:34:03.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.774+0000 7fa68cfc7700 1 -- 172.21.15.2:0/2123074689 shutdown_connections 2023-08-25T03:34:03.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.774+0000 7fa68cfc7700 1 --2- 172.21.15.2:0/2123074689 >> v2:172.21.15.31:3300/0 conn(0x7fa6880b1fb0 0x7fa688160f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:03.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.774+0000 7fa68cfc7700 1 --2- 172.21.15.2:0/2123074689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa688161450 0x7fa6880cdb70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:03.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.774+0000 7fa68cfc7700 1 -- 172.21.15.2:0/2123074689 >> 172.21.15.2:0/2123074689 conn(0x7fa6880cfea0 msgr2=0x7fa6880d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:03.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.774+0000 7fa68cfc7700 1 -- 172.21.15.2:0/2123074689 shutdown_connections 2023-08-25T03:34:03.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.774+0000 7fa68cfc7700 1 -- 172.21.15.2:0/2123074689 wait complete. 2023-08-25T03:34:03.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.776+0000 7fa68cfc7700 1 Processor -- start 2023-08-25T03:34:03.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.776+0000 7fa68cfc7700 1 -- start start 2023-08-25T03:34:03.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.776+0000 7fa68cfc7700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa6880b1fb0 0x7fa6881f6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:03.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.776+0000 7fa68cfc7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6880ce0b0 0x7fa6880b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:03.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.776+0000 7fa68cfc7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa688151600 0x7fa688156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:03.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.776+0000 7fa68cfc7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa688159680 con 0x7fa688151600 2023-08-25T03:34:03.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.776+0000 7fa68cfc7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6881f7470 con 0x7fa6880ce0b0 2023-08-25T03:34:03.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.776+0000 7fa68cfc7700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6880b3770 con 0x7fa6880b1fb0 2023-08-25T03:34:03.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.776+0000 7fa686d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa688151600 0x7fa688156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:03.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.777+0000 7fa686d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa688151600 0x7fa688156420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59122/0 (socket says 172.21.15.2:59122) 2023-08-25T03:34:03.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.777+0000 7fa686d9d700 1 -- 172.21.15.2:0/341769883 learned_addr learned my addr 172.21.15.2:0/341769883 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:03.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.777+0000 7fa68659c700 1 --2- 172.21.15.2:0/341769883 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa6880b1fb0 0x7fa6881f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:03.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.777+0000 7fa685d9b700 1 --2- 172.21.15.2:0/341769883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6880ce0b0 0x7fa6880b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:03.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.777+0000 7fa686d9d700 1 -- 172.21.15.2:0/341769883 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa6880b1fb0 msgr2=0x7fa6881f6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:03.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.777+0000 7fa686d9d700 1 --2- 172.21.15.2:0/341769883 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa6880b1fb0 0x7fa6881f6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:03.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.777+0000 7fa686d9d700 1 -- 172.21.15.2:0/341769883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6880ce0b0 msgr2=0x7fa6880b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:03.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.777+0000 7fa686d9d700 1 --2- 172.21.15.2:0/341769883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6880ce0b0 0x7fa6880b3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:03.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.777+0000 7fa686d9d700 1 -- 172.21.15.2:0/341769883 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa678009550 con 0x7fa688151600 2023-08-25T03:34:03.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.777+0000 7fa68659c700 1 --2- 172.21.15.2:0/341769883 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa6880b1fb0 0x7fa6881f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:03.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.777+0000 7fa685d9b700 1 --2- 172.21.15.2:0/341769883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6880ce0b0 0x7fa6880b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:03.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.777+0000 7fa686d9d700 1 --2- 172.21.15.2:0/341769883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa688151600 0x7fa688156420 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa680001560 tx=0x7fa68000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:03.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.777+0000 7fa676ffd700 1 -- 172.21.15.2:0/341769883 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa680003e30 con 0x7fa688151600 2023-08-25T03:34:03.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.777+0000 7fa676ffd700 1 -- 172.21.15.2:0/341769883 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa680005070 con 0x7fa688151600 2023-08-25T03:34:03.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.777+0000 7fa68cfc7700 1 -- 172.21.15.2:0/341769883 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6881f6bd0 con 0x7fa688151600 2023-08-25T03:34:03.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.777+0000 7fa676ffd700 1 -- 172.21.15.2:0/341769883 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa680005420 con 0x7fa688151600 2023-08-25T03:34:03.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.777+0000 7fa68cfc7700 1 -- 172.21.15.2:0/341769883 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6881f7190 con 0x7fa688151600 2023-08-25T03:34:03.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.778+0000 7fa68cfc7700 1 -- 172.21.15.2:0/341769883 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa688066f70 con 0x7fa688151600 2023-08-25T03:34:03.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.779+0000 7fa676ffd700 1 -- 172.21.15.2:0/341769883 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa680005580 con 0x7fa688151600 2023-08-25T03:34:03.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.779+0000 7fa676ffd700 1 --2- 172.21.15.2:0/341769883 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa66806a0e0 0x7fa66806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:03.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.779+0000 7fa676ffd700 1 -- 172.21.15.2:0/341769883 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa680087c70 con 0x7fa688151600 2023-08-25T03:34:03.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.779+0000 7fa68659c700 1 --2- 172.21.15.2:0/341769883 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa66806a0e0 0x7fa66806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:03.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.780+0000 7fa68659c700 1 --2- 172.21.15.2:0/341769883 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa66806a0e0 0x7fa66806c5a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009960 tx=0x7fa67c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:03.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.784+0000 7fa676ffd700 1 -- 172.21.15.2:0/341769883 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa6800533f0 con 0x7fa688151600 2023-08-25T03:34:03.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.984+0000 7fa68cfc7700 1 -- 172.21.15.2:0/341769883 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6880746d0 con 0x7fa688151600 2023-08-25T03:34:03.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.985+0000 7fa676ffd700 1 -- 172.21.15.2:0/341769883 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18148 (secure 0 0 0) 0x7fa680017020 con 0x7fa688151600 2023-08-25T03:34:03.986 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:03.986 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":23,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:34:03.008776+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:34:00.869118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T03:34:01.987016+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:34:03.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.987+0000 7fa68cfc7700 1 -- 172.21.15.2:0/341769883 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa66806a0e0 msgr2=0x7fa66806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:03.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.987+0000 7fa68cfc7700 1 --2- 172.21.15.2:0/341769883 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa66806a0e0 0x7fa66806c5a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009960 tx=0x7fa67c008040 comp rx=0 tx=0).stop 2023-08-25T03:34:03.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.987+0000 7fa68cfc7700 1 -- 172.21.15.2:0/341769883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa688151600 msgr2=0x7fa688156420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:03.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.987+0000 7fa68cfc7700 1 --2- 172.21.15.2:0/341769883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa688151600 0x7fa688156420 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa680001560 tx=0x7fa68000cc00 comp rx=0 tx=0).stop 2023-08-25T03:34:03.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.987+0000 7fa68cfc7700 1 -- 172.21.15.2:0/341769883 shutdown_connections 2023-08-25T03:34:03.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.987+0000 7fa68cfc7700 1 --2- 172.21.15.2:0/341769883 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa66806a0e0 0x7fa66806c5a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:03.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.987+0000 7fa68cfc7700 1 --2- 172.21.15.2:0/341769883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa688151600 0x7fa688156420 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:03.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.987+0000 7fa68cfc7700 1 --2- 172.21.15.2:0/341769883 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa6880b1fb0 0x7fa6881f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:03.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.987+0000 7fa68cfc7700 1 --2- 172.21.15.2:0/341769883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6880ce0b0 0x7fa6880b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:03.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.987+0000 7fa68cfc7700 1 -- 172.21.15.2:0/341769883 >> 172.21.15.2:0/341769883 conn(0x7fa6880cfea0 msgr2=0x7fa68806b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:03.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.987+0000 7fa68cfc7700 1 -- 172.21.15.2:0/341769883 shutdown_connections 2023-08-25T03:34:03.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:03.987+0000 7fa68cfc7700 1 -- 172.21.15.2:0/341769883 wait complete. 2023-08-25T03:34:04.000 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:34:04.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.125+0000 7efeb57ec700 1 Processor -- start 2023-08-25T03:34:04.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.125+0000 7efeb57ec700 1 -- start start 2023-08-25T03:34:04.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.126+0000 7efeb57ec700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efeb015a2e0 0x7efeb015dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:04.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.126+0000 7efeb57ec700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7efeb00b13b0 0x7efeb006d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:04.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.126+0000 7efeb57ec700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb006ff30 con 0x7efeb00ae3a0 2023-08-25T03:34:04.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.126+0000 7efeb57ec700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeb015e500 con 0x7efeb00b13b0 2023-08-25T03:34:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.126+0000 7efeb57ec700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeb015e7e0 con 0x7efeb015a2e0 2023-08-25T03:34:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.126+0000 7efeaf7fe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7efeb00b13b0 0x7efeb006d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.126+0000 7efeaf7fe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7efeb00b13b0 0x7efeb006d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59130/0 (socket says 172.21.15.2:59130) 2023-08-25T03:34:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.126+0000 7efeaf7fe700 1 -- 172.21.15.2:0/2920824362 learned_addr learned my addr 172.21.15.2:0/2920824362 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.126+0000 7efeaeffd700 1 --2- 172.21.15.2:0/2920824362 >> v2:172.21.15.31:3300/0 conn(0x7efeb015a2e0 0x7efeb015dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.126+0000 7efeaf7fe700 1 -- 172.21.15.2:0/2920824362 >> v2:172.21.15.31:3300/0 conn(0x7efeb015a2e0 msgr2=0x7efeb015dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:04.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.126+0000 7efeaf7fe700 1 --2- 172.21.15.2:0/2920824362 >> v2:172.21.15.31:3300/0 conn(0x7efeb015a2e0 0x7efeb015dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.126+0000 7efeaf7fe700 1 -- 172.21.15.2:0/2920824362 >> v1:172.21.15.31:6789/0 conn(0x7efeb00ae3a0 legacy=0x7efeb00b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.126+0000 7efeaf7fe700 1 -- 172.21.15.2:0/2920824362 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb015eac0 con 0x7efeb00b13b0 2023-08-25T03:34:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.126+0000 7efeaf7fe700 1 --2- 172.21.15.2:0/2920824362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efeb00b13b0 0x7efeb006d130 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efeb00c9df0 tx=0x7efea800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391c26fe6a183ef2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.127+0000 7efeadffb700 1 -- 172.21.15.2:0/2920824362 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea800e070 con 0x7efeb00b13b0 2023-08-25T03:34:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.127+0000 7efeadffb700 1 -- 172.21.15.2:0/2920824362 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea800b9c0 con 0x7efeb00b13b0 2023-08-25T03:34:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.127+0000 7efeadffb700 1 -- 172.21.15.2:0/2920824362 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea800bd70 con 0x7efeb00b13b0 2023-08-25T03:34:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.127+0000 7efeb57ec700 1 -- 172.21.15.2:0/2920824362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efeb00b13b0 msgr2=0x7efeb006d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:04.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.127+0000 7efeb57ec700 1 --2- 172.21.15.2:0/2920824362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efeb00b13b0 0x7efeb006d130 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efeb00c9df0 tx=0x7efea800b330 comp rx=0 tx=0).stop 2023-08-25T03:34:04.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.127+0000 7efeb57ec700 1 -- 172.21.15.2:0/2920824362 shutdown_connections 2023-08-25T03:34:04.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.127+0000 7efeb57ec700 1 --2- 172.21.15.2:0/2920824362 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efeb00b13b0 0x7efeb006d130 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:04.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.127+0000 7efeb57ec700 1 --2- 172.21.15.2:0/2920824362 >> v2:172.21.15.31:3300/0 conn(0x7efeb015a2e0 0x7efeb015dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:04.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.127+0000 7efeb57ec700 1 -- 172.21.15.2:0/2920824362 >> 172.21.15.2:0/2920824362 conn(0x7efeb00cbd30 msgr2=0x7efeb00cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:04.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.127+0000 7efeb57ec700 1 -- 172.21.15.2:0/2920824362 shutdown_connections 2023-08-25T03:34:04.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.127+0000 7efeb57ec700 1 -- 172.21.15.2:0/2920824362 wait complete. 2023-08-25T03:34:04.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.129+0000 7efeb57ec700 1 Processor -- start 2023-08-25T03:34:04.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.129+0000 7efeb57ec700 1 -- start start 2023-08-25T03:34:04.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.129+0000 7efeb57ec700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efeb00ae3a0 0x7efeb00c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:04.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.129+0000 7efeb57ec700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efeb00b13b0 0x7efeb009aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:04.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.129+0000 7efeb57ec700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efeb015a2e0 0x7efeb01588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:04.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.129+0000 7efeb57ec700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb00c33c0 con 0x7efeb015a2e0 2023-08-25T03:34:04.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.129+0000 7efeb57ec700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb00c3d50 con 0x7efeb00b13b0 2023-08-25T03:34:04.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.129+0000 7efeb57ec700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb00c4030 con 0x7efeb00ae3a0 2023-08-25T03:34:04.134 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.130+0000 7efeaf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efeb015a2e0 0x7efeb01588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:04.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.130+0000 7efeaf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efeb015a2e0 0x7efeb01588d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59134/0 (socket says 172.21.15.2:59134) 2023-08-25T03:34:04.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.130+0000 7efeaf7fe700 1 -- 172.21.15.2:0/3892214948 learned_addr learned my addr 172.21.15.2:0/3892214948 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:04.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.130+0000 7efeaeffd700 1 --2- 172.21.15.2:0/3892214948 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efeb00ae3a0 0x7efeb00c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:04.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.130+0000 7efeae7fc700 1 --2- 172.21.15.2:0/3892214948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efeb00b13b0 0x7efeb009aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:04.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.130+0000 7efeaf7fe700 1 -- 172.21.15.2:0/3892214948 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efeb00ae3a0 msgr2=0x7efeb00c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:04.135 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.130+0000 7efeaf7fe700 1 --2- 172.21.15.2:0/3892214948 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efeb00ae3a0 0x7efeb00c8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:04.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.130+0000 7efeaf7fe700 1 -- 172.21.15.2:0/3892214948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efeb00b13b0 msgr2=0x7efeb009aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:04.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.130+0000 7efeaf7fe700 1 --2- 172.21.15.2:0/3892214948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efeb00b13b0 0x7efeb009aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:04.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.130+0000 7efeaf7fe700 1 -- 172.21.15.2:0/3892214948 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea8009b30 con 0x7efeb015a2e0 2023-08-25T03:34:04.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.130+0000 7efeaf7fe700 1 --2- 172.21.15.2:0/3892214948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efeb015a2e0 0x7efeb01588d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efea800b730 tx=0x7efea8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:04.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.130+0000 7efe977fe700 1 -- 172.21.15.2:0/3892214948 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea8004830 con 0x7efeb015a2e0 2023-08-25T03:34:04.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.130+0000 7efe977fe700 1 -- 172.21.15.2:0/3892214948 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea80099b0 con 0x7efeb015a2e0 2023-08-25T03:34:04.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.130+0000 7efeb57ec700 1 -- 172.21.15.2:0/3892214948 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb0158e10 con 0x7efeb015a2e0 2023-08-25T03:34:04.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.130+0000 7efe977fe700 1 -- 172.21.15.2:0/3892214948 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea801c680 con 0x7efeb015a2e0 2023-08-25T03:34:04.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.130+0000 7efeb57ec700 1 -- 172.21.15.2:0/3892214948 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb00c3610 con 0x7efeb015a2e0 2023-08-25T03:34:04.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.131+0000 7efeb57ec700 1 -- 172.21.15.2:0/3892214948 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb00746d0 con 0x7efeb015a2e0 2023-08-25T03:34:04.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.132+0000 7efe977fe700 1 -- 172.21.15.2:0/3892214948 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7efea801c7e0 con 0x7efeb015a2e0 2023-08-25T03:34:04.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.132+0000 7efe977fe700 1 --2- 172.21.15.2:0/3892214948 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7efe9c06a0e0 0x7efe9c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:04.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.132+0000 7efe977fe700 1 -- 172.21.15.2:0/3892214948 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efea808a030 con 0x7efeb015a2e0 2023-08-25T03:34:04.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.132+0000 7efeaeffd700 1 --2- 172.21.15.2:0/3892214948 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7efe9c06a0e0 0x7efe9c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:04.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.133+0000 7efeaeffd700 1 --2- 172.21.15.2:0/3892214948 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7efe9c06a0e0 0x7efe9c06c5a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efea4006b20 tx=0x7efea4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:04.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.137+0000 7efe977fe700 1 -- 172.21.15.2:0/3892214948 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7efea80557b0 con 0x7efeb015a2e0 2023-08-25T03:34:04.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.338+0000 7efeb57ec700 1 -- 172.21.15.2:0/3892214948 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efeb00c4170 con 0x7efeb015a2e0 2023-08-25T03:34:04.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.339+0000 7efe977fe700 1 -- 172.21.15.2:0/3892214948 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+18148 (secure 0 0 0) 0x7efea8058dd0 con 0x7efeb015a2e0 2023-08-25T03:34:04.340 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:04.341 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":23,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:34:03.008776+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:34:00.869118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T03:34:01.987016+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:34:04.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.341+0000 7efeb57ec700 1 -- 172.21.15.2:0/3892214948 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7efe9c06a0e0 msgr2=0x7efe9c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:04.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.341+0000 7efeb57ec700 1 --2- 172.21.15.2:0/3892214948 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7efe9c06a0e0 0x7efe9c06c5a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efea4006b20 tx=0x7efea4008040 comp rx=0 tx=0).stop 2023-08-25T03:34:04.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.341+0000 7efeb57ec700 1 -- 172.21.15.2:0/3892214948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efeb015a2e0 msgr2=0x7efeb01588d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:04.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.341+0000 7efeb57ec700 1 --2- 172.21.15.2:0/3892214948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efeb015a2e0 0x7efeb01588d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efea800b730 tx=0x7efea8004ad0 comp rx=0 tx=0).stop 2023-08-25T03:34:04.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.341+0000 7efeb57ec700 1 -- 172.21.15.2:0/3892214948 shutdown_connections 2023-08-25T03:34:04.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.341+0000 7efeb57ec700 1 --2- 172.21.15.2:0/3892214948 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7efe9c06a0e0 0x7efe9c06c5a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:04.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.341+0000 7efeb57ec700 1 --2- 172.21.15.2:0/3892214948 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efeb015a2e0 0x7efeb01588d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:04.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.341+0000 7efeb57ec700 1 --2- 172.21.15.2:0/3892214948 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efeb00ae3a0 0x7efeb00c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:04.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.341+0000 7efeb57ec700 1 --2- 172.21.15.2:0/3892214948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efeb00b13b0 0x7efeb009aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:04.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.341+0000 7efeb57ec700 1 -- 172.21.15.2:0/3892214948 >> 172.21.15.2:0/3892214948 conn(0x7efeb00cbd30 msgr2=0x7efeb00af680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:04.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.341+0000 7efeb57ec700 1 -- 172.21.15.2:0/3892214948 shutdown_connections 2023-08-25T03:34:04.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.341+0000 7efeb57ec700 1 -- 172.21.15.2:0/3892214948 wait complete. 2023-08-25T03:34:04.356 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-25T03:34:04.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.483+0000 7f8fc0ee4700 1 Processor -- start 2023-08-25T03:34:04.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.483+0000 7f8fc0ee4700 1 -- start start 2023-08-25T03:34:04.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.484+0000 7f8fc0ee4700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8fbc159f10 0x7f8fbc15dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:04.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.484+0000 7f8fc0ee4700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fbc06eaf0 con 0x7f8fbc0cacf0 2023-08-25T03:34:04.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.484+0000 7f8fc0ee4700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fbc15e020 con 0x7f8fbc0cedf0 2023-08-25T03:34:04.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.484+0000 7f8fc0ee4700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fbc06e850 con 0x7f8fbc159f10 2023-08-25T03:34:04.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.484+0000 7f8fba59c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8fbc159f10 0x7f8fbc15dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:04.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.484+0000 7f8fba59c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8fbc159f10 0x7f8fbc15dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59140/0 (socket says 172.21.15.2:59140) 2023-08-25T03:34:04.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.484+0000 7f8fbad9d700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f8fbc0cacf0 0x7f8fbc06cf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45810/0 (socket says 172.21.15.2:45810) 2023-08-25T03:34:04.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.484+0000 7f8fba59c700 1 -- 172.21.15.2:0/1804440075 learned_addr learned my addr 172.21.15.2:0/1804440075 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:04.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.485+0000 7f8fba59c700 1 -- 172.21.15.2:0/1804440075 >> v1:172.21.15.31:6789/0 conn(0x7f8fbc0cedf0 legacy=0x7f8fbc06ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:04.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.485+0000 7f8fba59c700 1 -- 172.21.15.2:0/1804440075 >> v1:172.21.15.2:6789/0 conn(0x7f8fbc0cacf0 legacy=0x7f8fbc06cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:04.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.485+0000 7f8fba59c700 1 -- 172.21.15.2:0/1804440075 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fbc15e2c0 con 0x7f8fbc159f10 2023-08-25T03:34:04.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.485+0000 7f8fb959a700 1 -- 172.21.15.2:0/1804440075 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1593793440 0 0) 0x7f8fbc06eaf0 con 0x7f8fbc0cacf0 2023-08-25T03:34:04.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.485+0000 7f8fba59c700 1 --2- 172.21.15.2:0/1804440075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fbc159f10 0x7f8fbc15dae0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000bb90 tx=0x7f8fb000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70969b239cedbe7e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:04.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.485+0000 7f8fb959a700 1 -- 172.21.15.2:0/1804440075 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb000dd50 con 0x7f8fbc159f10 2023-08-25T03:34:04.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.485+0000 7f8fb959a700 1 -- 172.21.15.2:0/1804440075 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb0010470 con 0x7f8fbc159f10 2023-08-25T03:34:04.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.485+0000 7f8fb959a700 1 -- 172.21.15.2:0/1804440075 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0010800 con 0x7f8fbc159f10 2023-08-25T03:34:04.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.485+0000 7f8fc0ee4700 1 -- 172.21.15.2:0/1804440075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fbc159f10 msgr2=0x7f8fbc15dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:04.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.485+0000 7f8fc0ee4700 1 --2- 172.21.15.2:0/1804440075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fbc159f10 0x7f8fbc15dae0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000bb90 tx=0x7f8fb000bea0 comp rx=0 tx=0).stop 2023-08-25T03:34:04.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.485+0000 7f8fc0ee4700 1 -- 172.21.15.2:0/1804440075 shutdown_connections 2023-08-25T03:34:04.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.485+0000 7f8fc0ee4700 1 --2- 172.21.15.2:0/1804440075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fbc159f10 0x7f8fbc15dae0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:04.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.485+0000 7f8fc0ee4700 1 -- 172.21.15.2:0/1804440075 >> 172.21.15.2:0/1804440075 conn(0x7f8fbc0cfd90 msgr2=0x7f8fbc0d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:04.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.485+0000 7f8fc0ee4700 1 -- 172.21.15.2:0/1804440075 shutdown_connections 2023-08-25T03:34:04.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.486+0000 7f8fc0ee4700 1 -- 172.21.15.2:0/1804440075 wait complete. 2023-08-25T03:34:04.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.487+0000 7f8fc0ee4700 1 Processor -- start 2023-08-25T03:34:04.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.487+0000 7f8fc0ee4700 1 -- start start 2023-08-25T03:34:04.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.488+0000 7f8fc0ee4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fbc0cacf0 0x7f8fbc14d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:04.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.488+0000 7f8fc0ee4700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8fbc0cedf0 0x7f8fbc157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:04.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.488+0000 7f8fc0ee4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fbc159f10 0x7f8fbc158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:04.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.488+0000 7f8fc0ee4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc0c6240 con 0x7f8fbc159f10 2023-08-25T03:34:04.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.488+0000 7f8fc0ee4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc0c5630 con 0x7f8fbc0cacf0 2023-08-25T03:34:04.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.488+0000 7f8fc0ee4700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fbc0c6660 con 0x7f8fbc0cedf0 2023-08-25T03:34:04.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.488+0000 7f8fbad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fbc159f10 0x7f8fbc158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:04.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.488+0000 7f8fbad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fbc159f10 0x7f8fbc158100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59150/0 (socket says 172.21.15.2:59150) 2023-08-25T03:34:04.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.488+0000 7f8fbad9d700 1 -- 172.21.15.2:0/1777865146 learned_addr learned my addr 172.21.15.2:0/1777865146 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:04.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.488+0000 7f8fba59c700 1 --2- 172.21.15.2:0/1777865146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fbc0cacf0 0x7f8fbc14d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:04.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.488+0000 7f8fb9d9b700 1 --2- 172.21.15.2:0/1777865146 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8fbc0cedf0 0x7f8fbc157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:04.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.488+0000 7f8fbad9d700 1 -- 172.21.15.2:0/1777865146 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8fbc0cedf0 msgr2=0x7f8fbc157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:04.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.488+0000 7f8fbad9d700 1 --2- 172.21.15.2:0/1777865146 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8fbc0cedf0 0x7f8fbc157bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:04.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.488+0000 7f8fbad9d700 1 -- 172.21.15.2:0/1777865146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fbc0cacf0 msgr2=0x7f8fbc14d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:04.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.488+0000 7f8fbad9d700 1 --2- 172.21.15.2:0/1777865146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fbc0cacf0 0x7f8fbc14d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:04.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.488+0000 7f8fbad9d700 1 -- 172.21.15.2:0/1777865146 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb000b7f0 con 0x7f8fbc159f10 2023-08-25T03:34:04.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.488+0000 7f8fba59c700 1 --2- 172.21.15.2:0/1777865146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fbc0cacf0 0x7f8fbc14d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:04.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.488+0000 7f8fb9d9b700 1 --2- 172.21.15.2:0/1777865146 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8fbc0cedf0 0x7f8fbc157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:04.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.488+0000 7f8fbad9d700 1 --2- 172.21.15.2:0/1777865146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fbc159f10 0x7f8fbc158100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400cb70 tx=0x7f8fb400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:04.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.488+0000 7f8faaffd700 1 -- 172.21.15.2:0/1777865146 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb400ab10 con 0x7f8fbc159f10 2023-08-25T03:34:04.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.488+0000 7f8faaffd700 1 -- 172.21.15.2:0/1777865146 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb400ac90 con 0x7f8fbc159f10 2023-08-25T03:34:04.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.488+0000 7f8fc0ee4700 1 -- 172.21.15.2:0/1777865146 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fbc0c58a0 con 0x7f8fbc159f10 2023-08-25T03:34:04.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.488+0000 7f8faaffd700 1 -- 172.21.15.2:0/1777865146 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb40070d0 con 0x7f8fbc159f10 2023-08-25T03:34:04.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.488+0000 7f8fc0ee4700 1 -- 172.21.15.2:0/1777865146 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fbc0c5e60 con 0x7f8fbc159f10 2023-08-25T03:34:04.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.489+0000 7f8fc0ee4700 1 -- 172.21.15.2:0/1777865146 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fbc0c6380 con 0x7f8fbc159f10 2023-08-25T03:34:04.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.490+0000 7f8faaffd700 1 -- 172.21.15.2:0/1777865146 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8fb4007230 con 0x7f8fbc159f10 2023-08-25T03:34:04.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.490+0000 7f8faaffd700 1 --2- 172.21.15.2:0/1777865146 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8f9c06a0e0 0x7f8f9c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:04.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.491+0000 7f8faaffd700 1 -- 172.21.15.2:0/1777865146 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8fb4088360 con 0x7f8fbc159f10 2023-08-25T03:34:04.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.491+0000 7f8fba59c700 1 --2- 172.21.15.2:0/1777865146 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8f9c06a0e0 0x7f8f9c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:04.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.491+0000 7f8fba59c700 1 --2- 172.21.15.2:0/1777865146 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8f9c06a0e0 0x7f8f9c06c5a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000bee0 tx=0x7f8fb0013000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:04.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.495+0000 7f8faaffd700 1 -- 172.21.15.2:0/1777865146 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8fbc0c6380 con 0x7f8fbc159f10 2023-08-25T03:34:04.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:04.743+0000 7f8fc0ee4700 1 -- 172.21.15.2:0/1777865146 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8fbc0c6380 con 0x7f8fbc159f10 2023-08-25T03:34:05.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.614+0000 7f8faaffd700 1 -- 172.21.15.2:0/1777865146 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8fbc0c6380 con 0x7f8fbc159f10 2023-08-25T03:34:05.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.617+0000 7f8fc0ee4700 1 -- 172.21.15.2:0/1777865146 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8f9c06a0e0 msgr2=0x7f8f9c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:05.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.617+0000 7f8fc0ee4700 1 --2- 172.21.15.2:0/1777865146 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8f9c06a0e0 0x7f8f9c06c5a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000bee0 tx=0x7f8fb0013000 comp rx=0 tx=0).stop 2023-08-25T03:34:05.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.617+0000 7f8fc0ee4700 1 -- 172.21.15.2:0/1777865146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fbc159f10 msgr2=0x7f8fbc158100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:05.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.617+0000 7f8fc0ee4700 1 --2- 172.21.15.2:0/1777865146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fbc159f10 0x7f8fbc158100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8fb400cb70 tx=0x7f8fb400cf30 comp rx=0 tx=0).stop 2023-08-25T03:34:05.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.617+0000 7f8fc0ee4700 1 -- 172.21.15.2:0/1777865146 shutdown_connections 2023-08-25T03:34:05.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.617+0000 7f8fc0ee4700 1 --2- 172.21.15.2:0/1777865146 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8f9c06a0e0 0x7f8f9c06c5a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:05.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.617+0000 7f8fc0ee4700 1 --2- 172.21.15.2:0/1777865146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fbc159f10 0x7f8fbc158100 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:05.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.617+0000 7f8fc0ee4700 1 --2- 172.21.15.2:0/1777865146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fbc0cacf0 0x7f8fbc14d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:05.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.617+0000 7f8fc0ee4700 1 --2- 172.21.15.2:0/1777865146 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8fbc0cedf0 0x7f8fbc157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:05.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.617+0000 7f8fc0ee4700 1 -- 172.21.15.2:0/1777865146 >> 172.21.15.2:0/1777865146 conn(0x7f8fbc0cfd90 msgr2=0x7f8fbc0cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:05.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.617+0000 7f8fc0ee4700 1 -- 172.21.15.2:0/1777865146 shutdown_connections 2023-08-25T03:34:05.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.617+0000 7f8fc0ee4700 1 -- 172.21.15.2:0/1777865146 wait complete. 2023-08-25T03:34:05.633 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-25T03:34:05.633 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-25T03:34:05.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.768+0000 7efce821d700 1 Processor -- start 2023-08-25T03:34:05.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.768+0000 7efce821d700 1 -- start start 2023-08-25T03:34:05.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.769+0000 7efce821d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efce01607f0 0x7efce0070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:05.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.769+0000 7efce821d700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efce0160bd0 con 0x7efce00af180 2023-08-25T03:34:05.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.769+0000 7efce821d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efce00cdd00 con 0x7efce006a1e0 2023-08-25T03:34:05.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.769+0000 7efce821d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efce00713a0 con 0x7efce01607f0 2023-08-25T03:34:05.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.769+0000 7efce57b8700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7efce00af180 0x7efce00ac8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45822/0 (socket says 172.21.15.2:45822) 2023-08-25T03:34:05.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.769+0000 7efce57b8700 1 -- 172.21.15.2:0/3357288598 learned_addr learned my addr 172.21.15.2:0/3357288598 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:05.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.769+0000 7efce5fb9700 1 --2- 172.21.15.2:0/3357288598 >> v2:172.21.15.31:3300/0 conn(0x7efce01607f0 0x7efce0070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:05.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.769+0000 7efce4fb7700 1 -- 172.21.15.2:0/3357288598 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2633612106 0 0) 0x7efce0160bd0 con 0x7efce00af180 2023-08-25T03:34:05.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.770+0000 7efce4fb7700 1 -- 172.21.15.2:0/3357288598 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efccc0035b0 con 0x7efce00af180 2023-08-25T03:34:05.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.770+0000 7efce4fb7700 1 -- 172.21.15.2:0/3357288598 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 162188576 0 0) 0x7efce00cdd00 con 0x7efce006a1e0 2023-08-25T03:34:05.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.770+0000 7efce4fb7700 1 -- 172.21.15.2:0/3357288598 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efce0160bd0 con 0x7efce006a1e0 2023-08-25T03:34:05.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.770+0000 7efce4fb7700 1 -- 172.21.15.2:0/3357288598 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2086604155 0 0) 0x7efccc0035b0 con 0x7efce00af180 2023-08-25T03:34:05.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.770+0000 7efce4fb7700 1 -- 172.21.15.2:0/3357288598 >> v2:172.21.15.31:3300/0 conn(0x7efce01607f0 msgr2=0x7efce0070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:05.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.770+0000 7efce4fb7700 1 --2- 172.21.15.2:0/3357288598 >> v2:172.21.15.31:3300/0 conn(0x7efce01607f0 0x7efce0070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:05.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.770+0000 7efce4fb7700 1 -- 172.21.15.2:0/3357288598 >> v1:172.21.15.31:6789/0 conn(0x7efce006a1e0 legacy=0x7efce006ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:05.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.770+0000 7efce4fb7700 1 -- 172.21.15.2:0/3357288598 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce00cda00 con 0x7efce00af180 2023-08-25T03:34:05.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.770+0000 7efce4fb7700 1 -- 172.21.15.2:0/3357288598 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7efcdc001670 con 0x7efce00af180 2023-08-25T03:34:05.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.770+0000 7efce4fb7700 1 -- 172.21.15.2:0/3357288598 >> v1:172.21.15.2:6789/0 conn(0x7efce00af180 legacy=0x7efce00ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:05.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.770+0000 7efce4fb7700 1 --2- 172.21.15.2:0/3357288598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efccc003b90 0x7efccc003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:05.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.770+0000 7efce4fb7700 1 --2- 172.21.15.2:0/3357288598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efccc004850 0x7efccc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:05.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.770+0000 7efce4fb7700 1 --2- 172.21.15.2:0/3357288598 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efccc005210 0x7efccc00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:05.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.770+0000 7efce4fb7700 1 -- 172.21.15.2:0/3357288598 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efccc00c110 con 0x7efccc004850 2023-08-25T03:34:05.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.770+0000 7efce4fb7700 1 -- 172.21.15.2:0/3357288598 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efccc00c280 con 0x7efccc003b90 2023-08-25T03:34:05.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.770+0000 7efce4fb7700 1 -- 172.21.15.2:0/3357288598 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efccc00c440 con 0x7efccc005210 2023-08-25T03:34:05.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.770+0000 7efce57b8700 1 --2- 172.21.15.2:0/3357288598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efccc004850 0x7efccc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:05.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.770+0000 7efce5fb9700 1 --2- 172.21.15.2:0/3357288598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efccc003b90 0x7efccc003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:05.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.770+0000 7efce67ba700 1 --2- 172.21.15.2:0/3357288598 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efccc005210 0x7efccc00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:05.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.770+0000 7efce57b8700 1 -- 172.21.15.2:0/3357288598 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efccc005210 msgr2=0x7efccc00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:05.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.770+0000 7efce57b8700 1 --2- 172.21.15.2:0/3357288598 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efccc005210 0x7efccc00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:05.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.770+0000 7efce57b8700 1 -- 172.21.15.2:0/3357288598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efccc003b90 msgr2=0x7efccc003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:05.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.770+0000 7efce57b8700 1 --2- 172.21.15.2:0/3357288598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efccc003b90 0x7efccc003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:05.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.770+0000 7efce67ba700 1 -- 172.21.15.2:0/3357288598 reap_dead start 2023-08-25T03:34:05.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.770+0000 7efce57b8700 1 -- 172.21.15.2:0/3357288598 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efccc0035b0 con 0x7efccc004850 2023-08-25T03:34:05.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.771+0000 7efce57b8700 1 --2- 172.21.15.2:0/3357288598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efccc004850 0x7efccc004cd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00c240 tx=0x7efccc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fbae3f64eb9787 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:05.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.771+0000 7efce4fb7700 1 -- 172.21.15.2:0/3357288598 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc00cd80 con 0x7efccc004850 2023-08-25T03:34:05.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.771+0000 7efce4fb7700 1 -- 172.21.15.2:0/3357288598 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcdc005960 con 0x7efccc004850 2023-08-25T03:34:05.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.771+0000 7efce4fb7700 1 -- 172.21.15.2:0/3357288598 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcdc005d10 con 0x7efccc004850 2023-08-25T03:34:05.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.771+0000 7efce821d700 1 -- 172.21.15.2:0/3357288598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efccc004850 msgr2=0x7efccc004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:05.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.771+0000 7efce821d700 1 --2- 172.21.15.2:0/3357288598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efccc004850 0x7efccc004cd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efcdc00c240 tx=0x7efccc005880 comp rx=0 tx=0).stop 2023-08-25T03:34:05.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.771+0000 7efce821d700 1 -- 172.21.15.2:0/3357288598 shutdown_connections 2023-08-25T03:34:05.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.771+0000 7efce821d700 1 --2- 172.21.15.2:0/3357288598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efccc004850 0x7efccc004cd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:05.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.771+0000 7efce821d700 1 -- 172.21.15.2:0/3357288598 >> 172.21.15.2:0/3357288598 conn(0x7efce00cfd90 msgr2=0x7efce00d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:05.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.771+0000 7efce821d700 1 -- 172.21.15.2:0/3357288598 shutdown_connections 2023-08-25T03:34:05.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.771+0000 7efce821d700 1 -- 172.21.15.2:0/3357288598 wait complete. 2023-08-25T03:34:05.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.773+0000 7efce821d700 1 Processor -- start 2023-08-25T03:34:05.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.773+0000 7efce821d700 1 -- start start 2023-08-25T03:34:05.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.773+0000 7efce821d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efccc004850 0x7efce014ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:05.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.773+0000 7efce821d700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efce00c7e00 0x7efce00c5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:05.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.773+0000 7efce821d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efce00cacf0 0x7efce015f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:05.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.773+0000 7efce821d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce00c8f40 con 0x7efce00cacf0 2023-08-25T03:34:05.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.773+0000 7efce821d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efce00c82c0 con 0x7efccc004850 2023-08-25T03:34:05.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.773+0000 7efce821d700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efce00c85a0 con 0x7efce00c7e00 2023-08-25T03:34:05.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.773+0000 7efce67ba700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efce00cacf0 0x7efce015f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:05.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.773+0000 7efce67ba700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efce00cacf0 0x7efce015f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59170/0 (socket says 172.21.15.2:59170) 2023-08-25T03:34:05.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.773+0000 7efce67ba700 1 -- 172.21.15.2:0/3278531975 learned_addr learned my addr 172.21.15.2:0/3278531975 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:05.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.774+0000 7efce5fb9700 1 --2- 172.21.15.2:0/3278531975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efccc004850 0x7efce014ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:05.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.774+0000 7efce57b8700 1 --2- 172.21.15.2:0/3278531975 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efce00c7e00 0x7efce00c5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:05.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.774+0000 7efce67ba700 1 -- 172.21.15.2:0/3278531975 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efce00c7e00 msgr2=0x7efce00c5970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:05.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.774+0000 7efce67ba700 1 --2- 172.21.15.2:0/3278531975 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efce00c7e00 0x7efce00c5970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:05.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.774+0000 7efce67ba700 1 -- 172.21.15.2:0/3278531975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efccc004850 msgr2=0x7efce014ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:05.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.774+0000 7efce67ba700 1 --2- 172.21.15.2:0/3278531975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efccc004850 0x7efce014ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:05.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.774+0000 7efce67ba700 1 -- 172.21.15.2:0/3278531975 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efce00c8880 con 0x7efce00cacf0 2023-08-25T03:34:05.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.774+0000 7efce67ba700 1 --2- 172.21.15.2:0/3278531975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efce00cacf0 0x7efce015f700 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efce006cbd0 tx=0x7efcd800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:05.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.774+0000 7efcd67fc700 1 -- 172.21.15.2:0/3278531975 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd800fed0 con 0x7efce00cacf0 2023-08-25T03:34:05.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.774+0000 7efcd67fc700 1 -- 172.21.15.2:0/3278531975 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd8003c60 con 0x7efce00cacf0 2023-08-25T03:34:05.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.774+0000 7efce821d700 1 -- 172.21.15.2:0/3278531975 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efce00c8b60 con 0x7efce00cacf0 2023-08-25T03:34:05.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.774+0000 7efcd67fc700 1 -- 172.21.15.2:0/3278531975 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd800dbb0 con 0x7efce00cacf0 2023-08-25T03:34:05.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.774+0000 7efce821d700 1 -- 172.21.15.2:0/3278531975 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efce00c5eb0 con 0x7efce00cacf0 2023-08-25T03:34:05.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.775+0000 7efce821d700 1 -- 172.21.15.2:0/3278531975 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efce00c60d0 con 0x7efce00cacf0 2023-08-25T03:34:05.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.776+0000 7efcd67fc700 1 -- 172.21.15.2:0/3278531975 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7efcd800dd10 con 0x7efce00cacf0 2023-08-25T03:34:05.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.776+0000 7efcd67fc700 1 --2- 172.21.15.2:0/3278531975 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7efcc406a0e0 0x7efcc406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:05.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.776+0000 7efce5fb9700 1 --2- 172.21.15.2:0/3278531975 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7efcc406a0e0 0x7efcc406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:05.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.776+0000 7efcd67fc700 1 -- 172.21.15.2:0/3278531975 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efcd8088640 con 0x7efce00cacf0 2023-08-25T03:34:05.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.777+0000 7efce5fb9700 1 --2- 172.21.15.2:0/3278531975 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7efcc406a0e0 0x7efcc406c5a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009b80 tx=0x7efcd0009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:05.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:05.781+0000 7efcd67fc700 1 -- 172.21.15.2:0/3278531975 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7efcd8053dc0 con 0x7efce00cacf0 2023-08-25T03:34:06.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.029+0000 7efce821d700 1 -- 172.21.15.2:0/3278531975 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7efce00746d0 con 0x7efce00cacf0 2023-08-25T03:34:06.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.618+0000 7efcd67fc700 1 -- 172.21.15.2:0/3278531975 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7efcd80573e0 con 0x7efce00cacf0 2023-08-25T03:34:06.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.621+0000 7efce821d700 1 -- 172.21.15.2:0/3278531975 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7efcc406a0e0 msgr2=0x7efcc406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:06.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.621+0000 7efce821d700 1 --2- 172.21.15.2:0/3278531975 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7efcc406a0e0 0x7efcc406c5a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009b80 tx=0x7efcd0009210 comp rx=0 tx=0).stop 2023-08-25T03:34:06.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.621+0000 7efce821d700 1 -- 172.21.15.2:0/3278531975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efce00cacf0 msgr2=0x7efce015f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:06.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.621+0000 7efce821d700 1 --2- 172.21.15.2:0/3278531975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efce00cacf0 0x7efce015f700 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efce006cbd0 tx=0x7efcd800d430 comp rx=0 tx=0).stop 2023-08-25T03:34:06.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.621+0000 7efce821d700 1 -- 172.21.15.2:0/3278531975 shutdown_connections 2023-08-25T03:34:06.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.621+0000 7efce821d700 1 --2- 172.21.15.2:0/3278531975 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7efcc406a0e0 0x7efcc406c5a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:06.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.621+0000 7efce821d700 1 --2- 172.21.15.2:0/3278531975 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efce00cacf0 0x7efce015f700 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:06.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.621+0000 7efce821d700 1 --2- 172.21.15.2:0/3278531975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efccc004850 0x7efce014ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:06.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.621+0000 7efce821d700 1 --2- 172.21.15.2:0/3278531975 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efce00c7e00 0x7efce00c5970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:06.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.621+0000 7efce821d700 1 -- 172.21.15.2:0/3278531975 >> 172.21.15.2:0/3278531975 conn(0x7efce00cfd90 msgr2=0x7efce00adfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:06.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.621+0000 7efce821d700 1 -- 172.21.15.2:0/3278531975 shutdown_connections 2023-08-25T03:34:06.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.621+0000 7efce821d700 1 -- 172.21.15.2:0/3278531975 wait complete. 2023-08-25T03:34:06.637 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-25T03:34:06.637 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T03:34:06.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.769+0000 7fa0c7a1d700 1 Processor -- start 2023-08-25T03:34:06.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.769+0000 7fa0c7a1d700 1 -- start start 2023-08-25T03:34:06.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.770+0000 7fa0c7a1d700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa0c00cdb70 0x7fa0c006b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:06.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.770+0000 7fa0c7a1d700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa0c00ce000 0x7fa0c006be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:06.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.770+0000 7fa0c7a1d700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c0161620 con 0x7fa0c00b1fb0 2023-08-25T03:34:06.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.770+0000 7fa0c7a1d700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c006d450 con 0x7fa0c00cdb70 2023-08-25T03:34:06.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.770+0000 7fa0c7a1d700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c006d730 con 0x7fa0c00ce000 2023-08-25T03:34:06.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.770+0000 7fa0c4fb8700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa0c00cdb70 0x7fa0c006b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:06.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.770+0000 7fa0c4fb8700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa0c00cdb70 0x7fa0c006b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59176/0 (socket says 172.21.15.2:59176) 2023-08-25T03:34:06.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.770+0000 7fa0c4fb8700 1 -- 172.21.15.2:0/2451129834 learned_addr learned my addr 172.21.15.2:0/2451129834 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:06.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.770+0000 7fa0c57b9700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fa0c00b1fb0 0x7fa0c0160f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45840/0 (socket says 172.21.15.2:45840) 2023-08-25T03:34:06.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.770+0000 7fa0c5fba700 1 --2- 172.21.15.2:0/2451129834 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa0c00ce000 0x7fa0c006be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:06.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.770+0000 7fa0c4fb8700 1 -- 172.21.15.2:0/2451129834 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa0c00ce000 msgr2=0x7fa0c006be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:06.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.770+0000 7fa0c4fb8700 1 --2- 172.21.15.2:0/2451129834 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa0c00ce000 0x7fa0c006be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:06.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.770+0000 7fa0c4fb8700 1 -- 172.21.15.2:0/2451129834 >> v1:172.21.15.2:6789/0 conn(0x7fa0c00b1fb0 legacy=0x7fa0c0160f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:06.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.770+0000 7fa0b7fff700 1 -- 172.21.15.2:0/2451129834 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2093814224 0 0) 0x7fa0c0161620 con 0x7fa0c00b1fb0 2023-08-25T03:34:06.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.770+0000 7fa0c4fb8700 1 -- 172.21.15.2:0/2451129834 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c006da10 con 0x7fa0c00cdb70 2023-08-25T03:34:06.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.770+0000 7fa0c5fba700 1 --2- 172.21.15.2:0/2451129834 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa0c00ce000 0x7fa0c006be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:06.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.770+0000 7fa0c4fb8700 1 --2- 172.21.15.2:0/2451129834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0c00cdb70 0x7fa0c006b8c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa0b00098f0 tx=0x7fa0b0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3562c3ad78af12b6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:06.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.770+0000 7fa0b7fff700 1 -- 172.21.15.2:0/2451129834 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b000b8a0 con 0x7fa0c00cdb70 2023-08-25T03:34:06.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.770+0000 7fa0b7fff700 1 -- 172.21.15.2:0/2451129834 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b000ba00 con 0x7fa0c00cdb70 2023-08-25T03:34:06.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.771+0000 7fa0b7fff700 1 -- 172.21.15.2:0/2451129834 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b000bd90 con 0x7fa0c00cdb70 2023-08-25T03:34:06.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.771+0000 7fa0c7a1d700 1 -- 172.21.15.2:0/2451129834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0c00cdb70 msgr2=0x7fa0c006b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:06.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.771+0000 7fa0c7a1d700 1 --2- 172.21.15.2:0/2451129834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0c00cdb70 0x7fa0c006b8c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa0b00098f0 tx=0x7fa0b0009c00 comp rx=0 tx=0).stop 2023-08-25T03:34:06.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.771+0000 7fa0c7a1d700 1 -- 172.21.15.2:0/2451129834 shutdown_connections 2023-08-25T03:34:06.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.771+0000 7fa0c7a1d700 1 --2- 172.21.15.2:0/2451129834 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa0c00ce000 0x7fa0c006be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:06.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.771+0000 7fa0c7a1d700 1 --2- 172.21.15.2:0/2451129834 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0c00cdb70 0x7fa0c006b8c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:06.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.771+0000 7fa0c7a1d700 1 -- 172.21.15.2:0/2451129834 >> 172.21.15.2:0/2451129834 conn(0x7fa0c00cfea0 msgr2=0x7fa0c00d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:06.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.771+0000 7fa0c7a1d700 1 -- 172.21.15.2:0/2451129834 shutdown_connections 2023-08-25T03:34:06.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.771+0000 7fa0c7a1d700 1 -- 172.21.15.2:0/2451129834 wait complete. 2023-08-25T03:34:06.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.772+0000 7fa0c7a1d700 1 Processor -- start 2023-08-25T03:34:06.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.772+0000 7fa0c7a1d700 1 -- start start 2023-08-25T03:34:06.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.773+0000 7fa0c7a1d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c00b1fb0 0x7fa0c00c7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:06.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.773+0000 7fa0c7a1d700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa0c00cdb70 0x7fa0c00b4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:06.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.773+0000 7fa0c7a1d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0c00ce000 0x7fa0c01f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:06.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.773+0000 7fa0c7a1d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c01f4050 con 0x7fa0c00ce000 2023-08-25T03:34:06.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.773+0000 7fa0c7a1d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c01f3a90 con 0x7fa0c00b1fb0 2023-08-25T03:34:06.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.773+0000 7fa0c7a1d700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c01f3d70 con 0x7fa0c00cdb70 2023-08-25T03:34:06.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.773+0000 7fa0c5fba700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0c00ce000 0x7fa0c01f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:06.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.773+0000 7fa0c5fba700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0c00ce000 0x7fa0c01f30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59184/0 (socket says 172.21.15.2:59184) 2023-08-25T03:34:06.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.773+0000 7fa0c5fba700 1 -- 172.21.15.2:0/3114318728 learned_addr learned my addr 172.21.15.2:0/3114318728 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:06.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.773+0000 7fa0c4fb8700 1 --2- 172.21.15.2:0/3114318728 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa0c00cdb70 0x7fa0c00b4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:06.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.773+0000 7fa0c57b9700 1 --2- 172.21.15.2:0/3114318728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c00b1fb0 0x7fa0c00c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:06.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.774+0000 7fa0c5fba700 1 -- 172.21.15.2:0/3114318728 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa0c00cdb70 msgr2=0x7fa0c00b4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:06.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.774+0000 7fa0c5fba700 1 --2- 172.21.15.2:0/3114318728 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa0c00cdb70 0x7fa0c00b4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:06.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.774+0000 7fa0c5fba700 1 -- 172.21.15.2:0/3114318728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c00b1fb0 msgr2=0x7fa0c00c7880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:06.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.774+0000 7fa0c5fba700 1 --2- 172.21.15.2:0/3114318728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c00b1fb0 0x7fa0c00c7880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:06.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.774+0000 7fa0c5fba700 1 -- 172.21.15.2:0/3114318728 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b0009550 con 0x7fa0c00ce000 2023-08-25T03:34:06.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.774+0000 7fa0c5fba700 1 --2- 172.21.15.2:0/3114318728 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0c00ce000 0x7fa0c01f30d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800cef0 tx=0x7fa0b800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:06.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.774+0000 7fa0b5ffb700 1 -- 172.21.15.2:0/3114318728 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b800ad00 con 0x7fa0c00ce000 2023-08-25T03:34:06.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.774+0000 7fa0b5ffb700 1 -- 172.21.15.2:0/3114318728 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b800ae60 con 0x7fa0c00ce000 2023-08-25T03:34:06.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.774+0000 7fa0c7a1d700 1 -- 172.21.15.2:0/3114318728 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c00c7080 con 0x7fa0c00ce000 2023-08-25T03:34:06.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.774+0000 7fa0b5ffb700 1 -- 172.21.15.2:0/3114318728 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b8015680 con 0x7fa0c00ce000 2023-08-25T03:34:06.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.774+0000 7fa0c7a1d700 1 -- 172.21.15.2:0/3114318728 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c00c7dc0 con 0x7fa0c00ce000 2023-08-25T03:34:06.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.775+0000 7fa0c7a1d700 1 -- 172.21.15.2:0/3114318728 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c0066f70 con 0x7fa0c00ce000 2023-08-25T03:34:06.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.776+0000 7fa0b5ffb700 1 -- 172.21.15.2:0/3114318728 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa0b800d430 con 0x7fa0c00ce000 2023-08-25T03:34:06.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.776+0000 7fa0b5ffb700 1 --2- 172.21.15.2:0/3114318728 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa0a406a0e0 0x7fa0a406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:06.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.776+0000 7fa0b5ffb700 1 -- 172.21.15.2:0/3114318728 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa0b8088950 con 0x7fa0c00ce000 2023-08-25T03:34:06.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.776+0000 7fa0c57b9700 1 --2- 172.21.15.2:0/3114318728 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa0a406a0e0 0x7fa0a406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:06.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.776+0000 7fa0c57b9700 1 --2- 172.21.15.2:0/3114318728 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa0a406a0e0 0x7fa0a406c5a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc000f60 tx=0x7fa0bc007e70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:06.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:06.780+0000 7fa0b5ffb700 1 -- 172.21.15.2:0/3114318728 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa0b80540d0 con 0x7fa0c00ce000 2023-08-25T03:34:07.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.028+0000 7fa0c7a1d700 1 -- 172.21.15.2:0/3114318728 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0c00c6ee0 con 0x7fa0c00ce000 2023-08-25T03:34:07.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.028+0000 7fa0b5ffb700 1 -- 172.21.15.2:0/3114318728 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4857 (secure 0 0 0) 0x7fa0c00c6ee0 con 0x7fa0c00ce000 2023-08-25T03:34:07.029 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:07.029 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2074312983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2074312983},{"type":"v1","addr":"172.21.15.2:6837","nonce":2074312983}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4059180922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4059180922},{"type":"v1","addr":"172.21.15.31:6835","nonce":4059180922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/922384230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":922384230},{"type":"v1","addr":"172.21.15.31:6834","nonce":922384230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T03:34:03.009109+0000","modified":"2023-08-25T03:34:06.613200+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.2:6835/193990892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":193990892},{"type":"v1","addr":"172.21.15.2:6835","nonce":193990892}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T03:34:07.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.030+0000 7fa0c7a1d700 1 -- 172.21.15.2:0/3114318728 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa0a406a0e0 msgr2=0x7fa0a406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:07.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.030+0000 7fa0c7a1d700 1 --2- 172.21.15.2:0/3114318728 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa0a406a0e0 0x7fa0a406c5a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc000f60 tx=0x7fa0bc007e70 comp rx=0 tx=0).stop 2023-08-25T03:34:07.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.030+0000 7fa0c7a1d700 1 -- 172.21.15.2:0/3114318728 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0c00ce000 msgr2=0x7fa0c01f30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:07.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.030+0000 7fa0c7a1d700 1 --2- 172.21.15.2:0/3114318728 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0c00ce000 0x7fa0c01f30d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa0b800cef0 tx=0x7fa0b800cf20 comp rx=0 tx=0).stop 2023-08-25T03:34:07.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.030+0000 7fa0c7a1d700 1 -- 172.21.15.2:0/3114318728 shutdown_connections 2023-08-25T03:34:07.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.030+0000 7fa0c7a1d700 1 --2- 172.21.15.2:0/3114318728 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa0a406a0e0 0x7fa0a406c5a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.030+0000 7fa0c7a1d700 1 --2- 172.21.15.2:0/3114318728 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0c00ce000 0x7fa0c01f30d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.030+0000 7fa0c7a1d700 1 --2- 172.21.15.2:0/3114318728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c00b1fb0 0x7fa0c00c7880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.030+0000 7fa0c7a1d700 1 --2- 172.21.15.2:0/3114318728 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa0c00cdb70 0x7fa0c00b4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.030+0000 7fa0c7a1d700 1 -- 172.21.15.2:0/3114318728 >> 172.21.15.2:0/3114318728 conn(0x7fa0c00cfea0 msgr2=0x7fa0c015cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:07.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.030+0000 7fa0c7a1d700 1 -- 172.21.15.2:0/3114318728 shutdown_connections 2023-08-25T03:34:07.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.030+0000 7fa0c7a1d700 1 -- 172.21.15.2:0/3114318728 wait complete. 2023-08-25T03:34:07.033 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 8 2023-08-25T03:34:07.043 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:34:07.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.166+0000 7f70bb731700 1 Processor -- start 2023-08-25T03:34:07.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.166+0000 7f70bb731700 1 -- start start 2023-08-25T03:34:07.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.167+0000 7f70bb731700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f70b415d1f0 0x7f70b40ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:07.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.167+0000 7f70bb731700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f70b40d0720 0x7f70b406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:07.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.167+0000 7f70bb731700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f70b406ce10 0x7f70b406b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:07.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.167+0000 7f70bb731700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b40d0bb0 con 0x7f70b40d0720 2023-08-25T03:34:07.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.167+0000 7f70bb731700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70b406d240 con 0x7f70b415d1f0 2023-08-25T03:34:07.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.167+0000 7f70bb731700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b40cf120 con 0x7f70b406ce10 2023-08-25T03:34:07.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.167+0000 7f70b8ccc700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f70b40d0720 0x7f70b406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:07.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.167+0000 7f70b8ccc700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f70b40d0720 0x7f70b406c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59188/0 (socket says 172.21.15.2:59188) 2023-08-25T03:34:07.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.167+0000 7f70b8ccc700 1 -- 172.21.15.2:0/451036271 learned_addr learned my addr 172.21.15.2:0/451036271 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:07.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.167+0000 7f70b9cce700 1 --2- 172.21.15.2:0/451036271 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f70b406ce10 0x7f70b406b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:07.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.167+0000 7f70b94cd700 1 --2- 172.21.15.2:0/451036271 >> v2:172.21.15.31:3300/0 conn(0x7f70b415d1f0 0x7f70b40ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:07.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.168+0000 7f70b8ccc700 1 -- 172.21.15.2:0/451036271 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f70b406ce10 msgr2=0x7f70b406b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:07.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.168+0000 7f70b8ccc700 1 --2- 172.21.15.2:0/451036271 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f70b406ce10 0x7f70b406b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.168+0000 7f70b8ccc700 1 -- 172.21.15.2:0/451036271 >> v2:172.21.15.31:3300/0 conn(0x7f70b415d1f0 msgr2=0x7f70b40ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:07.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.168+0000 7f70b8ccc700 1 --2- 172.21.15.2:0/451036271 >> v2:172.21.15.31:3300/0 conn(0x7f70b415d1f0 0x7f70b40ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.168+0000 7f70b8ccc700 1 -- 172.21.15.2:0/451036271 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b40b29f0 con 0x7f70b40d0720 2023-08-25T03:34:07.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.168+0000 7f70b94cd700 1 --2- 172.21.15.2:0/451036271 >> v2:172.21.15.31:3300/0 conn(0x7f70b415d1f0 0x7f70b40ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:07.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.168+0000 7f70b8ccc700 1 --2- 172.21.15.2:0/451036271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b40d0720 0x7f70b406c8d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f70b406fe80 tx=0x7f70a4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c378d1ebd45b8991 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:07.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.168+0000 7f70abfff700 1 -- 172.21.15.2:0/451036271 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a4011830 con 0x7f70b40d0720 2023-08-25T03:34:07.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.168+0000 7f70abfff700 1 -- 172.21.15.2:0/451036271 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70a4011990 con 0x7f70b40d0720 2023-08-25T03:34:07.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.168+0000 7f70abfff700 1 -- 172.21.15.2:0/451036271 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70a4011d20 con 0x7f70b40d0720 2023-08-25T03:34:07.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.168+0000 7f70bb731700 1 -- 172.21.15.2:0/451036271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b40d0720 msgr2=0x7f70b406c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:07.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.168+0000 7f70bb731700 1 --2- 172.21.15.2:0/451036271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b40d0720 0x7f70b406c8d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f70b406fe80 tx=0x7f70a4009bd0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.169+0000 7f70bb731700 1 -- 172.21.15.2:0/451036271 shutdown_connections 2023-08-25T03:34:07.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.169+0000 7f70bb731700 1 --2- 172.21.15.2:0/451036271 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f70b406ce10 0x7f70b406b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.169+0000 7f70bb731700 1 --2- 172.21.15.2:0/451036271 >> v2:172.21.15.31:3300/0 conn(0x7f70b415d1f0 0x7f70b40ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.169+0000 7f70bb731700 1 --2- 172.21.15.2:0/451036271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b40d0720 0x7f70b406c8d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.169+0000 7f70bb731700 1 -- 172.21.15.2:0/451036271 >> 172.21.15.2:0/451036271 conn(0x7f70b40d0f70 msgr2=0x7f70b40d1350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:07.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.169+0000 7f70bb731700 1 -- 172.21.15.2:0/451036271 shutdown_connections 2023-08-25T03:34:07.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.169+0000 7f70bb731700 1 -- 172.21.15.2:0/451036271 wait complete. 2023-08-25T03:34:07.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.170+0000 7f70bb731700 1 Processor -- start 2023-08-25T03:34:07.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.170+0000 7f70bb731700 1 -- start start 2023-08-25T03:34:07.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.170+0000 7f70bb731700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b406ce10 0x7f70b4151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:07.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.171+0000 7f70bb731700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f70b40d0720 0x7f70b41582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:07.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.171+0000 7f70bb731700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70b415d1f0 0x7f70b4158820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:07.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.171+0000 7f70bb731700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b40ac8e0 con 0x7f70b406ce10 2023-08-25T03:34:07.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.171+0000 7f70bb731700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70b40ad180 con 0x7f70b415d1f0 2023-08-25T03:34:07.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.171+0000 7f70bb731700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70b41f7000 con 0x7f70b40d0720 2023-08-25T03:34:07.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.171+0000 7f70b94cd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b406ce10 0x7f70b4151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:07.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.171+0000 7f70b94cd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b406ce10 0x7f70b4151480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59192/0 (socket says 172.21.15.2:59192) 2023-08-25T03:34:07.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.171+0000 7f70b94cd700 1 -- 172.21.15.2:0/723636264 learned_addr learned my addr 172.21.15.2:0/723636264 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:07.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.171+0000 7f70b8ccc700 1 --2- 172.21.15.2:0/723636264 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f70b40d0720 0x7f70b41582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:07.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.171+0000 7f70b94cd700 1 -- 172.21.15.2:0/723636264 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f70b40d0720 msgr2=0x7f70b41582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:07.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.171+0000 7f70b9cce700 1 --2- 172.21.15.2:0/723636264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70b415d1f0 0x7f70b4158820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:07.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.171+0000 7f70b94cd700 1 --2- 172.21.15.2:0/723636264 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f70b40d0720 0x7f70b41582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.171+0000 7f70b94cd700 1 -- 172.21.15.2:0/723636264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70b415d1f0 msgr2=0x7f70b4158820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:07.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.171+0000 7f70b94cd700 1 --2- 172.21.15.2:0/723636264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70b415d1f0 0x7f70b4158820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.171+0000 7f70b94cd700 1 -- 172.21.15.2:0/723636264 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a4009550 con 0x7f70b406ce10 2023-08-25T03:34:07.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.171+0000 7f70b8ccc700 1 --2- 172.21.15.2:0/723636264 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f70b40d0720 0x7f70b41582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:07.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.171+0000 7f70b9cce700 1 --2- 172.21.15.2:0/723636264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70b415d1f0 0x7f70b4158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:07.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.171+0000 7f70b94cd700 1 --2- 172.21.15.2:0/723636264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b406ce10 0x7f70b4151480 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f70ac007fa0 tx=0x7f70ac00d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:07.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.171+0000 7f70a9ffb700 1 -- 172.21.15.2:0/723636264 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac003a20 con 0x7f70b406ce10 2023-08-25T03:34:07.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.171+0000 7f70a9ffb700 1 -- 172.21.15.2:0/723636264 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ac00f040 con 0x7f70b406ce10 2023-08-25T03:34:07.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.171+0000 7f70a9ffb700 1 -- 172.21.15.2:0/723636264 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ac003da0 con 0x7f70b406ce10 2023-08-25T03:34:07.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.171+0000 7f70bb731700 1 -- 172.21.15.2:0/723636264 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70b40acb30 con 0x7f70b406ce10 2023-08-25T03:34:07.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.172+0000 7f70bb731700 1 -- 172.21.15.2:0/723636264 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70b41519c0 con 0x7f70b406ce10 2023-08-25T03:34:07.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.172+0000 7f70bb731700 1 -- 172.21.15.2:0/723636264 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70b40746d0 con 0x7f70b406ce10 2023-08-25T03:34:07.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.173+0000 7f70a9ffb700 1 -- 172.21.15.2:0/723636264 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f70ac004030 con 0x7f70b406ce10 2023-08-25T03:34:07.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.173+0000 7f70a9ffb700 1 --2- 172.21.15.2:0/723636264 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f709806a0e0 0x7f709806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:07.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.173+0000 7f70a9ffb700 1 -- 172.21.15.2:0/723636264 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f70ac0885a0 con 0x7f70b406ce10 2023-08-25T03:34:07.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.173+0000 7f70b8ccc700 1 --2- 172.21.15.2:0/723636264 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f709806a0e0 0x7f709806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:07.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.174+0000 7f70b8ccc700 1 --2- 172.21.15.2:0/723636264 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f709806a0e0 0x7f709806c5a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f70b40c4330 tx=0x7f70a4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:07.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.178+0000 7f70a9ffb700 1 -- 172.21.15.2:0/723636264 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f70ac053dd0 con 0x7f70b406ce10 2023-08-25T03:34:07.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.379+0000 7f70bb731700 1 -- 172.21.15.2:0/723636264 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70b4070150 con 0x7f70b406ce10 2023-08-25T03:34:07.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.381+0000 7f70a9ffb700 1 -- 172.21.15.2:0/723636264 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18135 (secure 0 0 0) 0x7f70ac0573f0 con 0x7f70b406ce10 2023-08-25T03:34:07.382 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:07.383 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":24,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:34:04.604059+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:34:00.869118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T03:34:01.987016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:34:07.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.383+0000 7f70bb731700 1 -- 172.21.15.2:0/723636264 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f709806a0e0 msgr2=0x7f709806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:07.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.383+0000 7f70bb731700 1 --2- 172.21.15.2:0/723636264 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f709806a0e0 0x7f709806c5a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f70b40c4330 tx=0x7f70a4008030 comp rx=0 tx=0).stop 2023-08-25T03:34:07.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.383+0000 7f70bb731700 1 -- 172.21.15.2:0/723636264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b406ce10 msgr2=0x7f70b4151480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:07.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.383+0000 7f70bb731700 1 --2- 172.21.15.2:0/723636264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b406ce10 0x7f70b4151480 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f70ac007fa0 tx=0x7f70ac00d460 comp rx=0 tx=0).stop 2023-08-25T03:34:07.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.383+0000 7f70bb731700 1 -- 172.21.15.2:0/723636264 shutdown_connections 2023-08-25T03:34:07.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.383+0000 7f70bb731700 1 --2- 172.21.15.2:0/723636264 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f709806a0e0 0x7f709806c5a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.383+0000 7f70bb731700 1 --2- 172.21.15.2:0/723636264 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f70b406ce10 0x7f70b4151480 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.383+0000 7f70bb731700 1 --2- 172.21.15.2:0/723636264 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f70b415d1f0 0x7f70b4158820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.383+0000 7f70bb731700 1 --2- 172.21.15.2:0/723636264 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f70b40d0720 0x7f70b41582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.383+0000 7f70bb731700 1 -- 172.21.15.2:0/723636264 >> 172.21.15.2:0/723636264 conn(0x7f70b40d0f70 msgr2=0x7f70b40b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:07.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.383+0000 7f70bb731700 1 -- 172.21.15.2:0/723636264 shutdown_connections 2023-08-25T03:34:07.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.383+0000 7f70bb731700 1 -- 172.21.15.2:0/723636264 wait complete. 2023-08-25T03:34:07.397 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-25T03:34:07.398 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T03:34:07.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.523+0000 7f240be69700 1 Processor -- start 2023-08-25T03:34:07.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.523+0000 7f240be69700 1 -- start start 2023-08-25T03:34:07.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.524+0000 7f240be69700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f24040b1fb0 0x7f2404160f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:07.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.524+0000 7f240be69700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f240406b8c0 0x7f240406bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:07.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.524+0000 7f240be69700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f240406c240 con 0x7f2404161450 2023-08-25T03:34:07.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.524+0000 7f240be69700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f240406d450 con 0x7f24040b1fb0 2023-08-25T03:34:07.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.524+0000 7f240be69700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f240406d730 con 0x7f240406b8c0 2023-08-25T03:34:07.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.524+0000 7f2409c05700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f24040b1fb0 0x7f2404160f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:07.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.524+0000 7f2409c05700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f24040b1fb0 0x7f2404160f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59200/0 (socket says 172.21.15.2:59200) 2023-08-25T03:34:07.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.524+0000 7f2409c05700 1 -- 172.21.15.2:0/2718279627 learned_addr learned my addr 172.21.15.2:0/2718279627 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:07.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.524+0000 7f240a406700 1 --2- 172.21.15.2:0/2718279627 >> v2:172.21.15.31:3300/0 conn(0x7f240406b8c0 0x7f240406bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:07.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.524+0000 7f2409c05700 1 -- 172.21.15.2:0/2718279627 >> v2:172.21.15.31:3300/0 conn(0x7f240406b8c0 msgr2=0x7f240406bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:07.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.524+0000 7f2409c05700 1 --2- 172.21.15.2:0/2718279627 >> v2:172.21.15.31:3300/0 conn(0x7f240406b8c0 0x7f240406bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.524+0000 7f2409c05700 1 -- 172.21.15.2:0/2718279627 >> v1:172.21.15.31:6789/0 conn(0x7f2404161450 legacy=0x7f24040cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:07.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.524+0000 7f2409c05700 1 -- 172.21.15.2:0/2718279627 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f240406da10 con 0x7f24040b1fb0 2023-08-25T03:34:07.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.525+0000 7f2409c05700 1 --2- 172.21.15.2:0/2718279627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24040b1fb0 0x7f2404160f10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f24040b2aa0 tx=0x7f23f4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30de0b50bffa6dd7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:07.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.525+0000 7f2408c03700 1 -- 172.21.15.2:0/2718279627 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f4011830 con 0x7f24040b1fb0 2023-08-25T03:34:07.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.525+0000 7f2408c03700 1 -- 172.21.15.2:0/2718279627 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f4011990 con 0x7f24040b1fb0 2023-08-25T03:34:07.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.525+0000 7f2408c03700 1 -- 172.21.15.2:0/2718279627 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f4011d20 con 0x7f24040b1fb0 2023-08-25T03:34:07.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.525+0000 7f240be69700 1 -- 172.21.15.2:0/2718279627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24040b1fb0 msgr2=0x7f2404160f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:07.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.525+0000 7f240be69700 1 --2- 172.21.15.2:0/2718279627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24040b1fb0 0x7f2404160f10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f24040b2aa0 tx=0x7f23f4009bd0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.525+0000 7f240be69700 1 -- 172.21.15.2:0/2718279627 shutdown_connections 2023-08-25T03:34:07.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.525+0000 7f240be69700 1 --2- 172.21.15.2:0/2718279627 >> v2:172.21.15.31:3300/0 conn(0x7f240406b8c0 0x7f240406bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.525+0000 7f240be69700 1 --2- 172.21.15.2:0/2718279627 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24040b1fb0 0x7f2404160f10 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.525+0000 7f240be69700 1 -- 172.21.15.2:0/2718279627 >> 172.21.15.2:0/2718279627 conn(0x7f24040cfea0 msgr2=0x7f24040d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:07.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.525+0000 7f240be69700 1 -- 172.21.15.2:0/2718279627 shutdown_connections 2023-08-25T03:34:07.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.526+0000 7f240be69700 1 -- 172.21.15.2:0/2718279627 wait complete. 2023-08-25T03:34:07.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.527+0000 7f240be69700 1 Processor -- start 2023-08-25T03:34:07.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.527+0000 7f240be69700 1 -- start start 2023-08-25T03:34:07.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.527+0000 7f240be69700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f240406b8c0 0x7f24041f61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:07.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.527+0000 7f240be69700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24041f6730 0x7f2404149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:07.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.527+0000 7f240be69700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2404149c60 0x7f24041f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:07.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.527+0000 7f240be69700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24040b2ad0 con 0x7f240406b8c0 2023-08-25T03:34:07.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.527+0000 7f240be69700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24041f6bb0 con 0x7f24041f6730 2023-08-25T03:34:07.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.527+0000 7f240be69700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f240414a490 con 0x7f2404149c60 2023-08-25T03:34:07.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.528+0000 7f2409c05700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f240406b8c0 0x7f24041f61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:07.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.528+0000 7f2409c05700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f240406b8c0 0x7f24041f61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59206/0 (socket says 172.21.15.2:59206) 2023-08-25T03:34:07.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.528+0000 7f2409c05700 1 -- 172.21.15.2:0/1807961779 learned_addr learned my addr 172.21.15.2:0/1807961779 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:07.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.528+0000 7f2409404700 1 --2- 172.21.15.2:0/1807961779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24041f6730 0x7f2404149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:07.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.528+0000 7f240a406700 1 --2- 172.21.15.2:0/1807961779 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2404149c60 0x7f24041f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:07.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.528+0000 7f2409c05700 1 -- 172.21.15.2:0/1807961779 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2404149c60 msgr2=0x7f24041f1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:07.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.528+0000 7f2409c05700 1 --2- 172.21.15.2:0/1807961779 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2404149c60 0x7f24041f1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.528+0000 7f2409c05700 1 -- 172.21.15.2:0/1807961779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24041f6730 msgr2=0x7f2404149720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:07.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.528+0000 7f2409c05700 1 --2- 172.21.15.2:0/1807961779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24041f6730 0x7f2404149720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.528+0000 7f2409c05700 1 -- 172.21.15.2:0/1807961779 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f4009550 con 0x7f240406b8c0 2023-08-25T03:34:07.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.528+0000 7f240a406700 1 --2- 172.21.15.2:0/1807961779 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2404149c60 0x7f24041f1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:07.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.528+0000 7f2409404700 1 --2- 172.21.15.2:0/1807961779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24041f6730 0x7f2404149720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:07.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.528+0000 7f2409c05700 1 --2- 172.21.15.2:0/1807961779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f240406b8c0 0x7f24041f61f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f23f4000c00 tx=0x7f23f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:07.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.528+0000 7f23fa7fc700 1 -- 172.21.15.2:0/1807961779 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f40044b0 con 0x7f240406b8c0 2023-08-25T03:34:07.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.528+0000 7f23fa7fc700 1 -- 172.21.15.2:0/1807961779 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f401f070 con 0x7f240406b8c0 2023-08-25T03:34:07.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.528+0000 7f240be69700 1 -- 172.21.15.2:0/1807961779 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f240414a770 con 0x7f240406b8c0 2023-08-25T03:34:07.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.528+0000 7f23fa7fc700 1 -- 172.21.15.2:0/1807961779 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f40038e0 con 0x7f240406b8c0 2023-08-25T03:34:07.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.528+0000 7f240be69700 1 -- 172.21.15.2:0/1807961779 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24040b2fc0 con 0x7f240406b8c0 2023-08-25T03:34:07.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.529+0000 7f240be69700 1 -- 172.21.15.2:0/1807961779 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2404066f70 con 0x7f240406b8c0 2023-08-25T03:34:07.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.530+0000 7f23fa7fc700 1 -- 172.21.15.2:0/1807961779 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f23f4021030 con 0x7f240406b8c0 2023-08-25T03:34:07.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.530+0000 7f23fa7fc700 1 --2- 172.21.15.2:0/1807961779 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f23e806a0e0 0x7f23e806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:07.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.530+0000 7f23fa7fc700 1 -- 172.21.15.2:0/1807961779 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f23f4089890 con 0x7f240406b8c0 2023-08-25T03:34:07.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.530+0000 7f2409404700 1 --2- 172.21.15.2:0/1807961779 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f23e806a0e0 0x7f23e806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:07.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.530+0000 7f2409404700 1 --2- 172.21.15.2:0/1807961779 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f23e806a0e0 0x7f23e806c5a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f240415d230 tx=0x7f2400009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:07.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.534+0000 7f23fa7fc700 1 -- 172.21.15.2:0/1807961779 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f23f4055130 con 0x7f240406b8c0 2023-08-25T03:34:07.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.731+0000 7f240be69700 1 -- 172.21.15.2:0/1807961779 --> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f24040b2390 con 0x7f23e806a0e0 2023-08-25T03:34:07.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.740+0000 7f23fa7fc700 1 -- 172.21.15.2:0/1807961779 <== mgr.4108 v2:172.21.15.2:6832/3528779779 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284114 (secure 0 0 0) 0x7f24000050d0 con 0x7f23e806a0e0 2023-08-25T03:34:07.744 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:07.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.743+0000 7f240be69700 1 -- 172.21.15.2:0/1807961779 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f23e806a0e0 msgr2=0x7f23e806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:07.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.743+0000 7f240be69700 1 --2- 172.21.15.2:0/1807961779 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f23e806a0e0 0x7f23e806c5a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f240415d230 tx=0x7f2400009040 comp rx=0 tx=0).stop 2023-08-25T03:34:07.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.743+0000 7f240be69700 1 -- 172.21.15.2:0/1807961779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f240406b8c0 msgr2=0x7f24041f61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:07.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.743+0000 7f240be69700 1 --2- 172.21.15.2:0/1807961779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f240406b8c0 0x7f24041f61f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f23f4000c00 tx=0x7f23f4004750 comp rx=0 tx=0).stop 2023-08-25T03:34:07.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.743+0000 7f240be69700 1 -- 172.21.15.2:0/1807961779 shutdown_connections 2023-08-25T03:34:07.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.743+0000 7f240be69700 1 --2- 172.21.15.2:0/1807961779 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f23e806a0e0 0x7f23e806c5a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.743+0000 7f240be69700 1 --2- 172.21.15.2:0/1807961779 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f240406b8c0 0x7f24041f61f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.743+0000 7f240be69700 1 --2- 172.21.15.2:0/1807961779 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2404149c60 0x7f24041f1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.743+0000 7f240be69700 1 --2- 172.21.15.2:0/1807961779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f24041f6730 0x7f2404149720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.744+0000 7f240be69700 1 -- 172.21.15.2:0/1807961779 >> 172.21.15.2:0/1807961779 conn(0x7f24040cfea0 msgr2=0x7f240415dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:07.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.744+0000 7f240be69700 1 -- 172.21.15.2:0/1807961779 shutdown_connections 2023-08-25T03:34:07.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.744+0000 7f240be69700 1 -- 172.21.15.2:0/1807961779 wait complete. 2023-08-25T03:34:07.748 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-08-25T03:34:07.756 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-08-25T03:34:07.504648+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":195,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219164,"kb_used_data":2972,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754755556,"statfs":{"total":773094113280,"available":772869689344,"internally_reserved":0,"allocated":3043328,"data_stored":1832772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.915745"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.584233+0000","last_change":"2023-08-25T03:34:02.584233+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.584233+0000","last_undegraded":"2023-08-25T03:34:02.584233+0000","last_fullsized":"2023-08-25T03:34:02.584233+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.620161+0000","last_change":"2023-08-25T03:34:03.616304+0000","last_active":"2023-08-25T03:34:03.620161+0000","last_peered":"2023-08-25T03:34:03.620161+0000","last_clean":"2023-08-25T03:34:03.620161+0000","last_became_active":"2023-08-25T03:34:03.615917+0000","last_became_peered":"2023-08-25T03:34:03.615917+0000","last_unstale":"2023-08-25T03:34:03.620161+0000","last_undegraded":"2023-08-25T03:34:03.620161+0000","last_fullsized":"2023-08-25T03:34:03.620161+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:25:43.507515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627607+0000","last_change":"2023-08-25T03:34:03.622799+0000","last_active":"2023-08-25T03:34:03.627607+0000","last_peered":"2023-08-25T03:34:03.627607+0000","last_clean":"2023-08-25T03:34:03.627607+0000","last_became_active":"2023-08-25T03:34:03.622602+0000","last_became_peered":"2023-08-25T03:34:03.622602+0000","last_unstale":"2023-08-25T03:34:03.627607+0000","last_undegraded":"2023-08-25T03:34:03.627607+0000","last_fullsized":"2023-08-25T03:34:03.627607+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:43:29.368622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.616911+0000","last_change":"2023-08-25T03:34:03.615254+0000","last_active":"2023-08-25T03:34:03.616911+0000","last_peered":"2023-08-25T03:34:03.616911+0000","last_clean":"2023-08-25T03:34:03.616911+0000","last_became_active":"2023-08-25T03:34:03.614917+0000","last_became_peered":"2023-08-25T03:34:03.614917+0000","last_unstale":"2023-08-25T03:34:03.616911+0000","last_undegraded":"2023-08-25T03:34:03.616911+0000","last_fullsized":"2023-08-25T03:34:03.616911+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:10:59.547904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.590306+0000","last_change":"2023-08-25T03:34:02.590306+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.590306+0000","last_undegraded":"2023-08-25T03:34:02.590306+0000","last_fullsized":"2023-08-25T03:34:02.590306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627201+0000","last_change":"2023-08-25T03:34:03.619323+0000","last_active":"2023-08-25T03:34:03.627201+0000","last_peered":"2023-08-25T03:34:03.627201+0000","last_clean":"2023-08-25T03:34:03.627201+0000","last_became_active":"2023-08-25T03:34:03.618985+0000","last_became_peered":"2023-08-25T03:34:03.618985+0000","last_unstale":"2023-08-25T03:34:03.627201+0000","last_undegraded":"2023-08-25T03:34:03.627201+0000","last_fullsized":"2023-08-25T03:34:03.627201+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:47:39.963816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.622126+0000","last_change":"2023-08-25T03:34:03.617493+0000","last_active":"2023-08-25T03:34:03.622126+0000","last_peered":"2023-08-25T03:34:03.622126+0000","last_clean":"2023-08-25T03:34:03.622126+0000","last_became_active":"2023-08-25T03:34:03.617167+0000","last_became_peered":"2023-08-25T03:34:03.617167+0000","last_unstale":"2023-08-25T03:34:03.622126+0000","last_undegraded":"2023-08-25T03:34:03.622126+0000","last_fullsized":"2023-08-25T03:34:03.622126+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:41:33.001585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.629535+0000","last_change":"2023-08-25T03:34:03.625710+0000","last_active":"2023-08-25T03:34:03.629535+0000","last_peered":"2023-08-25T03:34:03.629535+0000","last_clean":"2023-08-25T03:34:03.629535+0000","last_became_active":"2023-08-25T03:34:03.625404+0000","last_became_peered":"2023-08-25T03:34:03.625404+0000","last_unstale":"2023-08-25T03:34:03.629535+0000","last_undegraded":"2023-08-25T03:34:03.629535+0000","last_fullsized":"2023-08-25T03:34:03.629535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:15:37.568529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.593242+0000","last_change":"2023-08-25T03:34:02.593242+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.593242+0000","last_undegraded":"2023-08-25T03:34:02.593242+0000","last_fullsized":"2023-08-25T03:34:02.593242+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.620872+0000","last_change":"2023-08-25T03:34:03.618247+0000","last_active":"2023-08-25T03:34:03.620872+0000","last_peered":"2023-08-25T03:34:03.620872+0000","last_clean":"2023-08-25T03:34:03.620872+0000","last_became_active":"2023-08-25T03:34:03.617830+0000","last_became_peered":"2023-08-25T03:34:03.617830+0000","last_unstale":"2023-08-25T03:34:03.620872+0000","last_undegraded":"2023-08-25T03:34:03.620872+0000","last_fullsized":"2023-08-25T03:34:03.620872+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:47:39.963816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627422+0000","last_change":"2023-08-25T03:34:03.622380+0000","last_active":"2023-08-25T03:34:03.627422+0000","last_peered":"2023-08-25T03:34:03.627422+0000","last_clean":"2023-08-25T03:34:03.627422+0000","last_became_active":"2023-08-25T03:34:03.619691+0000","last_became_peered":"2023-08-25T03:34:03.619691+0000","last_unstale":"2023-08-25T03:34:03.627422+0000","last_undegraded":"2023-08-25T03:34:03.627422+0000","last_fullsized":"2023-08-25T03:34:03.627422+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:25:49.895284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.584558+0000","last_change":"2023-08-25T03:34:02.584558+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.584558+0000","last_undegraded":"2023-08-25T03:34:02.584558+0000","last_fullsized":"2023-08-25T03:34:02.584558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.584609+0000","last_change":"2023-08-25T03:34:02.584609+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.584609+0000","last_undegraded":"2023-08-25T03:34:02.584609+0000","last_fullsized":"2023-08-25T03:34:02.584609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.620519+0000","last_change":"2023-08-25T03:34:03.616557+0000","last_active":"2023-08-25T03:34:03.620519+0000","last_peered":"2023-08-25T03:34:03.620519+0000","last_clean":"2023-08-25T03:34:03.620519+0000","last_became_active":"2023-08-25T03:34:03.615850+0000","last_became_peered":"2023-08-25T03:34:03.615850+0000","last_unstale":"2023-08-25T03:34:03.620519+0000","last_undegraded":"2023-08-25T03:34:03.620519+0000","last_fullsized":"2023-08-25T03:34:03.620519+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:01:15.550047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.621835+0000","last_change":"2023-08-25T03:34:03.617319+0000","last_active":"2023-08-25T03:34:03.621835+0000","last_peered":"2023-08-25T03:34:03.621835+0000","last_clean":"2023-08-25T03:34:03.621835+0000","last_became_active":"2023-08-25T03:34:03.617012+0000","last_became_peered":"2023-08-25T03:34:03.617012+0000","last_unstale":"2023-08-25T03:34:03.621835+0000","last_undegraded":"2023-08-25T03:34:03.621835+0000","last_fullsized":"2023-08-25T03:34:03.621835+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:22:25.881167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.624247+0000","last_change":"2023-08-25T03:34:03.615370+0000","last_active":"2023-08-25T03:34:03.624247+0000","last_peered":"2023-08-25T03:34:03.624247+0000","last_clean":"2023-08-25T03:34:03.624247+0000","last_became_active":"2023-08-25T03:34:03.615076+0000","last_became_peered":"2023-08-25T03:34:03.615076+0000","last_unstale":"2023-08-25T03:34:03.624247+0000","last_undegraded":"2023-08-25T03:34:03.624247+0000","last_fullsized":"2023-08-25T03:34:03.624247+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:16:06.275512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591771+0000","last_change":"2023-08-25T03:34:02.591771+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591771+0000","last_undegraded":"2023-08-25T03:34:02.591771+0000","last_fullsized":"2023-08-25T03:34:02.591771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627116+0000","last_change":"2023-08-25T03:34:03.618966+0000","last_active":"2023-08-25T03:34:03.627116+0000","last_peered":"2023-08-25T03:34:03.627116+0000","last_clean":"2023-08-25T03:34:03.627116+0000","last_became_active":"2023-08-25T03:34:03.618636+0000","last_became_peered":"2023-08-25T03:34:03.618636+0000","last_unstale":"2023-08-25T03:34:03.627116+0000","last_undegraded":"2023-08-25T03:34:03.627116+0000","last_fullsized":"2023-08-25T03:34:03.627116+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:15:37.568529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.583704+0000","last_change":"2023-08-25T03:34:02.583704+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.583704+0000","last_undegraded":"2023-08-25T03:34:02.583704+0000","last_fullsized":"2023-08-25T03:34:02.583704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627305+0000","last_change":"2023-08-25T03:34:03.621319+0000","last_active":"2023-08-25T03:34:03.627305+0000","last_peered":"2023-08-25T03:34:03.627305+0000","last_clean":"2023-08-25T03:34:03.627305+0000","last_became_active":"2023-08-25T03:34:03.619115+0000","last_became_peered":"2023-08-25T03:34:03.619115+0000","last_unstale":"2023-08-25T03:34:03.627305+0000","last_undegraded":"2023-08-25T03:34:03.627305+0000","last_fullsized":"2023-08-25T03:34:03.627305+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:12:38.282041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627798+0000","last_change":"2023-08-25T03:34:03.622785+0000","last_active":"2023-08-25T03:34:03.627798+0000","last_peered":"2023-08-25T03:34:03.627798+0000","last_clean":"2023-08-25T03:34:03.627798+0000","last_became_active":"2023-08-25T03:34:03.622469+0000","last_became_peered":"2023-08-25T03:34:03.622469+0000","last_unstale":"2023-08-25T03:34:03.627798+0000","last_undegraded":"2023-08-25T03:34:03.627798+0000","last_fullsized":"2023-08-25T03:34:03.627798+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:47:13.833805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.589817+0000","last_change":"2023-08-25T03:34:02.589817+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.589817+0000","last_undegraded":"2023-08-25T03:34:02.589817+0000","last_fullsized":"2023-08-25T03:34:02.589817+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591327+0000","last_change":"2023-08-25T03:34:02.591327+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591327+0000","last_undegraded":"2023-08-25T03:34:02.591327+0000","last_fullsized":"2023-08-25T03:34:02.591327+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628155+0000","last_change":"2023-08-25T03:34:03.623150+0000","last_active":"2023-08-25T03:34:03.628155+0000","last_peered":"2023-08-25T03:34:03.628155+0000","last_clean":"2023-08-25T03:34:03.628155+0000","last_became_active":"2023-08-25T03:34:03.622796+0000","last_became_peered":"2023-08-25T03:34:03.622796+0000","last_unstale":"2023-08-25T03:34:03.628155+0000","last_undegraded":"2023-08-25T03:34:03.628155+0000","last_fullsized":"2023-08-25T03:34:03.628155+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:49:11.917056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.588252+0000","last_change":"2023-08-25T03:34:02.588252+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.588252+0000","last_undegraded":"2023-08-25T03:34:02.588252+0000","last_fullsized":"2023-08-25T03:34:02.588252+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.586918+0000","last_change":"2023-08-25T03:34:02.586918+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.586918+0000","last_undegraded":"2023-08-25T03:34:02.586918+0000","last_fullsized":"2023-08-25T03:34:02.586918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.616368+0000","last_change":"2023-08-25T03:34:03.613385+0000","last_active":"2023-08-25T03:34:03.616368+0000","last_peered":"2023-08-25T03:34:03.616368+0000","last_clean":"2023-08-25T03:34:03.616368+0000","last_became_active":"2023-08-25T03:34:03.613110+0000","last_became_peered":"2023-08-25T03:34:03.613110+0000","last_unstale":"2023-08-25T03:34:03.616368+0000","last_undegraded":"2023-08-25T03:34:03.616368+0000","last_fullsized":"2023-08-25T03:34:03.616368+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:47:27.124952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628064+0000","last_change":"2023-08-25T03:34:02.608606+0000","last_active":"2023-08-25T03:34:03.628064+0000","last_peered":"2023-08-25T03:34:03.628064+0000","last_clean":"2023-08-25T03:34:03.628064+0000","last_became_active":"2023-08-25T03:34:02.608298+0000","last_became_peered":"2023-08-25T03:34:02.608298+0000","last_unstale":"2023-08-25T03:34:03.628064+0000","last_undegraded":"2023-08-25T03:34:03.628064+0000","last_fullsized":"2023-08-25T03:34:03.628064+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:27:59.879524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591347+0000","last_change":"2023-08-25T03:34:02.591347+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591347+0000","last_undegraded":"2023-08-25T03:34:02.591347+0000","last_fullsized":"2023-08-25T03:34:02.591347+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.603129+0000","last_change":"2023-08-25T03:34:02.599366+0000","last_active":"2023-08-25T03:34:02.603129+0000","last_peered":"2023-08-25T03:34:02.603129+0000","last_clean":"2023-08-25T03:34:02.603129+0000","last_became_active":"2023-08-25T03:34:02.599105+0000","last_became_peered":"2023-08-25T03:34:02.599105+0000","last_unstale":"2023-08-25T03:34:02.603129+0000","last_undegraded":"2023-08-25T03:34:02.603129+0000","last_fullsized":"2023-08-25T03:34:02.603129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:38:11.493391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591521+0000","last_change":"2023-08-25T03:34:02.591521+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591521+0000","last_undegraded":"2023-08-25T03:34:02.591521+0000","last_fullsized":"2023-08-25T03:34:02.591521+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.609089+0000","last_change":"2023-08-25T03:34:02.608397+0000","last_active":"2023-08-25T03:34:02.609089+0000","last_peered":"2023-08-25T03:34:02.609089+0000","last_clean":"2023-08-25T03:34:02.609089+0000","last_became_active":"2023-08-25T03:34:02.608067+0000","last_became_peered":"2023-08-25T03:34:02.608067+0000","last_unstale":"2023-08-25T03:34:02.609089+0000","last_undegraded":"2023-08-25T03:34:02.609089+0000","last_fullsized":"2023-08-25T03:34:02.609089+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:14:44.918756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.616644+0000","last_change":"2023-08-25T03:34:03.613614+0000","last_active":"2023-08-25T03:34:03.616644+0000","last_peered":"2023-08-25T03:34:03.616644+0000","last_clean":"2023-08-25T03:34:03.616644+0000","last_became_active":"2023-08-25T03:34:03.613291+0000","last_became_peered":"2023-08-25T03:34:03.613291+0000","last_unstale":"2023-08-25T03:34:03.616644+0000","last_undegraded":"2023-08-25T03:34:03.616644+0000","last_fullsized":"2023-08-25T03:34:03.616644+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:43:35.780927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.605932+0000","last_change":"2023-08-25T03:34:02.603475+0000","last_active":"2023-08-25T03:34:02.605932+0000","last_peered":"2023-08-25T03:34:02.605932+0000","last_clean":"2023-08-25T03:34:02.605932+0000","last_became_active":"2023-08-25T03:34:02.603129+0000","last_became_peered":"2023-08-25T03:34:02.603129+0000","last_unstale":"2023-08-25T03:34:02.605932+0000","last_undegraded":"2023-08-25T03:34:02.605932+0000","last_fullsized":"2023-08-25T03:34:02.605932+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:10:58.540490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.590564+0000","last_change":"2023-08-25T03:34:02.590564+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.590564+0000","last_undegraded":"2023-08-25T03:34:02.590564+0000","last_fullsized":"2023-08-25T03:34:02.590564+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610665+0000","last_change":"2023-08-25T03:34:02.597247+0000","last_active":"2023-08-25T03:34:03.610665+0000","last_peered":"2023-08-25T03:34:03.610665+0000","last_clean":"2023-08-25T03:34:03.610665+0000","last_became_active":"2023-08-25T03:34:02.596968+0000","last_became_peered":"2023-08-25T03:34:02.596968+0000","last_unstale":"2023-08-25T03:34:03.610665+0000","last_undegraded":"2023-08-25T03:34:03.610665+0000","last_fullsized":"2023-08-25T03:34:03.610665+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:51:52.093732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.592296+0000","last_change":"2023-08-25T03:34:02.592296+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.592296+0000","last_undegraded":"2023-08-25T03:34:02.592296+0000","last_fullsized":"2023-08-25T03:34:02.592296+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.633428+0000","last_change":"2023-08-25T03:34:02.603880+0000","last_active":"2023-08-25T03:34:03.633428+0000","last_peered":"2023-08-25T03:34:03.633428+0000","last_clean":"2023-08-25T03:34:03.633428+0000","last_became_active":"2023-08-25T03:34:02.603502+0000","last_became_peered":"2023-08-25T03:34:02.603502+0000","last_unstale":"2023-08-25T03:34:03.633428+0000","last_undegraded":"2023-08-25T03:34:03.633428+0000","last_fullsized":"2023-08-25T03:34:03.633428+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:05:24.650744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.586286+0000","last_change":"2023-08-25T03:34:02.586286+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.586286+0000","last_undegraded":"2023-08-25T03:34:02.586286+0000","last_fullsized":"2023-08-25T03:34:02.586286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'6","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.632748+0000","last_change":"2023-08-25T03:34:02.606875+0000","last_active":"2023-08-25T03:34:03.632748+0000","last_peered":"2023-08-25T03:34:03.632748+0000","last_clean":"2023-08-25T03:34:03.632748+0000","last_became_active":"2023-08-25T03:34:02.606567+0000","last_became_peered":"2023-08-25T03:34:02.606567+0000","last_unstale":"2023-08-25T03:34:03.632748+0000","last_undegraded":"2023-08-25T03:34:03.632748+0000","last_fullsized":"2023-08-25T03:34:03.632748+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:51:52.093732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627947+0000","last_change":"2023-08-25T03:34:03.619847+0000","last_active":"2023-08-25T03:34:03.627947+0000","last_peered":"2023-08-25T03:34:03.627947+0000","last_clean":"2023-08-25T03:34:03.627947+0000","last_became_active":"2023-08-25T03:34:03.619046+0000","last_became_peered":"2023-08-25T03:34:03.619046+0000","last_unstale":"2023-08-25T03:34:03.627947+0000","last_undegraded":"2023-08-25T03:34:03.627947+0000","last_fullsized":"2023-08-25T03:34:03.627947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:52:42.938242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628937+0000","last_change":"2023-08-25T03:34:02.604424+0000","last_active":"2023-08-25T03:34:03.628937+0000","last_peered":"2023-08-25T03:34:03.628937+0000","last_clean":"2023-08-25T03:34:03.628937+0000","last_became_active":"2023-08-25T03:34:02.604175+0000","last_became_peered":"2023-08-25T03:34:02.604175+0000","last_unstale":"2023-08-25T03:34:03.628937+0000","last_undegraded":"2023-08-25T03:34:03.628937+0000","last_fullsized":"2023-08-25T03:34:03.628937+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:57.997213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591307+0000","last_change":"2023-08-25T03:34:02.591307+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591307+0000","last_undegraded":"2023-08-25T03:34:02.591307+0000","last_fullsized":"2023-08-25T03:34:02.591307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.607623+0000","last_change":"2023-08-25T03:34:02.606889+0000","last_active":"2023-08-25T03:34:02.607623+0000","last_peered":"2023-08-25T03:34:02.607623+0000","last_clean":"2023-08-25T03:34:02.607623+0000","last_became_active":"2023-08-25T03:34:02.606424+0000","last_became_peered":"2023-08-25T03:34:02.606424+0000","last_unstale":"2023-08-25T03:34:02.607623+0000","last_undegraded":"2023-08-25T03:34:02.607623+0000","last_fullsized":"2023-08-25T03:34:02.607623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:04:02.000324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.616388+0000","last_change":"2023-08-25T03:34:03.613888+0000","last_active":"2023-08-25T03:34:03.616388+0000","last_peered":"2023-08-25T03:34:03.616388+0000","last_clean":"2023-08-25T03:34:03.616388+0000","last_became_active":"2023-08-25T03:34:03.612829+0000","last_became_peered":"2023-08-25T03:34:03.612829+0000","last_unstale":"2023-08-25T03:34:03.616388+0000","last_undegraded":"2023-08-25T03:34:03.616388+0000","last_fullsized":"2023-08-25T03:34:03.616388+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:23:49.310992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610728+0000","last_change":"2023-08-25T03:34:02.606279+0000","last_active":"2023-08-25T03:34:03.610728+0000","last_peered":"2023-08-25T03:34:03.610728+0000","last_clean":"2023-08-25T03:34:03.610728+0000","last_became_active":"2023-08-25T03:34:02.606005+0000","last_became_peered":"2023-08-25T03:34:02.606005+0000","last_unstale":"2023-08-25T03:34:03.610728+0000","last_undegraded":"2023-08-25T03:34:03.610728+0000","last_fullsized":"2023-08-25T03:34:03.610728+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:02:01.407883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.592567+0000","last_change":"2023-08-25T03:34:02.592567+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.592567+0000","last_undegraded":"2023-08-25T03:34:02.592567+0000","last_fullsized":"2023-08-25T03:34:02.592567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.633646+0000","last_change":"2023-08-25T03:34:02.608430+0000","last_active":"2023-08-25T03:34:03.633646+0000","last_peered":"2023-08-25T03:34:03.633646+0000","last_clean":"2023-08-25T03:34:03.633646+0000","last_became_active":"2023-08-25T03:34:02.608086+0000","last_became_peered":"2023-08-25T03:34:02.608086+0000","last_unstale":"2023-08-25T03:34:03.633646+0000","last_undegraded":"2023-08-25T03:34:03.633646+0000","last_fullsized":"2023-08-25T03:34:03.633646+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:52:41.930828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591676+0000","last_change":"2023-08-25T03:34:02.591676+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591676+0000","last_undegraded":"2023-08-25T03:34:02.591676+0000","last_fullsized":"2023-08-25T03:34:02.591676+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628287+0000","last_change":"2023-08-25T03:34:02.604587+0000","last_active":"2023-08-25T03:34:03.628287+0000","last_peered":"2023-08-25T03:34:03.628287+0000","last_clean":"2023-08-25T03:34:03.628287+0000","last_became_active":"2023-08-25T03:34:02.604290+0000","last_became_peered":"2023-08-25T03:34:02.604290+0000","last_unstale":"2023-08-25T03:34:03.628287+0000","last_undegraded":"2023-08-25T03:34:03.628287+0000","last_fullsized":"2023-08-25T03:34:03.628287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:52:25.303751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.615963+0000","last_change":"2023-08-25T03:34:03.613959+0000","last_active":"2023-08-25T03:34:03.615963+0000","last_peered":"2023-08-25T03:34:03.615963+0000","last_clean":"2023-08-25T03:34:03.615963+0000","last_became_active":"2023-08-25T03:34:03.613667+0000","last_became_peered":"2023-08-25T03:34:03.613667+0000","last_unstale":"2023-08-25T03:34:03.615963+0000","last_undegraded":"2023-08-25T03:34:03.615963+0000","last_fullsized":"2023-08-25T03:34:03.615963+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:25:49.895284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628771+0000","last_change":"2023-08-25T03:34:02.604585+0000","last_active":"2023-08-25T03:34:03.628771+0000","last_peered":"2023-08-25T03:34:03.628771+0000","last_clean":"2023-08-25T03:34:03.628771+0000","last_became_active":"2023-08-25T03:34:02.604255+0000","last_became_peered":"2023-08-25T03:34:02.604255+0000","last_unstale":"2023-08-25T03:34:03.628771+0000","last_undegraded":"2023-08-25T03:34:03.628771+0000","last_fullsized":"2023-08-25T03:34:03.628771+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:07.999174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627819+0000","last_change":"2023-08-25T03:34:03.624731+0000","last_active":"2023-08-25T03:34:03.627819+0000","last_peered":"2023-08-25T03:34:03.627819+0000","last_clean":"2023-08-25T03:34:03.627819+0000","last_became_active":"2023-08-25T03:34:03.619454+0000","last_became_peered":"2023-08-25T03:34:03.619454+0000","last_unstale":"2023-08-25T03:34:03.627819+0000","last_undegraded":"2023-08-25T03:34:03.627819+0000","last_fullsized":"2023-08-25T03:34:03.627819+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:25:43.507515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.629966+0000","last_change":"2023-08-25T03:34:02.605137+0000","last_active":"2023-08-25T03:34:03.629966+0000","last_peered":"2023-08-25T03:34:03.629966+0000","last_clean":"2023-08-25T03:34:03.629966+0000","last_became_active":"2023-08-25T03:34:02.604846+0000","last_became_peered":"2023-08-25T03:34:02.604846+0000","last_unstale":"2023-08-25T03:34:03.629966+0000","last_undegraded":"2023-08-25T03:34:03.629966+0000","last_fullsized":"2023-08-25T03:34:03.629966+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:07.243584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.616584+0000","last_change":"2023-08-25T03:34:03.614885+0000","last_active":"2023-08-25T03:34:03.616584+0000","last_peered":"2023-08-25T03:34:03.616584+0000","last_clean":"2023-08-25T03:34:03.616584+0000","last_became_active":"2023-08-25T03:34:03.614420+0000","last_became_peered":"2023-08-25T03:34:03.614420+0000","last_unstale":"2023-08-25T03:34:03.616584+0000","last_undegraded":"2023-08-25T03:34:03.616584+0000","last_fullsized":"2023-08-25T03:34:03.616584+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:30:21.048549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610137+0000","last_change":"2023-08-25T03:34:02.598840+0000","last_active":"2023-08-25T03:34:03.610137+0000","last_peered":"2023-08-25T03:34:03.610137+0000","last_clean":"2023-08-25T03:34:03.610137+0000","last_became_active":"2023-08-25T03:34:02.598544+0000","last_became_peered":"2023-08-25T03:34:02.598544+0000","last_unstale":"2023-08-25T03:34:03.610137+0000","last_undegraded":"2023-08-25T03:34:03.610137+0000","last_fullsized":"2023-08-25T03:34:03.610137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:43:28.361208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591069+0000","last_change":"2023-08-25T03:34:02.591069+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591069+0000","last_undegraded":"2023-08-25T03:34:02.591069+0000","last_fullsized":"2023-08-25T03:34:02.591069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.605947+0000","last_change":"2023-08-25T03:34:02.605001+0000","last_active":"2023-08-25T03:34:02.605947+0000","last_peered":"2023-08-25T03:34:02.605947+0000","last_clean":"2023-08-25T03:34:02.605947+0000","last_became_active":"2023-08-25T03:34:02.604686+0000","last_became_peered":"2023-08-25T03:34:02.604686+0000","last_unstale":"2023-08-25T03:34:02.605947+0000","last_undegraded":"2023-08-25T03:34:02.605947+0000","last_fullsized":"2023-08-25T03:34:02.605947+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:04:02.000324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.629507+0000","last_change":"2023-08-25T03:34:03.626817+0000","last_active":"2023-08-25T03:34:03.629507+0000","last_peered":"2023-08-25T03:34:03.629507+0000","last_clean":"2023-08-25T03:34:03.629507+0000","last_became_active":"2023-08-25T03:34:03.626483+0000","last_became_peered":"2023-08-25T03:34:03.626483+0000","last_unstale":"2023-08-25T03:34:03.629507+0000","last_undegraded":"2023-08-25T03:34:03.629507+0000","last_fullsized":"2023-08-25T03:34:03.629507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:30:21.048549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.600756+0000","last_change":"2023-08-25T03:34:02.601925+0000","last_active":"2023-08-25T03:34:03.600756+0000","last_peered":"2023-08-25T03:34:03.600756+0000","last_clean":"2023-08-25T03:34:03.600756+0000","last_became_active":"2023-08-25T03:34:02.601552+0000","last_became_peered":"2023-08-25T03:34:02.601552+0000","last_unstale":"2023-08-25T03:34:03.600756+0000","last_undegraded":"2023-08-25T03:34:03.600756+0000","last_fullsized":"2023-08-25T03:34:03.600756+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:25:48.887870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.630046+0000","last_change":"2023-08-25T03:34:03.627975+0000","last_active":"2023-08-25T03:34:03.630046+0000","last_peered":"2023-08-25T03:34:03.630046+0000","last_clean":"2023-08-25T03:34:03.630046+0000","last_became_active":"2023-08-25T03:34:03.627489+0000","last_became_peered":"2023-08-25T03:34:03.627489+0000","last_unstale":"2023-08-25T03:34:03.630046+0000","last_undegraded":"2023-08-25T03:34:03.630046+0000","last_fullsized":"2023-08-25T03:34:03.630046+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:23:49.310992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.605294+0000","last_change":"2023-08-25T03:34:02.603347+0000","last_active":"2023-08-25T03:34:03.605294+0000","last_peered":"2023-08-25T03:34:03.605294+0000","last_clean":"2023-08-25T03:34:03.605294+0000","last_became_active":"2023-08-25T03:34:02.603010+0000","last_became_peered":"2023-08-25T03:34:02.603010+0000","last_unstale":"2023-08-25T03:34:03.605294+0000","last_undegraded":"2023-08-25T03:34:03.605294+0000","last_fullsized":"2023-08-25T03:34:03.605294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:38:55.054764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.626681+0000","last_change":"2023-08-25T03:34:03.616962+0000","last_active":"2023-08-25T03:34:03.626681+0000","last_peered":"2023-08-25T03:34:03.626681+0000","last_clean":"2023-08-25T03:34:03.626681+0000","last_became_active":"2023-08-25T03:34:03.616656+0000","last_became_peered":"2023-08-25T03:34:03.616656+0000","last_unstale":"2023-08-25T03:34:03.626681+0000","last_undegraded":"2023-08-25T03:34:03.626681+0000","last_fullsized":"2023-08-25T03:34:03.626681+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:44:42.715336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.606366+0000","last_change":"2023-08-25T03:34:02.603820+0000","last_active":"2023-08-25T03:34:02.606366+0000","last_peered":"2023-08-25T03:34:02.606366+0000","last_clean":"2023-08-25T03:34:02.606366+0000","last_became_active":"2023-08-25T03:34:02.603493+0000","last_became_peered":"2023-08-25T03:34:02.603493+0000","last_unstale":"2023-08-25T03:34:02.606366+0000","last_undegraded":"2023-08-25T03:34:02.606366+0000","last_fullsized":"2023-08-25T03:34:02.606366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:10:58.540490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.586628+0000","last_change":"2023-08-25T03:34:02.586628+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.586628+0000","last_undegraded":"2023-08-25T03:34:02.586628+0000","last_fullsized":"2023-08-25T03:34:02.586628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.590381+0000","last_change":"2023-08-25T03:33:57.570267+0000","last_active":"2023-08-25T03:34:02.590381+0000","last_peered":"2023-08-25T03:34:02.590381+0000","last_clean":"2023-08-25T03:34:02.590381+0000","last_became_active":"2023-08-25T03:33:57.569882+0000","last_became_peered":"2023-08-25T03:33:57.569882+0000","last_unstale":"2023-08-25T03:34:02.590381+0000","last_undegraded":"2023-08-25T03:34:02.590381+0000","last_fullsized":"2023-08-25T03:34:02.590381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:52:36.914859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610822+0000","last_change":"2023-08-25T03:34:02.600129+0000","last_active":"2023-08-25T03:34:03.610822+0000","last_peered":"2023-08-25T03:34:03.610822+0000","last_clean":"2023-08-25T03:34:03.610822+0000","last_became_active":"2023-08-25T03:34:02.599762+0000","last_became_peered":"2023-08-25T03:34:02.599762+0000","last_unstale":"2023-08-25T03:34:03.610822+0000","last_undegraded":"2023-08-25T03:34:03.610822+0000","last_fullsized":"2023-08-25T03:34:03.610822+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:57.997213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.584740+0000","last_change":"2023-08-25T03:34:02.584740+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.584740+0000","last_undegraded":"2023-08-25T03:34:02.584740+0000","last_fullsized":"2023-08-25T03:34:02.584740+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.609973+0000","last_change":"2023-08-25T03:33:57.569595+0000","last_active":"2023-08-25T03:34:03.609973+0000","last_peered":"2023-08-25T03:34:03.609973+0000","last_clean":"2023-08-25T03:34:03.609973+0000","last_became_active":"2023-08-25T03:33:57.569307+0000","last_became_peered":"2023-08-25T03:33:57.569307+0000","last_unstale":"2023-08-25T03:34:03.609973+0000","last_undegraded":"2023-08-25T03:34:03.609973+0000","last_fullsized":"2023-08-25T03:34:03.609973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:38:50.038795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.605192+0000","last_change":"2023-08-25T03:34:02.605404+0000","last_active":"2023-08-25T03:34:03.605192+0000","last_peered":"2023-08-25T03:34:03.605192+0000","last_clean":"2023-08-25T03:34:03.605192+0000","last_became_active":"2023-08-25T03:34:02.604806+0000","last_became_peered":"2023-08-25T03:34:02.604806+0000","last_unstale":"2023-08-25T03:34:03.605192+0000","last_undegraded":"2023-08-25T03:34:03.605192+0000","last_fullsized":"2023-08-25T03:34:03.605192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:41:01.441055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.585339+0000","last_change":"2023-08-25T03:34:02.585339+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.585339+0000","last_undegraded":"2023-08-25T03:34:02.585339+0000","last_fullsized":"2023-08-25T03:34:02.585339+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.585854+0000","last_change":"2023-08-25T03:33:57.886704+0000","last_active":"2023-08-25T03:34:02.585854+0000","last_peered":"2023-08-25T03:34:02.585854+0000","last_clean":"2023-08-25T03:34:02.585854+0000","last_became_active":"2023-08-25T03:33:57.886294+0000","last_became_peered":"2023-08-25T03:33:57.886294+0000","last_unstale":"2023-08-25T03:34:02.585854+0000","last_undegraded":"2023-08-25T03:34:02.585854+0000","last_fullsized":"2023-08-25T03:34:02.585854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:03:56.984355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.598074+0000","last_change":"2023-08-25T03:34:02.589214+0000","last_active":"2023-08-25T03:34:02.598074+0000","last_peered":"2023-08-25T03:34:02.598074+0000","last_clean":"2023-08-25T03:34:02.598074+0000","last_became_active":"2023-08-25T03:34:02.588607+0000","last_became_peered":"2023-08-25T03:34:02.588607+0000","last_unstale":"2023-08-25T03:34:02.598074+0000","last_undegraded":"2023-08-25T03:34:02.598074+0000","last_fullsized":"2023-08-25T03:34:02.598074+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:43:34.773513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.587185+0000","last_change":"2023-08-25T03:34:02.587185+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.587185+0000","last_undegraded":"2023-08-25T03:34:02.587185+0000","last_fullsized":"2023-08-25T03:34:02.587185+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610764+0000","last_change":"2023-08-25T03:33:57.570657+0000","last_active":"2023-08-25T03:34:03.610764+0000","last_peered":"2023-08-25T03:34:03.610764+0000","last_clean":"2023-08-25T03:34:03.610764+0000","last_became_active":"2023-08-25T03:33:57.570425+0000","last_became_peered":"2023-08-25T03:33:57.570425+0000","last_unstale":"2023-08-25T03:34:03.610764+0000","last_undegraded":"2023-08-25T03:34:03.610764+0000","last_fullsized":"2023-08-25T03:34:03.610764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:59:31.618299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610198+0000","last_change":"2023-08-25T03:33:51.259898+0000","last_active":"2023-08-25T03:34:03.610198+0000","last_peered":"2023-08-25T03:34:03.610198+0000","last_clean":"2023-08-25T03:34:03.610198+0000","last_became_active":"2023-08-25T03:33:51.258780+0000","last_became_peered":"2023-08-25T03:33:51.258780+0000","last_unstale":"2023-08-25T03:34:03.610198+0000","last_undegraded":"2023-08-25T03:34:03.610198+0000","last_fullsized":"2023-08-25T03:34:03.610198+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:49.666858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:49.666858+0000","last_clean_scrub_stamp":"2023-08-25T03:33:49.666858+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:51:40.190283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.605411+0000","last_change":"2023-08-25T03:34:02.598738+0000","last_active":"2023-08-25T03:34:02.605411+0000","last_peered":"2023-08-25T03:34:02.605411+0000","last_clean":"2023-08-25T03:34:02.605411+0000","last_became_active":"2023-08-25T03:34:02.598219+0000","last_became_peered":"2023-08-25T03:34:02.598219+0000","last_unstale":"2023-08-25T03:34:02.605411+0000","last_undegraded":"2023-08-25T03:34:02.605411+0000","last_fullsized":"2023-08-25T03:34:02.605411+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:54:25.036156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.626310+0000","last_change":"2023-08-25T03:34:03.624616+0000","last_active":"2023-08-25T03:34:03.626310+0000","last_peered":"2023-08-25T03:34:03.626310+0000","last_clean":"2023-08-25T03:34:03.626310+0000","last_became_active":"2023-08-25T03:34:03.624330+0000","last_became_peered":"2023-08-25T03:34:03.624330+0000","last_unstale":"2023-08-25T03:34:03.626310+0000","last_undegraded":"2023-08-25T03:34:03.626310+0000","last_fullsized":"2023-08-25T03:34:03.626310+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:44:42.715336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610373+0000","last_change":"2023-08-25T03:33:57.571821+0000","last_active":"2023-08-25T03:34:03.610373+0000","last_peered":"2023-08-25T03:34:03.610373+0000","last_clean":"2023-08-25T03:34:03.610373+0000","last_became_active":"2023-08-25T03:33:57.571528+0000","last_became_peered":"2023-08-25T03:33:57.571528+0000","last_unstale":"2023-08-25T03:34:03.610373+0000","last_undegraded":"2023-08-25T03:34:03.610373+0000","last_fullsized":"2023-08-25T03:34:03.610373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:59:31.618299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.604252+0000","last_change":"2023-08-25T03:34:02.602275+0000","last_active":"2023-08-25T03:34:02.604252+0000","last_peered":"2023-08-25T03:34:02.604252+0000","last_clean":"2023-08-25T03:34:02.604252+0000","last_became_active":"2023-08-25T03:34:02.601960+0000","last_became_peered":"2023-08-25T03:34:02.601960+0000","last_unstale":"2023-08-25T03:34:02.604252+0000","last_undegraded":"2023-08-25T03:34:02.604252+0000","last_fullsized":"2023-08-25T03:34:02.604252+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:14:44.918756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.590059+0000","last_change":"2023-08-25T03:34:02.590059+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.590059+0000","last_undegraded":"2023-08-25T03:34:02.590059+0000","last_fullsized":"2023-08-25T03:34:02.590059+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.611012+0000","last_change":"2023-08-25T03:33:57.571730+0000","last_active":"2023-08-25T03:34:03.611012+0000","last_peered":"2023-08-25T03:34:03.611012+0000","last_clean":"2023-08-25T03:34:03.611012+0000","last_became_active":"2023-08-25T03:33:57.571442+0000","last_became_peered":"2023-08-25T03:33:57.571442+0000","last_unstale":"2023-08-25T03:34:03.611012+0000","last_undegraded":"2023-08-25T03:34:03.611012+0000","last_fullsized":"2023-08-25T03:34:03.611012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:09:58.529844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.603118+0000","last_change":"2023-08-25T03:34:02.596876+0000","last_active":"2023-08-25T03:34:02.603118+0000","last_peered":"2023-08-25T03:34:02.603118+0000","last_clean":"2023-08-25T03:34:02.603118+0000","last_became_active":"2023-08-25T03:34:02.596556+0000","last_became_peered":"2023-08-25T03:34:02.596556+0000","last_unstale":"2023-08-25T03:34:02.603118+0000","last_undegraded":"2023-08-25T03:34:02.603118+0000","last_fullsized":"2023-08-25T03:34:02.603118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:12:37.274627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.589064+0000","last_change":"2023-08-25T03:34:02.589064+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.589064+0000","last_undegraded":"2023-08-25T03:34:02.589064+0000","last_fullsized":"2023-08-25T03:34:02.589064+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.605794+0000","last_change":"2023-08-25T03:34:02.604771+0000","last_active":"2023-08-25T03:34:02.605794+0000","last_peered":"2023-08-25T03:34:02.605794+0000","last_clean":"2023-08-25T03:34:02.605794+0000","last_became_active":"2023-08-25T03:34:02.604509+0000","last_became_peered":"2023-08-25T03:34:02.604509+0000","last_unstale":"2023-08-25T03:34:02.605794+0000","last_undegraded":"2023-08-25T03:34:02.605794+0000","last_fullsized":"2023-08-25T03:34:02.605794+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:16:05.268097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.590154+0000","last_change":"2023-08-25T03:33:57.570506+0000","last_active":"2023-08-25T03:34:02.590154+0000","last_peered":"2023-08-25T03:34:02.590154+0000","last_clean":"2023-08-25T03:34:02.590154+0000","last_became_active":"2023-08-25T03:33:57.570171+0000","last_became_peered":"2023-08-25T03:33:57.570171+0000","last_unstale":"2023-08-25T03:34:02.590154+0000","last_undegraded":"2023-08-25T03:34:02.590154+0000","last_fullsized":"2023-08-25T03:34:02.590154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:30:15.025166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.594873+0000","last_change":"2023-08-25T03:34:02.594873+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.594873+0000","last_undegraded":"2023-08-25T03:34:02.594873+0000","last_fullsized":"2023-08-25T03:34:02.594873+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.600483+0000","last_change":"2023-08-25T03:33:57.570598+0000","last_active":"2023-08-25T03:34:03.600483+0000","last_peered":"2023-08-25T03:34:03.600483+0000","last_clean":"2023-08-25T03:34:03.600483+0000","last_became_active":"2023-08-25T03:33:57.570215+0000","last_became_peered":"2023-08-25T03:33:57.570215+0000","last_unstale":"2023-08-25T03:34:03.600483+0000","last_undegraded":"2023-08-25T03:34:03.600483+0000","last_fullsized":"2023-08-25T03:34:03.600483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:12:32.258657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.606611+0000","last_change":"2023-08-25T03:34:02.605323+0000","last_active":"2023-08-25T03:34:02.606611+0000","last_peered":"2023-08-25T03:34:02.606611+0000","last_clean":"2023-08-25T03:34:02.606611+0000","last_became_active":"2023-08-25T03:34:02.605111+0000","last_became_peered":"2023-08-25T03:34:02.605111+0000","last_unstale":"2023-08-25T03:34:02.606611+0000","last_undegraded":"2023-08-25T03:34:02.606611+0000","last_fullsized":"2023-08-25T03:34:02.606611+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:44:41.707921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627797+0000","last_change":"2023-08-25T03:34:03.624353+0000","last_active":"2023-08-25T03:34:03.627797+0000","last_peered":"2023-08-25T03:34:03.627797+0000","last_clean":"2023-08-25T03:34:03.627797+0000","last_became_active":"2023-08-25T03:34:03.618468+0000","last_became_peered":"2023-08-25T03:34:03.618468+0000","last_unstale":"2023-08-25T03:34:03.627797+0000","last_undegraded":"2023-08-25T03:34:03.627797+0000","last_fullsized":"2023-08-25T03:34:03.627797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:38:12.500805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.605909+0000","last_change":"2023-08-25T03:34:02.603419+0000","last_active":"2023-08-25T03:34:02.605909+0000","last_peered":"2023-08-25T03:34:02.605909+0000","last_clean":"2023-08-25T03:34:02.605909+0000","last_became_active":"2023-08-25T03:34:02.603082+0000","last_became_peered":"2023-08-25T03:34:02.603082+0000","last_unstale":"2023-08-25T03:34:02.605909+0000","last_undegraded":"2023-08-25T03:34:02.605909+0000","last_fullsized":"2023-08-25T03:34:02.605909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:16:05.268097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.583635+0000","last_change":"2023-08-25T03:34:02.583635+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.583635+0000","last_undegraded":"2023-08-25T03:34:02.583635+0000","last_fullsized":"2023-08-25T03:34:02.583635+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.629772+0000","last_change":"2023-08-25T03:34:02.601201+0000","last_active":"2023-08-25T03:34:03.629772+0000","last_peered":"2023-08-25T03:34:03.629772+0000","last_clean":"2023-08-25T03:34:03.629772+0000","last_became_active":"2023-08-25T03:34:02.600904+0000","last_became_peered":"2023-08-25T03:34:02.600904+0000","last_unstale":"2023-08-25T03:34:03.629772+0000","last_undegraded":"2023-08-25T03:34:03.629772+0000","last_fullsized":"2023-08-25T03:34:03.629772+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:41:01.441055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.629645+0000","last_change":"2023-08-25T03:34:03.626366+0000","last_active":"2023-08-25T03:34:03.629645+0000","last_peered":"2023-08-25T03:34:03.629645+0000","last_clean":"2023-08-25T03:34:03.629645+0000","last_became_active":"2023-08-25T03:34:03.625496+0000","last_became_peered":"2023-08-25T03:34:03.625496+0000","last_unstale":"2023-08-25T03:34:03.629645+0000","last_undegraded":"2023-08-25T03:34:03.629645+0000","last_fullsized":"2023-08-25T03:34:03.629645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:01:15.550047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.629783+0000","last_change":"2023-08-25T03:34:02.603423+0000","last_active":"2023-08-25T03:34:03.629783+0000","last_peered":"2023-08-25T03:34:03.629783+0000","last_clean":"2023-08-25T03:34:03.629783+0000","last_became_active":"2023-08-25T03:34:02.603077+0000","last_became_peered":"2023-08-25T03:34:02.603077+0000","last_unstale":"2023-08-25T03:34:03.629783+0000","last_undegraded":"2023-08-25T03:34:03.629783+0000","last_fullsized":"2023-08-25T03:34:03.629783+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:27:59.879524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628025+0000","last_change":"2023-08-25T03:34:03.622038+0000","last_active":"2023-08-25T03:34:03.628025+0000","last_peered":"2023-08-25T03:34:03.628025+0000","last_clean":"2023-08-25T03:34:03.628025+0000","last_became_active":"2023-08-25T03:34:03.621747+0000","last_became_peered":"2023-08-25T03:34:03.621747+0000","last_unstale":"2023-08-25T03:34:03.628025+0000","last_undegraded":"2023-08-25T03:34:03.628025+0000","last_fullsized":"2023-08-25T03:34:03.628025+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:43:35.780927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.605434+0000","last_change":"2023-08-25T03:34:02.598272+0000","last_active":"2023-08-25T03:34:02.605434+0000","last_peered":"2023-08-25T03:34:02.605434+0000","last_clean":"2023-08-25T03:34:02.605434+0000","last_became_active":"2023-08-25T03:34:02.597630+0000","last_became_peered":"2023-08-25T03:34:02.597630+0000","last_unstale":"2023-08-25T03:34:02.605434+0000","last_undegraded":"2023-08-25T03:34:02.605434+0000","last_fullsized":"2023-08-25T03:34:02.605434+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:02:33.121969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.592797+0000","last_change":"2023-08-25T03:34:02.592797+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.592797+0000","last_undegraded":"2023-08-25T03:34:02.592797+0000","last_fullsized":"2023-08-25T03:34:02.592797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.604249+0000","last_change":"2023-08-25T03:34:02.597748+0000","last_active":"2023-08-25T03:34:02.604249+0000","last_peered":"2023-08-25T03:34:02.604249+0000","last_clean":"2023-08-25T03:34:02.604249+0000","last_became_active":"2023-08-25T03:34:02.597456+0000","last_became_peered":"2023-08-25T03:34:02.597456+0000","last_unstale":"2023-08-25T03:34:02.604249+0000","last_undegraded":"2023-08-25T03:34:02.604249+0000","last_fullsized":"2023-08-25T03:34:02.604249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:02:33.121969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627853+0000","last_change":"2023-08-25T03:34:03.619886+0000","last_active":"2023-08-25T03:34:03.627853+0000","last_peered":"2023-08-25T03:34:03.627853+0000","last_clean":"2023-08-25T03:34:03.627853+0000","last_became_active":"2023-08-25T03:34:03.618445+0000","last_became_peered":"2023-08-25T03:34:03.618445+0000","last_unstale":"2023-08-25T03:34:03.627853+0000","last_undegraded":"2023-08-25T03:34:03.627853+0000","last_fullsized":"2023-08-25T03:34:03.627853+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:10:59.547904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.616143+0000","last_change":"2023-08-25T03:34:03.612902+0000","last_active":"2023-08-25T03:34:03.616143+0000","last_peered":"2023-08-25T03:34:03.616143+0000","last_clean":"2023-08-25T03:34:03.616143+0000","last_became_active":"2023-08-25T03:34:03.612582+0000","last_became_peered":"2023-08-25T03:34:03.612582+0000","last_unstale":"2023-08-25T03:34:03.616143+0000","last_undegraded":"2023-08-25T03:34:03.616143+0000","last_fullsized":"2023-08-25T03:34:03.616143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:38:12.500805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.593258+0000","last_change":"2023-08-25T03:34:02.593258+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.593258+0000","last_undegraded":"2023-08-25T03:34:02.593258+0000","last_fullsized":"2023-08-25T03:34:02.593258+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628003+0000","last_change":"2023-08-25T03:34:03.622024+0000","last_active":"2023-08-25T03:34:03.628003+0000","last_peered":"2023-08-25T03:34:03.628003+0000","last_clean":"2023-08-25T03:34:03.628003+0000","last_became_active":"2023-08-25T03:34:03.619673+0000","last_became_peered":"2023-08-25T03:34:03.619673+0000","last_unstale":"2023-08-25T03:34:03.628003+0000","last_undegraded":"2023-08-25T03:34:03.628003+0000","last_fullsized":"2023-08-25T03:34:03.628003+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:47:27.124952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.589875+0000","last_change":"2023-08-25T03:34:02.589875+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.589875+0000","last_undegraded":"2023-08-25T03:34:02.589875+0000","last_fullsized":"2023-08-25T03:34:02.589875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.586986+0000","last_change":"2023-08-25T03:34:02.586986+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.586986+0000","last_undegraded":"2023-08-25T03:34:02.586986+0000","last_fullsized":"2023-08-25T03:34:02.586986+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":669737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]}]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":86497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":79369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.249}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":669737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":72749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":79369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":88165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":245760,"data_stored":87149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-25T03:34:07.764 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T03:34:07.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.920+0000 7f3799998700 1 Processor -- start 2023-08-25T03:34:07.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.920+0000 7f3799998700 1 -- start start 2023-08-25T03:34:07.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.921+0000 7f3799998700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3794159ff0 0x7f37940cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:07.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.921+0000 7f3799998700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f379406ba00 0x7f37940cb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:07.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.921+0000 7f3799998700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37940cf0a0 con 0x7f3794071db0 2023-08-25T03:34:07.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.921+0000 7f3799998700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37940ce770 con 0x7f379406ba00 2023-08-25T03:34:07.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.921+0000 7f3799998700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37940cea50 con 0x7f3794159ff0 2023-08-25T03:34:07.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.921+0000 7f37927fc700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f3794071db0 0x7f379406b2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45882/0 (socket says 172.21.15.2:45882) 2023-08-25T03:34:07.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.921+0000 7f37927fc700 1 -- 172.21.15.2:0/4149836736 learned_addr learned my addr 172.21.15.2:0/4149836736 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:07.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.922+0000 7f3792ffd700 1 --2- 172.21.15.2:0/4149836736 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3794159ff0 0x7f37940cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:07.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.922+0000 7f37937fe700 1 --2- 172.21.15.2:0/4149836736 >> v2:172.21.15.31:3300/0 conn(0x7f379406ba00 0x7f37940cb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:07.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.922+0000 7f3791ffb700 1 -- 172.21.15.2:0/4149836736 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1388583280 0 0) 0x7f37940cf0a0 con 0x7f3794071db0 2023-08-25T03:34:07.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.922+0000 7f3791ffb700 1 -- 172.21.15.2:0/4149836736 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f377c0035b0 con 0x7f3794071db0 2023-08-25T03:34:07.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.922+0000 7f3791ffb700 1 -- 172.21.15.2:0/4149836736 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3612118026 0 0) 0x7f377c0035b0 con 0x7f3794071db0 2023-08-25T03:34:07.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.922+0000 7f3791ffb700 1 -- 172.21.15.2:0/4149836736 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3794159ff0 msgr2=0x7f37940cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:07.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.922+0000 7f3791ffb700 1 --2- 172.21.15.2:0/4149836736 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3794159ff0 0x7f37940cde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.922+0000 7f3791ffb700 1 -- 172.21.15.2:0/4149836736 >> v2:172.21.15.31:3300/0 conn(0x7f379406ba00 msgr2=0x7f37940cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:07.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.922+0000 7f3791ffb700 1 --2- 172.21.15.2:0/4149836736 >> v2:172.21.15.31:3300/0 conn(0x7f379406ba00 0x7f37940cb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.922+0000 7f3791ffb700 1 -- 172.21.15.2:0/4149836736 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37940ced30 con 0x7f3794071db0 2023-08-25T03:34:07.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.922+0000 7f3791ffb700 1 -- 172.21.15.2:0/4149836736 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f3788001670 con 0x7f3794071db0 2023-08-25T03:34:07.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.922+0000 7f3791ffb700 1 -- 172.21.15.2:0/4149836736 >> v1:172.21.15.2:6789/0 conn(0x7f3794071db0 legacy=0x7f379406b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:07.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.922+0000 7f3791ffb700 1 --2- 172.21.15.2:0/4149836736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f377c003a00 0x7f377c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:07.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.922+0000 7f3791ffb700 1 --2- 172.21.15.2:0/4149836736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f377c004850 0x7f377c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:07.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.922+0000 7f3791ffb700 1 --2- 172.21.15.2:0/4149836736 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3794159ff0 0x7f377c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:07.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.922+0000 7f3791ffb700 1 -- 172.21.15.2:0/4149836736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37940ce770 con 0x7f377c004850 2023-08-25T03:34:07.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.922+0000 7f3791ffb700 1 -- 172.21.15.2:0/4149836736 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37940cea50 con 0x7f377c003a00 2023-08-25T03:34:07.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.922+0000 7f3791ffb700 1 -- 172.21.15.2:0/4149836736 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f377c00ba10 con 0x7f3794159ff0 2023-08-25T03:34:07.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.922+0000 7f37927fc700 1 --2- 172.21.15.2:0/4149836736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f377c004850 0x7f377c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:07.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.922+0000 7f3792ffd700 1 --2- 172.21.15.2:0/4149836736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f377c003a00 0x7f377c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:07.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.922+0000 7f37937fe700 1 --2- 172.21.15.2:0/4149836736 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3794159ff0 0x7f377c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:07.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.922+0000 7f37927fc700 1 -- 172.21.15.2:0/4149836736 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3794159ff0 msgr2=0x7f377c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:07.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.922+0000 7f37927fc700 1 --2- 172.21.15.2:0/4149836736 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3794159ff0 0x7f377c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.922+0000 7f37927fc700 1 -- 172.21.15.2:0/4149836736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f377c003a00 msgr2=0x7f377c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:07.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.922+0000 7f37927fc700 1 --2- 172.21.15.2:0/4149836736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f377c003a00 0x7f377c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.922+0000 7f37927fc700 1 -- 172.21.15.2:0/4149836736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377c0035b0 con 0x7f377c004850 2023-08-25T03:34:07.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.923+0000 7f37927fc700 1 --2- 172.21.15.2:0/4149836736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f377c004850 0x7f377c004cd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f37940721f0 tx=0x7f377c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84427b82406d8cc9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:07.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.923+0000 7f3791ffb700 1 -- 172.21.15.2:0/4149836736 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3788007c60 con 0x7f377c004850 2023-08-25T03:34:07.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.923+0000 7f3791ffb700 1 -- 172.21.15.2:0/4149836736 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f378800ce80 con 0x7f377c004850 2023-08-25T03:34:07.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.923+0000 7f3791ffb700 1 -- 172.21.15.2:0/4149836736 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3788005ba0 con 0x7f377c004850 2023-08-25T03:34:07.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.923+0000 7f3799998700 1 -- 172.21.15.2:0/4149836736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f377c004850 msgr2=0x7f377c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:07.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.923+0000 7f3799998700 1 --2- 172.21.15.2:0/4149836736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f377c004850 0x7f377c004cd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f37940721f0 tx=0x7f377c005880 comp rx=0 tx=0).stop 2023-08-25T03:34:07.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.923+0000 7f37937fe700 1 -- 172.21.15.2:0/4149836736 reap_dead start 2023-08-25T03:34:07.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.923+0000 7f3799998700 1 -- 172.21.15.2:0/4149836736 shutdown_connections 2023-08-25T03:34:07.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.923+0000 7f3799998700 1 -- 172.21.15.2:0/4149836736 >> 172.21.15.2:0/4149836736 conn(0x7f37940b1bf0 msgr2=0x7f37940b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:07.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.923+0000 7f3799998700 1 -- 172.21.15.2:0/4149836736 shutdown_connections 2023-08-25T03:34:07.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.923+0000 7f3799998700 1 -- 172.21.15.2:0/4149836736 wait complete. 2023-08-25T03:34:07.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.925+0000 7f3799998700 1 Processor -- start 2023-08-25T03:34:07.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.925+0000 7f3799998700 1 -- start start 2023-08-25T03:34:07.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.925+0000 7f3799998700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f377c004850 0x7f37940b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:07.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.925+0000 7f3799998700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37940ad8f0 0x7f37940aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:07.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.925+0000 7f3799998700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f379414ecd0 0x7f37940cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:07.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.926+0000 7f3799998700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37940af3b0 con 0x7f37940ad8f0 2023-08-25T03:34:07.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.926+0000 7f3799998700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37940ac8e0 con 0x7f379414ecd0 2023-08-25T03:34:07.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.926+0000 7f3799998700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37940ad460 con 0x7f377c004850 2023-08-25T03:34:07.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.926+0000 7f37927fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37940ad8f0 0x7f37940aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:07.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.926+0000 7f37927fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37940ad8f0 0x7f37940aee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59228/0 (socket says 172.21.15.2:59228) 2023-08-25T03:34:07.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.926+0000 7f37927fc700 1 -- 172.21.15.2:0/1873421553 learned_addr learned my addr 172.21.15.2:0/1873421553 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:07.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.926+0000 7f3792ffd700 1 --2- 172.21.15.2:0/1873421553 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f377c004850 0x7f37940b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:07.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.926+0000 7f37937fe700 1 --2- 172.21.15.2:0/1873421553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f379414ecd0 0x7f37940cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:07.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.926+0000 7f37927fc700 1 -- 172.21.15.2:0/1873421553 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f377c004850 msgr2=0x7f37940b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:07.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.926+0000 7f37927fc700 1 --2- 172.21.15.2:0/1873421553 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f377c004850 0x7f37940b0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.926+0000 7f37927fc700 1 -- 172.21.15.2:0/1873421553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f379414ecd0 msgr2=0x7f37940cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:07.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.926+0000 7f37927fc700 1 --2- 172.21.15.2:0/1873421553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f379414ecd0 0x7f37940cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:07.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.926+0000 7f37927fc700 1 -- 172.21.15.2:0/1873421553 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37940acb30 con 0x7f37940ad8f0 2023-08-25T03:34:07.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.926+0000 7f37937fe700 1 --2- 172.21.15.2:0/1873421553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f379414ecd0 0x7f37940cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:07.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.926+0000 7f3792ffd700 1 --2- 172.21.15.2:0/1873421553 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f377c004850 0x7f37940b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:07.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.926+0000 7f37927fc700 1 --2- 172.21.15.2:0/1873421553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37940ad8f0 0x7f37940aee70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3788001930 tx=0x7f3788000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:07.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.926+0000 7f377bfff700 1 -- 172.21.15.2:0/1873421553 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3788001ad0 con 0x7f37940ad8f0 2023-08-25T03:34:07.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.926+0000 7f377bfff700 1 -- 172.21.15.2:0/1873421553 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37880070d0 con 0x7f37940ad8f0 2023-08-25T03:34:07.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.926+0000 7f3799998700 1 -- 172.21.15.2:0/1873421553 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37940ace10 con 0x7f37940ad8f0 2023-08-25T03:34:07.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.926+0000 7f377bfff700 1 -- 172.21.15.2:0/1873421553 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3788007460 con 0x7f37940ad8f0 2023-08-25T03:34:07.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.926+0000 7f3799998700 1 -- 172.21.15.2:0/1873421553 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f379414f150 con 0x7f37940ad8f0 2023-08-25T03:34:07.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.928+0000 7f3799998700 1 -- 172.21.15.2:0/1873421553 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3794066f70 con 0x7f37940ad8f0 2023-08-25T03:34:07.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.928+0000 7f377bfff700 1 -- 172.21.15.2:0/1873421553 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f37880075c0 con 0x7f37940ad8f0 2023-08-25T03:34:07.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.928+0000 7f377bfff700 1 --2- 172.21.15.2:0/1873421553 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f378006a0e0 0x7f378006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:07.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.928+0000 7f377bfff700 1 -- 172.21.15.2:0/1873421553 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f37880899b0 con 0x7f37940ad8f0 2023-08-25T03:34:07.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.928+0000 7f3792ffd700 1 --2- 172.21.15.2:0/1873421553 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f378006a0e0 0x7f378006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:07.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.929+0000 7f3792ffd700 1 --2- 172.21.15.2:0/1873421553 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f378006a0e0 0x7f378006c5a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3784009b00 tx=0x7f3784008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:07.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:07.933+0000 7f377bfff700 1 -- 172.21.15.2:0/1873421553 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3788055250 con 0x7f37940ad8f0 2023-08-25T03:34:08.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.128+0000 7f3799998700 1 -- 172.21.15.2:0/1873421553 --> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f37940cf380 con 0x7f378006a0e0 2023-08-25T03:34:08.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.136+0000 7f377bfff700 1 -- 172.21.15.2:0/1873421553 <== mgr.4108 v2:172.21.15.2:6832/3528779779 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284114 (secure 0 0 0) 0x7f3784009790 con 0x7f378006a0e0 2023-08-25T03:34:08.140 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:08.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.139+0000 7f3799998700 1 -- 172.21.15.2:0/1873421553 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f378006a0e0 msgr2=0x7f378006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:08.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.139+0000 7f3799998700 1 --2- 172.21.15.2:0/1873421553 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f378006a0e0 0x7f378006c5a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3784009b00 tx=0x7f3784008040 comp rx=0 tx=0).stop 2023-08-25T03:34:08.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.139+0000 7f3799998700 1 -- 172.21.15.2:0/1873421553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37940ad8f0 msgr2=0x7f37940aee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:08.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.139+0000 7f3799998700 1 --2- 172.21.15.2:0/1873421553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37940ad8f0 0x7f37940aee70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3788001930 tx=0x7f3788000fe0 comp rx=0 tx=0).stop 2023-08-25T03:34:08.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.139+0000 7f3799998700 1 -- 172.21.15.2:0/1873421553 shutdown_connections 2023-08-25T03:34:08.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.139+0000 7f3799998700 1 --2- 172.21.15.2:0/1873421553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f379414ecd0 0x7f37940cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:08.142 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.139+0000 7f3799998700 1 --2- 172.21.15.2:0/1873421553 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f377c004850 0x7f37940b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:08.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.139+0000 7f3799998700 1 --2- 172.21.15.2:0/1873421553 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f378006a0e0 0x7f378006c5a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:08.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.139+0000 7f3799998700 1 --2- 172.21.15.2:0/1873421553 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f37940ad8f0 0x7f37940aee70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:08.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.139+0000 7f3799998700 1 -- 172.21.15.2:0/1873421553 >> 172.21.15.2:0/1873421553 conn(0x7f37940b1bf0 msgr2=0x7f37940b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:08.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.139+0000 7f3799998700 1 -- 172.21.15.2:0/1873421553 shutdown_connections 2023-08-25T03:34:08.143 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.139+0000 7f3799998700 1 -- 172.21.15.2:0/1873421553 wait complete. 2023-08-25T03:34:08.143 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-08-25T03:34:08.152 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-08-25T03:34:07.504648+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":195,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219164,"kb_used_data":2972,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754755556,"statfs":{"total":773094113280,"available":772869689344,"internally_reserved":0,"allocated":3043328,"data_stored":1832772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.915745"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.584233+0000","last_change":"2023-08-25T03:34:02.584233+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.584233+0000","last_undegraded":"2023-08-25T03:34:02.584233+0000","last_fullsized":"2023-08-25T03:34:02.584233+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.620161+0000","last_change":"2023-08-25T03:34:03.616304+0000","last_active":"2023-08-25T03:34:03.620161+0000","last_peered":"2023-08-25T03:34:03.620161+0000","last_clean":"2023-08-25T03:34:03.620161+0000","last_became_active":"2023-08-25T03:34:03.615917+0000","last_became_peered":"2023-08-25T03:34:03.615917+0000","last_unstale":"2023-08-25T03:34:03.620161+0000","last_undegraded":"2023-08-25T03:34:03.620161+0000","last_fullsized":"2023-08-25T03:34:03.620161+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:25:43.507515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627607+0000","last_change":"2023-08-25T03:34:03.622799+0000","last_active":"2023-08-25T03:34:03.627607+0000","last_peered":"2023-08-25T03:34:03.627607+0000","last_clean":"2023-08-25T03:34:03.627607+0000","last_became_active":"2023-08-25T03:34:03.622602+0000","last_became_peered":"2023-08-25T03:34:03.622602+0000","last_unstale":"2023-08-25T03:34:03.627607+0000","last_undegraded":"2023-08-25T03:34:03.627607+0000","last_fullsized":"2023-08-25T03:34:03.627607+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:43:29.368622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.616911+0000","last_change":"2023-08-25T03:34:03.615254+0000","last_active":"2023-08-25T03:34:03.616911+0000","last_peered":"2023-08-25T03:34:03.616911+0000","last_clean":"2023-08-25T03:34:03.616911+0000","last_became_active":"2023-08-25T03:34:03.614917+0000","last_became_peered":"2023-08-25T03:34:03.614917+0000","last_unstale":"2023-08-25T03:34:03.616911+0000","last_undegraded":"2023-08-25T03:34:03.616911+0000","last_fullsized":"2023-08-25T03:34:03.616911+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:10:59.547904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.590306+0000","last_change":"2023-08-25T03:34:02.590306+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.590306+0000","last_undegraded":"2023-08-25T03:34:02.590306+0000","last_fullsized":"2023-08-25T03:34:02.590306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627201+0000","last_change":"2023-08-25T03:34:03.619323+0000","last_active":"2023-08-25T03:34:03.627201+0000","last_peered":"2023-08-25T03:34:03.627201+0000","last_clean":"2023-08-25T03:34:03.627201+0000","last_became_active":"2023-08-25T03:34:03.618985+0000","last_became_peered":"2023-08-25T03:34:03.618985+0000","last_unstale":"2023-08-25T03:34:03.627201+0000","last_undegraded":"2023-08-25T03:34:03.627201+0000","last_fullsized":"2023-08-25T03:34:03.627201+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:47:39.963816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.622126+0000","last_change":"2023-08-25T03:34:03.617493+0000","last_active":"2023-08-25T03:34:03.622126+0000","last_peered":"2023-08-25T03:34:03.622126+0000","last_clean":"2023-08-25T03:34:03.622126+0000","last_became_active":"2023-08-25T03:34:03.617167+0000","last_became_peered":"2023-08-25T03:34:03.617167+0000","last_unstale":"2023-08-25T03:34:03.622126+0000","last_undegraded":"2023-08-25T03:34:03.622126+0000","last_fullsized":"2023-08-25T03:34:03.622126+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:41:33.001585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.629535+0000","last_change":"2023-08-25T03:34:03.625710+0000","last_active":"2023-08-25T03:34:03.629535+0000","last_peered":"2023-08-25T03:34:03.629535+0000","last_clean":"2023-08-25T03:34:03.629535+0000","last_became_active":"2023-08-25T03:34:03.625404+0000","last_became_peered":"2023-08-25T03:34:03.625404+0000","last_unstale":"2023-08-25T03:34:03.629535+0000","last_undegraded":"2023-08-25T03:34:03.629535+0000","last_fullsized":"2023-08-25T03:34:03.629535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:15:37.568529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.593242+0000","last_change":"2023-08-25T03:34:02.593242+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.593242+0000","last_undegraded":"2023-08-25T03:34:02.593242+0000","last_fullsized":"2023-08-25T03:34:02.593242+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.620872+0000","last_change":"2023-08-25T03:34:03.618247+0000","last_active":"2023-08-25T03:34:03.620872+0000","last_peered":"2023-08-25T03:34:03.620872+0000","last_clean":"2023-08-25T03:34:03.620872+0000","last_became_active":"2023-08-25T03:34:03.617830+0000","last_became_peered":"2023-08-25T03:34:03.617830+0000","last_unstale":"2023-08-25T03:34:03.620872+0000","last_undegraded":"2023-08-25T03:34:03.620872+0000","last_fullsized":"2023-08-25T03:34:03.620872+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:47:39.963816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627422+0000","last_change":"2023-08-25T03:34:03.622380+0000","last_active":"2023-08-25T03:34:03.627422+0000","last_peered":"2023-08-25T03:34:03.627422+0000","last_clean":"2023-08-25T03:34:03.627422+0000","last_became_active":"2023-08-25T03:34:03.619691+0000","last_became_peered":"2023-08-25T03:34:03.619691+0000","last_unstale":"2023-08-25T03:34:03.627422+0000","last_undegraded":"2023-08-25T03:34:03.627422+0000","last_fullsized":"2023-08-25T03:34:03.627422+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:25:49.895284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.584558+0000","last_change":"2023-08-25T03:34:02.584558+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.584558+0000","last_undegraded":"2023-08-25T03:34:02.584558+0000","last_fullsized":"2023-08-25T03:34:02.584558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.584609+0000","last_change":"2023-08-25T03:34:02.584609+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.584609+0000","last_undegraded":"2023-08-25T03:34:02.584609+0000","last_fullsized":"2023-08-25T03:34:02.584609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.620519+0000","last_change":"2023-08-25T03:34:03.616557+0000","last_active":"2023-08-25T03:34:03.620519+0000","last_peered":"2023-08-25T03:34:03.620519+0000","last_clean":"2023-08-25T03:34:03.620519+0000","last_became_active":"2023-08-25T03:34:03.615850+0000","last_became_peered":"2023-08-25T03:34:03.615850+0000","last_unstale":"2023-08-25T03:34:03.620519+0000","last_undegraded":"2023-08-25T03:34:03.620519+0000","last_fullsized":"2023-08-25T03:34:03.620519+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:01:15.550047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.621835+0000","last_change":"2023-08-25T03:34:03.617319+0000","last_active":"2023-08-25T03:34:03.621835+0000","last_peered":"2023-08-25T03:34:03.621835+0000","last_clean":"2023-08-25T03:34:03.621835+0000","last_became_active":"2023-08-25T03:34:03.617012+0000","last_became_peered":"2023-08-25T03:34:03.617012+0000","last_unstale":"2023-08-25T03:34:03.621835+0000","last_undegraded":"2023-08-25T03:34:03.621835+0000","last_fullsized":"2023-08-25T03:34:03.621835+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:22:25.881167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.624247+0000","last_change":"2023-08-25T03:34:03.615370+0000","last_active":"2023-08-25T03:34:03.624247+0000","last_peered":"2023-08-25T03:34:03.624247+0000","last_clean":"2023-08-25T03:34:03.624247+0000","last_became_active":"2023-08-25T03:34:03.615076+0000","last_became_peered":"2023-08-25T03:34:03.615076+0000","last_unstale":"2023-08-25T03:34:03.624247+0000","last_undegraded":"2023-08-25T03:34:03.624247+0000","last_fullsized":"2023-08-25T03:34:03.624247+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:16:06.275512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591771+0000","last_change":"2023-08-25T03:34:02.591771+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591771+0000","last_undegraded":"2023-08-25T03:34:02.591771+0000","last_fullsized":"2023-08-25T03:34:02.591771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627116+0000","last_change":"2023-08-25T03:34:03.618966+0000","last_active":"2023-08-25T03:34:03.627116+0000","last_peered":"2023-08-25T03:34:03.627116+0000","last_clean":"2023-08-25T03:34:03.627116+0000","last_became_active":"2023-08-25T03:34:03.618636+0000","last_became_peered":"2023-08-25T03:34:03.618636+0000","last_unstale":"2023-08-25T03:34:03.627116+0000","last_undegraded":"2023-08-25T03:34:03.627116+0000","last_fullsized":"2023-08-25T03:34:03.627116+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:15:37.568529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.583704+0000","last_change":"2023-08-25T03:34:02.583704+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.583704+0000","last_undegraded":"2023-08-25T03:34:02.583704+0000","last_fullsized":"2023-08-25T03:34:02.583704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627305+0000","last_change":"2023-08-25T03:34:03.621319+0000","last_active":"2023-08-25T03:34:03.627305+0000","last_peered":"2023-08-25T03:34:03.627305+0000","last_clean":"2023-08-25T03:34:03.627305+0000","last_became_active":"2023-08-25T03:34:03.619115+0000","last_became_peered":"2023-08-25T03:34:03.619115+0000","last_unstale":"2023-08-25T03:34:03.627305+0000","last_undegraded":"2023-08-25T03:34:03.627305+0000","last_fullsized":"2023-08-25T03:34:03.627305+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:12:38.282041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627798+0000","last_change":"2023-08-25T03:34:03.622785+0000","last_active":"2023-08-25T03:34:03.627798+0000","last_peered":"2023-08-25T03:34:03.627798+0000","last_clean":"2023-08-25T03:34:03.627798+0000","last_became_active":"2023-08-25T03:34:03.622469+0000","last_became_peered":"2023-08-25T03:34:03.622469+0000","last_unstale":"2023-08-25T03:34:03.627798+0000","last_undegraded":"2023-08-25T03:34:03.627798+0000","last_fullsized":"2023-08-25T03:34:03.627798+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:47:13.833805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.589817+0000","last_change":"2023-08-25T03:34:02.589817+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.589817+0000","last_undegraded":"2023-08-25T03:34:02.589817+0000","last_fullsized":"2023-08-25T03:34:02.589817+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591327+0000","last_change":"2023-08-25T03:34:02.591327+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591327+0000","last_undegraded":"2023-08-25T03:34:02.591327+0000","last_fullsized":"2023-08-25T03:34:02.591327+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628155+0000","last_change":"2023-08-25T03:34:03.623150+0000","last_active":"2023-08-25T03:34:03.628155+0000","last_peered":"2023-08-25T03:34:03.628155+0000","last_clean":"2023-08-25T03:34:03.628155+0000","last_became_active":"2023-08-25T03:34:03.622796+0000","last_became_peered":"2023-08-25T03:34:03.622796+0000","last_unstale":"2023-08-25T03:34:03.628155+0000","last_undegraded":"2023-08-25T03:34:03.628155+0000","last_fullsized":"2023-08-25T03:34:03.628155+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:49:11.917056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.588252+0000","last_change":"2023-08-25T03:34:02.588252+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.588252+0000","last_undegraded":"2023-08-25T03:34:02.588252+0000","last_fullsized":"2023-08-25T03:34:02.588252+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.586918+0000","last_change":"2023-08-25T03:34:02.586918+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.586918+0000","last_undegraded":"2023-08-25T03:34:02.586918+0000","last_fullsized":"2023-08-25T03:34:02.586918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.616368+0000","last_change":"2023-08-25T03:34:03.613385+0000","last_active":"2023-08-25T03:34:03.616368+0000","last_peered":"2023-08-25T03:34:03.616368+0000","last_clean":"2023-08-25T03:34:03.616368+0000","last_became_active":"2023-08-25T03:34:03.613110+0000","last_became_peered":"2023-08-25T03:34:03.613110+0000","last_unstale":"2023-08-25T03:34:03.616368+0000","last_undegraded":"2023-08-25T03:34:03.616368+0000","last_fullsized":"2023-08-25T03:34:03.616368+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:47:27.124952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628064+0000","last_change":"2023-08-25T03:34:02.608606+0000","last_active":"2023-08-25T03:34:03.628064+0000","last_peered":"2023-08-25T03:34:03.628064+0000","last_clean":"2023-08-25T03:34:03.628064+0000","last_became_active":"2023-08-25T03:34:02.608298+0000","last_became_peered":"2023-08-25T03:34:02.608298+0000","last_unstale":"2023-08-25T03:34:03.628064+0000","last_undegraded":"2023-08-25T03:34:03.628064+0000","last_fullsized":"2023-08-25T03:34:03.628064+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:27:59.879524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591347+0000","last_change":"2023-08-25T03:34:02.591347+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591347+0000","last_undegraded":"2023-08-25T03:34:02.591347+0000","last_fullsized":"2023-08-25T03:34:02.591347+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.603129+0000","last_change":"2023-08-25T03:34:02.599366+0000","last_active":"2023-08-25T03:34:02.603129+0000","last_peered":"2023-08-25T03:34:02.603129+0000","last_clean":"2023-08-25T03:34:02.603129+0000","last_became_active":"2023-08-25T03:34:02.599105+0000","last_became_peered":"2023-08-25T03:34:02.599105+0000","last_unstale":"2023-08-25T03:34:02.603129+0000","last_undegraded":"2023-08-25T03:34:02.603129+0000","last_fullsized":"2023-08-25T03:34:02.603129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:38:11.493391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591521+0000","last_change":"2023-08-25T03:34:02.591521+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591521+0000","last_undegraded":"2023-08-25T03:34:02.591521+0000","last_fullsized":"2023-08-25T03:34:02.591521+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.609089+0000","last_change":"2023-08-25T03:34:02.608397+0000","last_active":"2023-08-25T03:34:02.609089+0000","last_peered":"2023-08-25T03:34:02.609089+0000","last_clean":"2023-08-25T03:34:02.609089+0000","last_became_active":"2023-08-25T03:34:02.608067+0000","last_became_peered":"2023-08-25T03:34:02.608067+0000","last_unstale":"2023-08-25T03:34:02.609089+0000","last_undegraded":"2023-08-25T03:34:02.609089+0000","last_fullsized":"2023-08-25T03:34:02.609089+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:14:44.918756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.616644+0000","last_change":"2023-08-25T03:34:03.613614+0000","last_active":"2023-08-25T03:34:03.616644+0000","last_peered":"2023-08-25T03:34:03.616644+0000","last_clean":"2023-08-25T03:34:03.616644+0000","last_became_active":"2023-08-25T03:34:03.613291+0000","last_became_peered":"2023-08-25T03:34:03.613291+0000","last_unstale":"2023-08-25T03:34:03.616644+0000","last_undegraded":"2023-08-25T03:34:03.616644+0000","last_fullsized":"2023-08-25T03:34:03.616644+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:43:35.780927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.605932+0000","last_change":"2023-08-25T03:34:02.603475+0000","last_active":"2023-08-25T03:34:02.605932+0000","last_peered":"2023-08-25T03:34:02.605932+0000","last_clean":"2023-08-25T03:34:02.605932+0000","last_became_active":"2023-08-25T03:34:02.603129+0000","last_became_peered":"2023-08-25T03:34:02.603129+0000","last_unstale":"2023-08-25T03:34:02.605932+0000","last_undegraded":"2023-08-25T03:34:02.605932+0000","last_fullsized":"2023-08-25T03:34:02.605932+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:10:58.540490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.590564+0000","last_change":"2023-08-25T03:34:02.590564+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.590564+0000","last_undegraded":"2023-08-25T03:34:02.590564+0000","last_fullsized":"2023-08-25T03:34:02.590564+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610665+0000","last_change":"2023-08-25T03:34:02.597247+0000","last_active":"2023-08-25T03:34:03.610665+0000","last_peered":"2023-08-25T03:34:03.610665+0000","last_clean":"2023-08-25T03:34:03.610665+0000","last_became_active":"2023-08-25T03:34:02.596968+0000","last_became_peered":"2023-08-25T03:34:02.596968+0000","last_unstale":"2023-08-25T03:34:03.610665+0000","last_undegraded":"2023-08-25T03:34:03.610665+0000","last_fullsized":"2023-08-25T03:34:03.610665+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:51:52.093732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.592296+0000","last_change":"2023-08-25T03:34:02.592296+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.592296+0000","last_undegraded":"2023-08-25T03:34:02.592296+0000","last_fullsized":"2023-08-25T03:34:02.592296+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.633428+0000","last_change":"2023-08-25T03:34:02.603880+0000","last_active":"2023-08-25T03:34:03.633428+0000","last_peered":"2023-08-25T03:34:03.633428+0000","last_clean":"2023-08-25T03:34:03.633428+0000","last_became_active":"2023-08-25T03:34:02.603502+0000","last_became_peered":"2023-08-25T03:34:02.603502+0000","last_unstale":"2023-08-25T03:34:03.633428+0000","last_undegraded":"2023-08-25T03:34:03.633428+0000","last_fullsized":"2023-08-25T03:34:03.633428+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:05:24.650744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.586286+0000","last_change":"2023-08-25T03:34:02.586286+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.586286+0000","last_undegraded":"2023-08-25T03:34:02.586286+0000","last_fullsized":"2023-08-25T03:34:02.586286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'6","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.632748+0000","last_change":"2023-08-25T03:34:02.606875+0000","last_active":"2023-08-25T03:34:03.632748+0000","last_peered":"2023-08-25T03:34:03.632748+0000","last_clean":"2023-08-25T03:34:03.632748+0000","last_became_active":"2023-08-25T03:34:02.606567+0000","last_became_peered":"2023-08-25T03:34:02.606567+0000","last_unstale":"2023-08-25T03:34:03.632748+0000","last_undegraded":"2023-08-25T03:34:03.632748+0000","last_fullsized":"2023-08-25T03:34:03.632748+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:51:52.093732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627947+0000","last_change":"2023-08-25T03:34:03.619847+0000","last_active":"2023-08-25T03:34:03.627947+0000","last_peered":"2023-08-25T03:34:03.627947+0000","last_clean":"2023-08-25T03:34:03.627947+0000","last_became_active":"2023-08-25T03:34:03.619046+0000","last_became_peered":"2023-08-25T03:34:03.619046+0000","last_unstale":"2023-08-25T03:34:03.627947+0000","last_undegraded":"2023-08-25T03:34:03.627947+0000","last_fullsized":"2023-08-25T03:34:03.627947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:52:42.938242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628937+0000","last_change":"2023-08-25T03:34:02.604424+0000","last_active":"2023-08-25T03:34:03.628937+0000","last_peered":"2023-08-25T03:34:03.628937+0000","last_clean":"2023-08-25T03:34:03.628937+0000","last_became_active":"2023-08-25T03:34:02.604175+0000","last_became_peered":"2023-08-25T03:34:02.604175+0000","last_unstale":"2023-08-25T03:34:03.628937+0000","last_undegraded":"2023-08-25T03:34:03.628937+0000","last_fullsized":"2023-08-25T03:34:03.628937+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:57.997213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591307+0000","last_change":"2023-08-25T03:34:02.591307+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591307+0000","last_undegraded":"2023-08-25T03:34:02.591307+0000","last_fullsized":"2023-08-25T03:34:02.591307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.607623+0000","last_change":"2023-08-25T03:34:02.606889+0000","last_active":"2023-08-25T03:34:02.607623+0000","last_peered":"2023-08-25T03:34:02.607623+0000","last_clean":"2023-08-25T03:34:02.607623+0000","last_became_active":"2023-08-25T03:34:02.606424+0000","last_became_peered":"2023-08-25T03:34:02.606424+0000","last_unstale":"2023-08-25T03:34:02.607623+0000","last_undegraded":"2023-08-25T03:34:02.607623+0000","last_fullsized":"2023-08-25T03:34:02.607623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:04:02.000324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.616388+0000","last_change":"2023-08-25T03:34:03.613888+0000","last_active":"2023-08-25T03:34:03.616388+0000","last_peered":"2023-08-25T03:34:03.616388+0000","last_clean":"2023-08-25T03:34:03.616388+0000","last_became_active":"2023-08-25T03:34:03.612829+0000","last_became_peered":"2023-08-25T03:34:03.612829+0000","last_unstale":"2023-08-25T03:34:03.616388+0000","last_undegraded":"2023-08-25T03:34:03.616388+0000","last_fullsized":"2023-08-25T03:34:03.616388+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:23:49.310992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610728+0000","last_change":"2023-08-25T03:34:02.606279+0000","last_active":"2023-08-25T03:34:03.610728+0000","last_peered":"2023-08-25T03:34:03.610728+0000","last_clean":"2023-08-25T03:34:03.610728+0000","last_became_active":"2023-08-25T03:34:02.606005+0000","last_became_peered":"2023-08-25T03:34:02.606005+0000","last_unstale":"2023-08-25T03:34:03.610728+0000","last_undegraded":"2023-08-25T03:34:03.610728+0000","last_fullsized":"2023-08-25T03:34:03.610728+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:02:01.407883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.592567+0000","last_change":"2023-08-25T03:34:02.592567+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.592567+0000","last_undegraded":"2023-08-25T03:34:02.592567+0000","last_fullsized":"2023-08-25T03:34:02.592567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.633646+0000","last_change":"2023-08-25T03:34:02.608430+0000","last_active":"2023-08-25T03:34:03.633646+0000","last_peered":"2023-08-25T03:34:03.633646+0000","last_clean":"2023-08-25T03:34:03.633646+0000","last_became_active":"2023-08-25T03:34:02.608086+0000","last_became_peered":"2023-08-25T03:34:02.608086+0000","last_unstale":"2023-08-25T03:34:03.633646+0000","last_undegraded":"2023-08-25T03:34:03.633646+0000","last_fullsized":"2023-08-25T03:34:03.633646+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:52:41.930828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591676+0000","last_change":"2023-08-25T03:34:02.591676+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591676+0000","last_undegraded":"2023-08-25T03:34:02.591676+0000","last_fullsized":"2023-08-25T03:34:02.591676+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628287+0000","last_change":"2023-08-25T03:34:02.604587+0000","last_active":"2023-08-25T03:34:03.628287+0000","last_peered":"2023-08-25T03:34:03.628287+0000","last_clean":"2023-08-25T03:34:03.628287+0000","last_became_active":"2023-08-25T03:34:02.604290+0000","last_became_peered":"2023-08-25T03:34:02.604290+0000","last_unstale":"2023-08-25T03:34:03.628287+0000","last_undegraded":"2023-08-25T03:34:03.628287+0000","last_fullsized":"2023-08-25T03:34:03.628287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:52:25.303751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.615963+0000","last_change":"2023-08-25T03:34:03.613959+0000","last_active":"2023-08-25T03:34:03.615963+0000","last_peered":"2023-08-25T03:34:03.615963+0000","last_clean":"2023-08-25T03:34:03.615963+0000","last_became_active":"2023-08-25T03:34:03.613667+0000","last_became_peered":"2023-08-25T03:34:03.613667+0000","last_unstale":"2023-08-25T03:34:03.615963+0000","last_undegraded":"2023-08-25T03:34:03.615963+0000","last_fullsized":"2023-08-25T03:34:03.615963+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:25:49.895284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628771+0000","last_change":"2023-08-25T03:34:02.604585+0000","last_active":"2023-08-25T03:34:03.628771+0000","last_peered":"2023-08-25T03:34:03.628771+0000","last_clean":"2023-08-25T03:34:03.628771+0000","last_became_active":"2023-08-25T03:34:02.604255+0000","last_became_peered":"2023-08-25T03:34:02.604255+0000","last_unstale":"2023-08-25T03:34:03.628771+0000","last_undegraded":"2023-08-25T03:34:03.628771+0000","last_fullsized":"2023-08-25T03:34:03.628771+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:07.999174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627819+0000","last_change":"2023-08-25T03:34:03.624731+0000","last_active":"2023-08-25T03:34:03.627819+0000","last_peered":"2023-08-25T03:34:03.627819+0000","last_clean":"2023-08-25T03:34:03.627819+0000","last_became_active":"2023-08-25T03:34:03.619454+0000","last_became_peered":"2023-08-25T03:34:03.619454+0000","last_unstale":"2023-08-25T03:34:03.627819+0000","last_undegraded":"2023-08-25T03:34:03.627819+0000","last_fullsized":"2023-08-25T03:34:03.627819+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:25:43.507515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.629966+0000","last_change":"2023-08-25T03:34:02.605137+0000","last_active":"2023-08-25T03:34:03.629966+0000","last_peered":"2023-08-25T03:34:03.629966+0000","last_clean":"2023-08-25T03:34:03.629966+0000","last_became_active":"2023-08-25T03:34:02.604846+0000","last_became_peered":"2023-08-25T03:34:02.604846+0000","last_unstale":"2023-08-25T03:34:03.629966+0000","last_undegraded":"2023-08-25T03:34:03.629966+0000","last_fullsized":"2023-08-25T03:34:03.629966+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:07.243584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.616584+0000","last_change":"2023-08-25T03:34:03.614885+0000","last_active":"2023-08-25T03:34:03.616584+0000","last_peered":"2023-08-25T03:34:03.616584+0000","last_clean":"2023-08-25T03:34:03.616584+0000","last_became_active":"2023-08-25T03:34:03.614420+0000","last_became_peered":"2023-08-25T03:34:03.614420+0000","last_unstale":"2023-08-25T03:34:03.616584+0000","last_undegraded":"2023-08-25T03:34:03.616584+0000","last_fullsized":"2023-08-25T03:34:03.616584+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:30:21.048549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610137+0000","last_change":"2023-08-25T03:34:02.598840+0000","last_active":"2023-08-25T03:34:03.610137+0000","last_peered":"2023-08-25T03:34:03.610137+0000","last_clean":"2023-08-25T03:34:03.610137+0000","last_became_active":"2023-08-25T03:34:02.598544+0000","last_became_peered":"2023-08-25T03:34:02.598544+0000","last_unstale":"2023-08-25T03:34:03.610137+0000","last_undegraded":"2023-08-25T03:34:03.610137+0000","last_fullsized":"2023-08-25T03:34:03.610137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:43:28.361208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591069+0000","last_change":"2023-08-25T03:34:02.591069+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591069+0000","last_undegraded":"2023-08-25T03:34:02.591069+0000","last_fullsized":"2023-08-25T03:34:02.591069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.605947+0000","last_change":"2023-08-25T03:34:02.605001+0000","last_active":"2023-08-25T03:34:02.605947+0000","last_peered":"2023-08-25T03:34:02.605947+0000","last_clean":"2023-08-25T03:34:02.605947+0000","last_became_active":"2023-08-25T03:34:02.604686+0000","last_became_peered":"2023-08-25T03:34:02.604686+0000","last_unstale":"2023-08-25T03:34:02.605947+0000","last_undegraded":"2023-08-25T03:34:02.605947+0000","last_fullsized":"2023-08-25T03:34:02.605947+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:04:02.000324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.629507+0000","last_change":"2023-08-25T03:34:03.626817+0000","last_active":"2023-08-25T03:34:03.629507+0000","last_peered":"2023-08-25T03:34:03.629507+0000","last_clean":"2023-08-25T03:34:03.629507+0000","last_became_active":"2023-08-25T03:34:03.626483+0000","last_became_peered":"2023-08-25T03:34:03.626483+0000","last_unstale":"2023-08-25T03:34:03.629507+0000","last_undegraded":"2023-08-25T03:34:03.629507+0000","last_fullsized":"2023-08-25T03:34:03.629507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:30:21.048549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.600756+0000","last_change":"2023-08-25T03:34:02.601925+0000","last_active":"2023-08-25T03:34:03.600756+0000","last_peered":"2023-08-25T03:34:03.600756+0000","last_clean":"2023-08-25T03:34:03.600756+0000","last_became_active":"2023-08-25T03:34:02.601552+0000","last_became_peered":"2023-08-25T03:34:02.601552+0000","last_unstale":"2023-08-25T03:34:03.600756+0000","last_undegraded":"2023-08-25T03:34:03.600756+0000","last_fullsized":"2023-08-25T03:34:03.600756+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:25:48.887870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.630046+0000","last_change":"2023-08-25T03:34:03.627975+0000","last_active":"2023-08-25T03:34:03.630046+0000","last_peered":"2023-08-25T03:34:03.630046+0000","last_clean":"2023-08-25T03:34:03.630046+0000","last_became_active":"2023-08-25T03:34:03.627489+0000","last_became_peered":"2023-08-25T03:34:03.627489+0000","last_unstale":"2023-08-25T03:34:03.630046+0000","last_undegraded":"2023-08-25T03:34:03.630046+0000","last_fullsized":"2023-08-25T03:34:03.630046+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:23:49.310992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.605294+0000","last_change":"2023-08-25T03:34:02.603347+0000","last_active":"2023-08-25T03:34:03.605294+0000","last_peered":"2023-08-25T03:34:03.605294+0000","last_clean":"2023-08-25T03:34:03.605294+0000","last_became_active":"2023-08-25T03:34:02.603010+0000","last_became_peered":"2023-08-25T03:34:02.603010+0000","last_unstale":"2023-08-25T03:34:03.605294+0000","last_undegraded":"2023-08-25T03:34:03.605294+0000","last_fullsized":"2023-08-25T03:34:03.605294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:38:55.054764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.626681+0000","last_change":"2023-08-25T03:34:03.616962+0000","last_active":"2023-08-25T03:34:03.626681+0000","last_peered":"2023-08-25T03:34:03.626681+0000","last_clean":"2023-08-25T03:34:03.626681+0000","last_became_active":"2023-08-25T03:34:03.616656+0000","last_became_peered":"2023-08-25T03:34:03.616656+0000","last_unstale":"2023-08-25T03:34:03.626681+0000","last_undegraded":"2023-08-25T03:34:03.626681+0000","last_fullsized":"2023-08-25T03:34:03.626681+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:44:42.715336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.606366+0000","last_change":"2023-08-25T03:34:02.603820+0000","last_active":"2023-08-25T03:34:02.606366+0000","last_peered":"2023-08-25T03:34:02.606366+0000","last_clean":"2023-08-25T03:34:02.606366+0000","last_became_active":"2023-08-25T03:34:02.603493+0000","last_became_peered":"2023-08-25T03:34:02.603493+0000","last_unstale":"2023-08-25T03:34:02.606366+0000","last_undegraded":"2023-08-25T03:34:02.606366+0000","last_fullsized":"2023-08-25T03:34:02.606366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:10:58.540490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.586628+0000","last_change":"2023-08-25T03:34:02.586628+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.586628+0000","last_undegraded":"2023-08-25T03:34:02.586628+0000","last_fullsized":"2023-08-25T03:34:02.586628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.590381+0000","last_change":"2023-08-25T03:33:57.570267+0000","last_active":"2023-08-25T03:34:02.590381+0000","last_peered":"2023-08-25T03:34:02.590381+0000","last_clean":"2023-08-25T03:34:02.590381+0000","last_became_active":"2023-08-25T03:33:57.569882+0000","last_became_peered":"2023-08-25T03:33:57.569882+0000","last_unstale":"2023-08-25T03:34:02.590381+0000","last_undegraded":"2023-08-25T03:34:02.590381+0000","last_fullsized":"2023-08-25T03:34:02.590381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:52:36.914859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610822+0000","last_change":"2023-08-25T03:34:02.600129+0000","last_active":"2023-08-25T03:34:03.610822+0000","last_peered":"2023-08-25T03:34:03.610822+0000","last_clean":"2023-08-25T03:34:03.610822+0000","last_became_active":"2023-08-25T03:34:02.599762+0000","last_became_peered":"2023-08-25T03:34:02.599762+0000","last_unstale":"2023-08-25T03:34:03.610822+0000","last_undegraded":"2023-08-25T03:34:03.610822+0000","last_fullsized":"2023-08-25T03:34:03.610822+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:57.997213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.584740+0000","last_change":"2023-08-25T03:34:02.584740+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.584740+0000","last_undegraded":"2023-08-25T03:34:02.584740+0000","last_fullsized":"2023-08-25T03:34:02.584740+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.609973+0000","last_change":"2023-08-25T03:33:57.569595+0000","last_active":"2023-08-25T03:34:03.609973+0000","last_peered":"2023-08-25T03:34:03.609973+0000","last_clean":"2023-08-25T03:34:03.609973+0000","last_became_active":"2023-08-25T03:33:57.569307+0000","last_became_peered":"2023-08-25T03:33:57.569307+0000","last_unstale":"2023-08-25T03:34:03.609973+0000","last_undegraded":"2023-08-25T03:34:03.609973+0000","last_fullsized":"2023-08-25T03:34:03.609973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:38:50.038795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.605192+0000","last_change":"2023-08-25T03:34:02.605404+0000","last_active":"2023-08-25T03:34:03.605192+0000","last_peered":"2023-08-25T03:34:03.605192+0000","last_clean":"2023-08-25T03:34:03.605192+0000","last_became_active":"2023-08-25T03:34:02.604806+0000","last_became_peered":"2023-08-25T03:34:02.604806+0000","last_unstale":"2023-08-25T03:34:03.605192+0000","last_undegraded":"2023-08-25T03:34:03.605192+0000","last_fullsized":"2023-08-25T03:34:03.605192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:41:01.441055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.585339+0000","last_change":"2023-08-25T03:34:02.585339+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.585339+0000","last_undegraded":"2023-08-25T03:34:02.585339+0000","last_fullsized":"2023-08-25T03:34:02.585339+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.585854+0000","last_change":"2023-08-25T03:33:57.886704+0000","last_active":"2023-08-25T03:34:02.585854+0000","last_peered":"2023-08-25T03:34:02.585854+0000","last_clean":"2023-08-25T03:34:02.585854+0000","last_became_active":"2023-08-25T03:33:57.886294+0000","last_became_peered":"2023-08-25T03:33:57.886294+0000","last_unstale":"2023-08-25T03:34:02.585854+0000","last_undegraded":"2023-08-25T03:34:02.585854+0000","last_fullsized":"2023-08-25T03:34:02.585854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:03:56.984355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.598074+0000","last_change":"2023-08-25T03:34:02.589214+0000","last_active":"2023-08-25T03:34:02.598074+0000","last_peered":"2023-08-25T03:34:02.598074+0000","last_clean":"2023-08-25T03:34:02.598074+0000","last_became_active":"2023-08-25T03:34:02.588607+0000","last_became_peered":"2023-08-25T03:34:02.588607+0000","last_unstale":"2023-08-25T03:34:02.598074+0000","last_undegraded":"2023-08-25T03:34:02.598074+0000","last_fullsized":"2023-08-25T03:34:02.598074+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:43:34.773513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.587185+0000","last_change":"2023-08-25T03:34:02.587185+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.587185+0000","last_undegraded":"2023-08-25T03:34:02.587185+0000","last_fullsized":"2023-08-25T03:34:02.587185+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610764+0000","last_change":"2023-08-25T03:33:57.570657+0000","last_active":"2023-08-25T03:34:03.610764+0000","last_peered":"2023-08-25T03:34:03.610764+0000","last_clean":"2023-08-25T03:34:03.610764+0000","last_became_active":"2023-08-25T03:33:57.570425+0000","last_became_peered":"2023-08-25T03:33:57.570425+0000","last_unstale":"2023-08-25T03:34:03.610764+0000","last_undegraded":"2023-08-25T03:34:03.610764+0000","last_fullsized":"2023-08-25T03:34:03.610764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:59:31.618299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610198+0000","last_change":"2023-08-25T03:33:51.259898+0000","last_active":"2023-08-25T03:34:03.610198+0000","last_peered":"2023-08-25T03:34:03.610198+0000","last_clean":"2023-08-25T03:34:03.610198+0000","last_became_active":"2023-08-25T03:33:51.258780+0000","last_became_peered":"2023-08-25T03:33:51.258780+0000","last_unstale":"2023-08-25T03:34:03.610198+0000","last_undegraded":"2023-08-25T03:34:03.610198+0000","last_fullsized":"2023-08-25T03:34:03.610198+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:49.666858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:49.666858+0000","last_clean_scrub_stamp":"2023-08-25T03:33:49.666858+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:51:40.190283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.605411+0000","last_change":"2023-08-25T03:34:02.598738+0000","last_active":"2023-08-25T03:34:02.605411+0000","last_peered":"2023-08-25T03:34:02.605411+0000","last_clean":"2023-08-25T03:34:02.605411+0000","last_became_active":"2023-08-25T03:34:02.598219+0000","last_became_peered":"2023-08-25T03:34:02.598219+0000","last_unstale":"2023-08-25T03:34:02.605411+0000","last_undegraded":"2023-08-25T03:34:02.605411+0000","last_fullsized":"2023-08-25T03:34:02.605411+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:54:25.036156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.626310+0000","last_change":"2023-08-25T03:34:03.624616+0000","last_active":"2023-08-25T03:34:03.626310+0000","last_peered":"2023-08-25T03:34:03.626310+0000","last_clean":"2023-08-25T03:34:03.626310+0000","last_became_active":"2023-08-25T03:34:03.624330+0000","last_became_peered":"2023-08-25T03:34:03.624330+0000","last_unstale":"2023-08-25T03:34:03.626310+0000","last_undegraded":"2023-08-25T03:34:03.626310+0000","last_fullsized":"2023-08-25T03:34:03.626310+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:44:42.715336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610373+0000","last_change":"2023-08-25T03:33:57.571821+0000","last_active":"2023-08-25T03:34:03.610373+0000","last_peered":"2023-08-25T03:34:03.610373+0000","last_clean":"2023-08-25T03:34:03.610373+0000","last_became_active":"2023-08-25T03:33:57.571528+0000","last_became_peered":"2023-08-25T03:33:57.571528+0000","last_unstale":"2023-08-25T03:34:03.610373+0000","last_undegraded":"2023-08-25T03:34:03.610373+0000","last_fullsized":"2023-08-25T03:34:03.610373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:59:31.618299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.604252+0000","last_change":"2023-08-25T03:34:02.602275+0000","last_active":"2023-08-25T03:34:02.604252+0000","last_peered":"2023-08-25T03:34:02.604252+0000","last_clean":"2023-08-25T03:34:02.604252+0000","last_became_active":"2023-08-25T03:34:02.601960+0000","last_became_peered":"2023-08-25T03:34:02.601960+0000","last_unstale":"2023-08-25T03:34:02.604252+0000","last_undegraded":"2023-08-25T03:34:02.604252+0000","last_fullsized":"2023-08-25T03:34:02.604252+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:14:44.918756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.590059+0000","last_change":"2023-08-25T03:34:02.590059+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.590059+0000","last_undegraded":"2023-08-25T03:34:02.590059+0000","last_fullsized":"2023-08-25T03:34:02.590059+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.611012+0000","last_change":"2023-08-25T03:33:57.571730+0000","last_active":"2023-08-25T03:34:03.611012+0000","last_peered":"2023-08-25T03:34:03.611012+0000","last_clean":"2023-08-25T03:34:03.611012+0000","last_became_active":"2023-08-25T03:33:57.571442+0000","last_became_peered":"2023-08-25T03:33:57.571442+0000","last_unstale":"2023-08-25T03:34:03.611012+0000","last_undegraded":"2023-08-25T03:34:03.611012+0000","last_fullsized":"2023-08-25T03:34:03.611012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:09:58.529844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.603118+0000","last_change":"2023-08-25T03:34:02.596876+0000","last_active":"2023-08-25T03:34:02.603118+0000","last_peered":"2023-08-25T03:34:02.603118+0000","last_clean":"2023-08-25T03:34:02.603118+0000","last_became_active":"2023-08-25T03:34:02.596556+0000","last_became_peered":"2023-08-25T03:34:02.596556+0000","last_unstale":"2023-08-25T03:34:02.603118+0000","last_undegraded":"2023-08-25T03:34:02.603118+0000","last_fullsized":"2023-08-25T03:34:02.603118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:12:37.274627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.589064+0000","last_change":"2023-08-25T03:34:02.589064+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.589064+0000","last_undegraded":"2023-08-25T03:34:02.589064+0000","last_fullsized":"2023-08-25T03:34:02.589064+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.605794+0000","last_change":"2023-08-25T03:34:02.604771+0000","last_active":"2023-08-25T03:34:02.605794+0000","last_peered":"2023-08-25T03:34:02.605794+0000","last_clean":"2023-08-25T03:34:02.605794+0000","last_became_active":"2023-08-25T03:34:02.604509+0000","last_became_peered":"2023-08-25T03:34:02.604509+0000","last_unstale":"2023-08-25T03:34:02.605794+0000","last_undegraded":"2023-08-25T03:34:02.605794+0000","last_fullsized":"2023-08-25T03:34:02.605794+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:16:05.268097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.590154+0000","last_change":"2023-08-25T03:33:57.570506+0000","last_active":"2023-08-25T03:34:02.590154+0000","last_peered":"2023-08-25T03:34:02.590154+0000","last_clean":"2023-08-25T03:34:02.590154+0000","last_became_active":"2023-08-25T03:33:57.570171+0000","last_became_peered":"2023-08-25T03:33:57.570171+0000","last_unstale":"2023-08-25T03:34:02.590154+0000","last_undegraded":"2023-08-25T03:34:02.590154+0000","last_fullsized":"2023-08-25T03:34:02.590154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:30:15.025166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.594873+0000","last_change":"2023-08-25T03:34:02.594873+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.594873+0000","last_undegraded":"2023-08-25T03:34:02.594873+0000","last_fullsized":"2023-08-25T03:34:02.594873+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.600483+0000","last_change":"2023-08-25T03:33:57.570598+0000","last_active":"2023-08-25T03:34:03.600483+0000","last_peered":"2023-08-25T03:34:03.600483+0000","last_clean":"2023-08-25T03:34:03.600483+0000","last_became_active":"2023-08-25T03:33:57.570215+0000","last_became_peered":"2023-08-25T03:33:57.570215+0000","last_unstale":"2023-08-25T03:34:03.600483+0000","last_undegraded":"2023-08-25T03:34:03.600483+0000","last_fullsized":"2023-08-25T03:34:03.600483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:12:32.258657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.606611+0000","last_change":"2023-08-25T03:34:02.605323+0000","last_active":"2023-08-25T03:34:02.606611+0000","last_peered":"2023-08-25T03:34:02.606611+0000","last_clean":"2023-08-25T03:34:02.606611+0000","last_became_active":"2023-08-25T03:34:02.605111+0000","last_became_peered":"2023-08-25T03:34:02.605111+0000","last_unstale":"2023-08-25T03:34:02.606611+0000","last_undegraded":"2023-08-25T03:34:02.606611+0000","last_fullsized":"2023-08-25T03:34:02.606611+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:44:41.707921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627797+0000","last_change":"2023-08-25T03:34:03.624353+0000","last_active":"2023-08-25T03:34:03.627797+0000","last_peered":"2023-08-25T03:34:03.627797+0000","last_clean":"2023-08-25T03:34:03.627797+0000","last_became_active":"2023-08-25T03:34:03.618468+0000","last_became_peered":"2023-08-25T03:34:03.618468+0000","last_unstale":"2023-08-25T03:34:03.627797+0000","last_undegraded":"2023-08-25T03:34:03.627797+0000","last_fullsized":"2023-08-25T03:34:03.627797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:38:12.500805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.605909+0000","last_change":"2023-08-25T03:34:02.603419+0000","last_active":"2023-08-25T03:34:02.605909+0000","last_peered":"2023-08-25T03:34:02.605909+0000","last_clean":"2023-08-25T03:34:02.605909+0000","last_became_active":"2023-08-25T03:34:02.603082+0000","last_became_peered":"2023-08-25T03:34:02.603082+0000","last_unstale":"2023-08-25T03:34:02.605909+0000","last_undegraded":"2023-08-25T03:34:02.605909+0000","last_fullsized":"2023-08-25T03:34:02.605909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:16:05.268097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.583635+0000","last_change":"2023-08-25T03:34:02.583635+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.583635+0000","last_undegraded":"2023-08-25T03:34:02.583635+0000","last_fullsized":"2023-08-25T03:34:02.583635+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.629772+0000","last_change":"2023-08-25T03:34:02.601201+0000","last_active":"2023-08-25T03:34:03.629772+0000","last_peered":"2023-08-25T03:34:03.629772+0000","last_clean":"2023-08-25T03:34:03.629772+0000","last_became_active":"2023-08-25T03:34:02.600904+0000","last_became_peered":"2023-08-25T03:34:02.600904+0000","last_unstale":"2023-08-25T03:34:03.629772+0000","last_undegraded":"2023-08-25T03:34:03.629772+0000","last_fullsized":"2023-08-25T03:34:03.629772+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:41:01.441055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.629645+0000","last_change":"2023-08-25T03:34:03.626366+0000","last_active":"2023-08-25T03:34:03.629645+0000","last_peered":"2023-08-25T03:34:03.629645+0000","last_clean":"2023-08-25T03:34:03.629645+0000","last_became_active":"2023-08-25T03:34:03.625496+0000","last_became_peered":"2023-08-25T03:34:03.625496+0000","last_unstale":"2023-08-25T03:34:03.629645+0000","last_undegraded":"2023-08-25T03:34:03.629645+0000","last_fullsized":"2023-08-25T03:34:03.629645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:01:15.550047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.629783+0000","last_change":"2023-08-25T03:34:02.603423+0000","last_active":"2023-08-25T03:34:03.629783+0000","last_peered":"2023-08-25T03:34:03.629783+0000","last_clean":"2023-08-25T03:34:03.629783+0000","last_became_active":"2023-08-25T03:34:02.603077+0000","last_became_peered":"2023-08-25T03:34:02.603077+0000","last_unstale":"2023-08-25T03:34:03.629783+0000","last_undegraded":"2023-08-25T03:34:03.629783+0000","last_fullsized":"2023-08-25T03:34:03.629783+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:27:59.879524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628025+0000","last_change":"2023-08-25T03:34:03.622038+0000","last_active":"2023-08-25T03:34:03.628025+0000","last_peered":"2023-08-25T03:34:03.628025+0000","last_clean":"2023-08-25T03:34:03.628025+0000","last_became_active":"2023-08-25T03:34:03.621747+0000","last_became_peered":"2023-08-25T03:34:03.621747+0000","last_unstale":"2023-08-25T03:34:03.628025+0000","last_undegraded":"2023-08-25T03:34:03.628025+0000","last_fullsized":"2023-08-25T03:34:03.628025+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:43:35.780927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.605434+0000","last_change":"2023-08-25T03:34:02.598272+0000","last_active":"2023-08-25T03:34:02.605434+0000","last_peered":"2023-08-25T03:34:02.605434+0000","last_clean":"2023-08-25T03:34:02.605434+0000","last_became_active":"2023-08-25T03:34:02.597630+0000","last_became_peered":"2023-08-25T03:34:02.597630+0000","last_unstale":"2023-08-25T03:34:02.605434+0000","last_undegraded":"2023-08-25T03:34:02.605434+0000","last_fullsized":"2023-08-25T03:34:02.605434+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:02:33.121969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.592797+0000","last_change":"2023-08-25T03:34:02.592797+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.592797+0000","last_undegraded":"2023-08-25T03:34:02.592797+0000","last_fullsized":"2023-08-25T03:34:02.592797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.604249+0000","last_change":"2023-08-25T03:34:02.597748+0000","last_active":"2023-08-25T03:34:02.604249+0000","last_peered":"2023-08-25T03:34:02.604249+0000","last_clean":"2023-08-25T03:34:02.604249+0000","last_became_active":"2023-08-25T03:34:02.597456+0000","last_became_peered":"2023-08-25T03:34:02.597456+0000","last_unstale":"2023-08-25T03:34:02.604249+0000","last_undegraded":"2023-08-25T03:34:02.604249+0000","last_fullsized":"2023-08-25T03:34:02.604249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:02:33.121969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627853+0000","last_change":"2023-08-25T03:34:03.619886+0000","last_active":"2023-08-25T03:34:03.627853+0000","last_peered":"2023-08-25T03:34:03.627853+0000","last_clean":"2023-08-25T03:34:03.627853+0000","last_became_active":"2023-08-25T03:34:03.618445+0000","last_became_peered":"2023-08-25T03:34:03.618445+0000","last_unstale":"2023-08-25T03:34:03.627853+0000","last_undegraded":"2023-08-25T03:34:03.627853+0000","last_fullsized":"2023-08-25T03:34:03.627853+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:10:59.547904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.616143+0000","last_change":"2023-08-25T03:34:03.612902+0000","last_active":"2023-08-25T03:34:03.616143+0000","last_peered":"2023-08-25T03:34:03.616143+0000","last_clean":"2023-08-25T03:34:03.616143+0000","last_became_active":"2023-08-25T03:34:03.612582+0000","last_became_peered":"2023-08-25T03:34:03.612582+0000","last_unstale":"2023-08-25T03:34:03.616143+0000","last_undegraded":"2023-08-25T03:34:03.616143+0000","last_fullsized":"2023-08-25T03:34:03.616143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:38:12.500805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.593258+0000","last_change":"2023-08-25T03:34:02.593258+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.593258+0000","last_undegraded":"2023-08-25T03:34:02.593258+0000","last_fullsized":"2023-08-25T03:34:02.593258+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628003+0000","last_change":"2023-08-25T03:34:03.622024+0000","last_active":"2023-08-25T03:34:03.628003+0000","last_peered":"2023-08-25T03:34:03.628003+0000","last_clean":"2023-08-25T03:34:03.628003+0000","last_became_active":"2023-08-25T03:34:03.619673+0000","last_became_peered":"2023-08-25T03:34:03.619673+0000","last_unstale":"2023-08-25T03:34:03.628003+0000","last_undegraded":"2023-08-25T03:34:03.628003+0000","last_fullsized":"2023-08-25T03:34:03.628003+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:47:27.124952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.589875+0000","last_change":"2023-08-25T03:34:02.589875+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.589875+0000","last_undegraded":"2023-08-25T03:34:02.589875+0000","last_fullsized":"2023-08-25T03:34:02.589875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.586986+0000","last_change":"2023-08-25T03:34:02.586986+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.586986+0000","last_undegraded":"2023-08-25T03:34:02.586986+0000","last_fullsized":"2023-08-25T03:34:02.586986+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":669737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]}]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":86497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":79369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.249}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":669737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":72749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":79369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":88165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":245760,"data_stored":87149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-25T03:34:08.160 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T03:34:08.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.316+0000 7f21b2863700 1 Processor -- start 2023-08-25T03:34:08.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.316+0000 7f21b2863700 1 -- start start 2023-08-25T03:34:08.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.317+0000 7f21b2863700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f21ac15e890 0x7f21ac071430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:08.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.317+0000 7f21b2863700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f21ac0caea0 0x7f21ac0cb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:08.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.317+0000 7f21b2863700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21ac0720b0 con 0x7f21ac0b19a0 2023-08-25T03:34:08.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.317+0000 7f21b2863700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21ac071dd0 con 0x7f21ac0caea0 2023-08-25T03:34:08.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.317+0000 7f21b2863700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac15ddb0 con 0x7f21ac15e890 2023-08-25T03:34:08.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.317+0000 7f21ab7fe700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f21ac15e890 0x7f21ac071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:08.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.317+0000 7f21b0e00700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f21ac0caea0 0x7f21ac0cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:08.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.317+0000 7f21ab7fe700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f21ac15e890 0x7f21ac071430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.2:59480/0 (socket says 172.21.15.2:59480) 2023-08-25T03:34:08.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.317+0000 7f21ab7fe700 1 -- 172.21.15.2:0/951123700 learned_addr learned my addr 172.21.15.2:0/951123700 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:08.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.318+0000 7f21b0e00700 1 -- 172.21.15.2:0/951123700 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f21ac15e890 msgr2=0x7f21ac071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:08.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.318+0000 7f21b0e00700 1 --2- 172.21.15.2:0/951123700 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f21ac15e890 0x7f21ac071430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:08.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.318+0000 7f21b0e00700 1 -- 172.21.15.2:0/951123700 >> v1:172.21.15.31:6789/0 conn(0x7f21ac0b19a0 legacy=0x7f21ac0b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:08.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.318+0000 7f21b0e00700 1 -- 172.21.15.2:0/951123700 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ac15e090 con 0x7f21ac0caea0 2023-08-25T03:34:08.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.318+0000 7f21ab7fe700 1 --2- 172.21.15.2:0/951123700 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f21ac15e890 0x7f21ac071430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:34:08.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.318+0000 7f21b0e00700 1 --2- 172.21.15.2:0/951123700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21ac0caea0 0x7f21ac0cb330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f21a4009d90 tx=0x7f21a400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccb4153f52f10552 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:08.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.318+0000 7f21aaffd700 1 -- 172.21.15.2:0/951123700 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a400e070 con 0x7f21ac0caea0 2023-08-25T03:34:08.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.318+0000 7f21aaffd700 1 -- 172.21.15.2:0/951123700 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a400bb20 con 0x7f21ac0caea0 2023-08-25T03:34:08.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.318+0000 7f21aaffd700 1 -- 172.21.15.2:0/951123700 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a4012430 con 0x7f21ac0caea0 2023-08-25T03:34:08.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.319+0000 7f21b2863700 1 -- 172.21.15.2:0/951123700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21ac0caea0 msgr2=0x7f21ac0cb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:08.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.319+0000 7f21b2863700 1 --2- 172.21.15.2:0/951123700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21ac0caea0 0x7f21ac0cb330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f21a4009d90 tx=0x7f21a400b330 comp rx=0 tx=0).stop 2023-08-25T03:34:08.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.319+0000 7f21b2863700 1 -- 172.21.15.2:0/951123700 shutdown_connections 2023-08-25T03:34:08.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.319+0000 7f21b2863700 1 --2- 172.21.15.2:0/951123700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21ac0caea0 0x7f21ac0cb330 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:08.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.319+0000 7f21b2863700 1 --2- 172.21.15.2:0/951123700 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f21ac15e890 0x7f21ac071430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:08.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.319+0000 7f21b2863700 1 -- 172.21.15.2:0/951123700 >> 172.21.15.2:0/951123700 conn(0x7f21ac0cfea0 msgr2=0x7f21ac0d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:08.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.319+0000 7f21b2863700 1 -- 172.21.15.2:0/951123700 shutdown_connections 2023-08-25T03:34:08.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.319+0000 7f21b2863700 1 -- 172.21.15.2:0/951123700 wait complete. 2023-08-25T03:34:08.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.320+0000 7f21b2863700 1 Processor -- start 2023-08-25T03:34:08.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.320+0000 7f21b2863700 1 -- start start 2023-08-25T03:34:08.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.321+0000 7f21b2863700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21ac0b19a0 0x7f21ac14ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:08.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.321+0000 7f21b2863700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21ac0caea0 0x7f21ac147990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:08.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.321+0000 7f21b2863700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f21ac15e890 0x7f21ac0b4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:08.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.321+0000 7f21b2863700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac0b50e0 con 0x7f21ac0caea0 2023-08-25T03:34:08.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.321+0000 7f21b2863700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac0b53c0 con 0x7f21ac0b19a0 2023-08-25T03:34:08.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.321+0000 7f21b2863700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21ac0b56a0 con 0x7f21ac15e890 2023-08-25T03:34:08.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.321+0000 7f21ab7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21ac0caea0 0x7f21ac147990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:08.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.321+0000 7f21ab7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21ac0caea0 0x7f21ac147990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59242/0 (socket says 172.21.15.2:59242) 2023-08-25T03:34:08.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.321+0000 7f21ab7fe700 1 -- 172.21.15.2:0/541595801 learned_addr learned my addr 172.21.15.2:0/541595801 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:08.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.321+0000 7f21b0e00700 1 --2- 172.21.15.2:0/541595801 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f21ac15e890 0x7f21ac0b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:08.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.321+0000 7f21abfff700 1 --2- 172.21.15.2:0/541595801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21ac0b19a0 0x7f21ac14ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:08.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.321+0000 7f21ab7fe700 1 -- 172.21.15.2:0/541595801 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f21ac15e890 msgr2=0x7f21ac0b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:08.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.321+0000 7f21ab7fe700 1 --2- 172.21.15.2:0/541595801 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f21ac15e890 0x7f21ac0b4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:08.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.321+0000 7f21ab7fe700 1 -- 172.21.15.2:0/541595801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21ac0b19a0 msgr2=0x7f21ac14ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:08.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.321+0000 7f21ab7fe700 1 --2- 172.21.15.2:0/541595801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21ac0b19a0 0x7f21ac14ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:08.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.321+0000 7f21ab7fe700 1 -- 172.21.15.2:0/541595801 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a0009550 con 0x7f21ac0caea0 2023-08-25T03:34:08.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.322+0000 7f21ab7fe700 1 --2- 172.21.15.2:0/541595801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21ac0caea0 0x7f21ac147990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f21a000ebe0 tx=0x7f21a000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:08.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.322+0000 7f21a8ff9700 1 -- 172.21.15.2:0/541595801 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a000c850 con 0x7f21ac0caea0 2023-08-25T03:34:08.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.322+0000 7f21a8ff9700 1 -- 172.21.15.2:0/541595801 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21a000c9b0 con 0x7f21ac0caea0 2023-08-25T03:34:08.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.322+0000 7f21b2863700 1 -- 172.21.15.2:0/541595801 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a4009a70 con 0x7f21ac0caea0 2023-08-25T03:34:08.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.322+0000 7f21a8ff9700 1 -- 172.21.15.2:0/541595801 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21a000cd40 con 0x7f21ac0caea0 2023-08-25T03:34:08.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.322+0000 7f21b2863700 1 -- 172.21.15.2:0/541595801 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21ac0b5c60 con 0x7f21ac0caea0 2023-08-25T03:34:08.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.323+0000 7f21b2863700 1 -- 172.21.15.2:0/541595801 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21ac066f70 con 0x7f21ac0caea0 2023-08-25T03:34:08.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.323+0000 7f21a8ff9700 1 -- 172.21.15.2:0/541595801 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f21a0013470 con 0x7f21ac0caea0 2023-08-25T03:34:08.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.324+0000 7f21a8ff9700 1 --2- 172.21.15.2:0/541595801 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f219806a0e0 0x7f219806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:08.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.324+0000 7f21abfff700 1 --2- 172.21.15.2:0/541595801 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f219806a0e0 0x7f219806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:08.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.324+0000 7f21a8ff9700 1 -- 172.21.15.2:0/541595801 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f21a00886b0 con 0x7f21ac0caea0 2023-08-25T03:34:08.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.324+0000 7f21abfff700 1 --2- 172.21.15.2:0/541595801 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f219806a0e0 0x7f219806c5a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f219c007fd0 tx=0x7f219c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:08.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.328+0000 7f21a8ff9700 1 -- 172.21.15.2:0/541595801 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f21a0053da0 con 0x7f21ac0caea0 2023-08-25T03:34:08.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.525+0000 7f21b2863700 1 -- 172.21.15.2:0/541595801 --> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f21ac15e370 con 0x7f219806a0e0 2023-08-25T03:34:08.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.532+0000 7f21a8ff9700 1 -- 172.21.15.2:0/541595801 <== mgr.4108 v2:172.21.15.2:6832/3528779779 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284114 (secure 0 0 0) 0x7f219c001cc0 con 0x7f219806a0e0 2023-08-25T03:34:08.539 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:08.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.535+0000 7f21b2863700 1 -- 172.21.15.2:0/541595801 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f219806a0e0 msgr2=0x7f219806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:08.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.535+0000 7f21b2863700 1 --2- 172.21.15.2:0/541595801 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f219806a0e0 0x7f219806c5a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f219c007fd0 tx=0x7f219c009040 comp rx=0 tx=0).stop 2023-08-25T03:34:08.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.535+0000 7f21b2863700 1 -- 172.21.15.2:0/541595801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21ac0caea0 msgr2=0x7f21ac147990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:08.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.535+0000 7f21b2863700 1 --2- 172.21.15.2:0/541595801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21ac0caea0 0x7f21ac147990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f21a000ebe0 tx=0x7f21a000eef0 comp rx=0 tx=0).stop 2023-08-25T03:34:08.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.535+0000 7f21b2863700 1 -- 172.21.15.2:0/541595801 shutdown_connections 2023-08-25T03:34:08.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.535+0000 7f21b2863700 1 --2- 172.21.15.2:0/541595801 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f21ac15e890 0x7f21ac0b4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:08.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.535+0000 7f21b2863700 1 --2- 172.21.15.2:0/541595801 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21ac0b19a0 0x7f21ac14ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:08.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.535+0000 7f21b2863700 1 --2- 172.21.15.2:0/541595801 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f219806a0e0 0x7f219806c5a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:08.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.535+0000 7f21b2863700 1 --2- 172.21.15.2:0/541595801 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21ac0caea0 0x7f21ac147990 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:08.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.535+0000 7f21b2863700 1 -- 172.21.15.2:0/541595801 >> 172.21.15.2:0/541595801 conn(0x7f21ac0cfea0 msgr2=0x7f21ac0734b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:08.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.536+0000 7f21b2863700 1 -- 172.21.15.2:0/541595801 shutdown_connections 2023-08-25T03:34:08.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.536+0000 7f21b2863700 1 -- 172.21.15.2:0/541595801 wait complete. 2023-08-25T03:34:08.545 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-08-25T03:34:08.549 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-08-25T03:34:07.504648+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":195,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219164,"kb_used_data":2972,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754755556,"statfs":{"total":773094113280,"available":772869689344,"internally_reserved":0,"allocated":3043328,"data_stored":1832772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.915745"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.584233+0000","last_change":"2023-08-25T03:34:02.584233+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.584233+0000","last_undegraded":"2023-08-25T03:34:02.584233+0000","last_fullsized":"2023-08-25T03:34:02.584233+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.620161+0000","last_change":"2023-08-25T03:34:03.616304+0000","last_active":"2023-08-25T03:34:03.620161+0000","last_peered":"2023-08-25T03:34:03.620161+0000","last_clean":"2023-08-25T03:34:03.620161+0000","last_became_active":"2023-08-25T03:34:03.615917+0000","last_became_peered":"2023-08-25T03:34:03.615917+0000","last_unstale":"2023-08-25T03:34:03.620161+0000","last_undegraded":"2023-08-25T03:34:03.620161+0000","last_fullsized":"2023-08-25T03:34:03.620161+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:25:43.507515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627607+0000","last_change":"2023-08-25T03:34:03.622799+0000","last_active":"2023-08-25T03:34:03.627607+0000","last_peered":"2023-08-25T03:34:03.627607+0000","last_clean":"2023-08-25T03:34:03.627607+0000","last_became_active":"2023-08-25T03:34:03.622602+0000","last_became_peered":"2023-08-25T03:34:03.622602+0000","last_unstale":"2023-08-25T03:34:03.627607+0000","last_undegraded":"2023-08-25T03:34:03.627607+0000","last_fullsized":"2023-08-25T03:34:03.627607+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:43:29.368622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.616911+0000","last_change":"2023-08-25T03:34:03.615254+0000","last_active":"2023-08-25T03:34:03.616911+0000","last_peered":"2023-08-25T03:34:03.616911+0000","last_clean":"2023-08-25T03:34:03.616911+0000","last_became_active":"2023-08-25T03:34:03.614917+0000","last_became_peered":"2023-08-25T03:34:03.614917+0000","last_unstale":"2023-08-25T03:34:03.616911+0000","last_undegraded":"2023-08-25T03:34:03.616911+0000","last_fullsized":"2023-08-25T03:34:03.616911+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:10:59.547904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.590306+0000","last_change":"2023-08-25T03:34:02.590306+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.590306+0000","last_undegraded":"2023-08-25T03:34:02.590306+0000","last_fullsized":"2023-08-25T03:34:02.590306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627201+0000","last_change":"2023-08-25T03:34:03.619323+0000","last_active":"2023-08-25T03:34:03.627201+0000","last_peered":"2023-08-25T03:34:03.627201+0000","last_clean":"2023-08-25T03:34:03.627201+0000","last_became_active":"2023-08-25T03:34:03.618985+0000","last_became_peered":"2023-08-25T03:34:03.618985+0000","last_unstale":"2023-08-25T03:34:03.627201+0000","last_undegraded":"2023-08-25T03:34:03.627201+0000","last_fullsized":"2023-08-25T03:34:03.627201+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:47:39.963816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.622126+0000","last_change":"2023-08-25T03:34:03.617493+0000","last_active":"2023-08-25T03:34:03.622126+0000","last_peered":"2023-08-25T03:34:03.622126+0000","last_clean":"2023-08-25T03:34:03.622126+0000","last_became_active":"2023-08-25T03:34:03.617167+0000","last_became_peered":"2023-08-25T03:34:03.617167+0000","last_unstale":"2023-08-25T03:34:03.622126+0000","last_undegraded":"2023-08-25T03:34:03.622126+0000","last_fullsized":"2023-08-25T03:34:03.622126+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:41:33.001585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.629535+0000","last_change":"2023-08-25T03:34:03.625710+0000","last_active":"2023-08-25T03:34:03.629535+0000","last_peered":"2023-08-25T03:34:03.629535+0000","last_clean":"2023-08-25T03:34:03.629535+0000","last_became_active":"2023-08-25T03:34:03.625404+0000","last_became_peered":"2023-08-25T03:34:03.625404+0000","last_unstale":"2023-08-25T03:34:03.629535+0000","last_undegraded":"2023-08-25T03:34:03.629535+0000","last_fullsized":"2023-08-25T03:34:03.629535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:15:37.568529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.593242+0000","last_change":"2023-08-25T03:34:02.593242+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.593242+0000","last_undegraded":"2023-08-25T03:34:02.593242+0000","last_fullsized":"2023-08-25T03:34:02.593242+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.620872+0000","last_change":"2023-08-25T03:34:03.618247+0000","last_active":"2023-08-25T03:34:03.620872+0000","last_peered":"2023-08-25T03:34:03.620872+0000","last_clean":"2023-08-25T03:34:03.620872+0000","last_became_active":"2023-08-25T03:34:03.617830+0000","last_became_peered":"2023-08-25T03:34:03.617830+0000","last_unstale":"2023-08-25T03:34:03.620872+0000","last_undegraded":"2023-08-25T03:34:03.620872+0000","last_fullsized":"2023-08-25T03:34:03.620872+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:47:39.963816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627422+0000","last_change":"2023-08-25T03:34:03.622380+0000","last_active":"2023-08-25T03:34:03.627422+0000","last_peered":"2023-08-25T03:34:03.627422+0000","last_clean":"2023-08-25T03:34:03.627422+0000","last_became_active":"2023-08-25T03:34:03.619691+0000","last_became_peered":"2023-08-25T03:34:03.619691+0000","last_unstale":"2023-08-25T03:34:03.627422+0000","last_undegraded":"2023-08-25T03:34:03.627422+0000","last_fullsized":"2023-08-25T03:34:03.627422+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:25:49.895284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.584558+0000","last_change":"2023-08-25T03:34:02.584558+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.584558+0000","last_undegraded":"2023-08-25T03:34:02.584558+0000","last_fullsized":"2023-08-25T03:34:02.584558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.584609+0000","last_change":"2023-08-25T03:34:02.584609+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.584609+0000","last_undegraded":"2023-08-25T03:34:02.584609+0000","last_fullsized":"2023-08-25T03:34:02.584609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.620519+0000","last_change":"2023-08-25T03:34:03.616557+0000","last_active":"2023-08-25T03:34:03.620519+0000","last_peered":"2023-08-25T03:34:03.620519+0000","last_clean":"2023-08-25T03:34:03.620519+0000","last_became_active":"2023-08-25T03:34:03.615850+0000","last_became_peered":"2023-08-25T03:34:03.615850+0000","last_unstale":"2023-08-25T03:34:03.620519+0000","last_undegraded":"2023-08-25T03:34:03.620519+0000","last_fullsized":"2023-08-25T03:34:03.620519+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:01:15.550047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.621835+0000","last_change":"2023-08-25T03:34:03.617319+0000","last_active":"2023-08-25T03:34:03.621835+0000","last_peered":"2023-08-25T03:34:03.621835+0000","last_clean":"2023-08-25T03:34:03.621835+0000","last_became_active":"2023-08-25T03:34:03.617012+0000","last_became_peered":"2023-08-25T03:34:03.617012+0000","last_unstale":"2023-08-25T03:34:03.621835+0000","last_undegraded":"2023-08-25T03:34:03.621835+0000","last_fullsized":"2023-08-25T03:34:03.621835+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:22:25.881167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.624247+0000","last_change":"2023-08-25T03:34:03.615370+0000","last_active":"2023-08-25T03:34:03.624247+0000","last_peered":"2023-08-25T03:34:03.624247+0000","last_clean":"2023-08-25T03:34:03.624247+0000","last_became_active":"2023-08-25T03:34:03.615076+0000","last_became_peered":"2023-08-25T03:34:03.615076+0000","last_unstale":"2023-08-25T03:34:03.624247+0000","last_undegraded":"2023-08-25T03:34:03.624247+0000","last_fullsized":"2023-08-25T03:34:03.624247+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:16:06.275512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591771+0000","last_change":"2023-08-25T03:34:02.591771+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591771+0000","last_undegraded":"2023-08-25T03:34:02.591771+0000","last_fullsized":"2023-08-25T03:34:02.591771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627116+0000","last_change":"2023-08-25T03:34:03.618966+0000","last_active":"2023-08-25T03:34:03.627116+0000","last_peered":"2023-08-25T03:34:03.627116+0000","last_clean":"2023-08-25T03:34:03.627116+0000","last_became_active":"2023-08-25T03:34:03.618636+0000","last_became_peered":"2023-08-25T03:34:03.618636+0000","last_unstale":"2023-08-25T03:34:03.627116+0000","last_undegraded":"2023-08-25T03:34:03.627116+0000","last_fullsized":"2023-08-25T03:34:03.627116+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:15:37.568529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.583704+0000","last_change":"2023-08-25T03:34:02.583704+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.583704+0000","last_undegraded":"2023-08-25T03:34:02.583704+0000","last_fullsized":"2023-08-25T03:34:02.583704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627305+0000","last_change":"2023-08-25T03:34:03.621319+0000","last_active":"2023-08-25T03:34:03.627305+0000","last_peered":"2023-08-25T03:34:03.627305+0000","last_clean":"2023-08-25T03:34:03.627305+0000","last_became_active":"2023-08-25T03:34:03.619115+0000","last_became_peered":"2023-08-25T03:34:03.619115+0000","last_unstale":"2023-08-25T03:34:03.627305+0000","last_undegraded":"2023-08-25T03:34:03.627305+0000","last_fullsized":"2023-08-25T03:34:03.627305+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:12:38.282041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627798+0000","last_change":"2023-08-25T03:34:03.622785+0000","last_active":"2023-08-25T03:34:03.627798+0000","last_peered":"2023-08-25T03:34:03.627798+0000","last_clean":"2023-08-25T03:34:03.627798+0000","last_became_active":"2023-08-25T03:34:03.622469+0000","last_became_peered":"2023-08-25T03:34:03.622469+0000","last_unstale":"2023-08-25T03:34:03.627798+0000","last_undegraded":"2023-08-25T03:34:03.627798+0000","last_fullsized":"2023-08-25T03:34:03.627798+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:47:13.833805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.589817+0000","last_change":"2023-08-25T03:34:02.589817+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.589817+0000","last_undegraded":"2023-08-25T03:34:02.589817+0000","last_fullsized":"2023-08-25T03:34:02.589817+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591327+0000","last_change":"2023-08-25T03:34:02.591327+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591327+0000","last_undegraded":"2023-08-25T03:34:02.591327+0000","last_fullsized":"2023-08-25T03:34:02.591327+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628155+0000","last_change":"2023-08-25T03:34:03.623150+0000","last_active":"2023-08-25T03:34:03.628155+0000","last_peered":"2023-08-25T03:34:03.628155+0000","last_clean":"2023-08-25T03:34:03.628155+0000","last_became_active":"2023-08-25T03:34:03.622796+0000","last_became_peered":"2023-08-25T03:34:03.622796+0000","last_unstale":"2023-08-25T03:34:03.628155+0000","last_undegraded":"2023-08-25T03:34:03.628155+0000","last_fullsized":"2023-08-25T03:34:03.628155+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:49:11.917056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.588252+0000","last_change":"2023-08-25T03:34:02.588252+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.588252+0000","last_undegraded":"2023-08-25T03:34:02.588252+0000","last_fullsized":"2023-08-25T03:34:02.588252+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.586918+0000","last_change":"2023-08-25T03:34:02.586918+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.586918+0000","last_undegraded":"2023-08-25T03:34:02.586918+0000","last_fullsized":"2023-08-25T03:34:02.586918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.616368+0000","last_change":"2023-08-25T03:34:03.613385+0000","last_active":"2023-08-25T03:34:03.616368+0000","last_peered":"2023-08-25T03:34:03.616368+0000","last_clean":"2023-08-25T03:34:03.616368+0000","last_became_active":"2023-08-25T03:34:03.613110+0000","last_became_peered":"2023-08-25T03:34:03.613110+0000","last_unstale":"2023-08-25T03:34:03.616368+0000","last_undegraded":"2023-08-25T03:34:03.616368+0000","last_fullsized":"2023-08-25T03:34:03.616368+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:47:27.124952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628064+0000","last_change":"2023-08-25T03:34:02.608606+0000","last_active":"2023-08-25T03:34:03.628064+0000","last_peered":"2023-08-25T03:34:03.628064+0000","last_clean":"2023-08-25T03:34:03.628064+0000","last_became_active":"2023-08-25T03:34:02.608298+0000","last_became_peered":"2023-08-25T03:34:02.608298+0000","last_unstale":"2023-08-25T03:34:03.628064+0000","last_undegraded":"2023-08-25T03:34:03.628064+0000","last_fullsized":"2023-08-25T03:34:03.628064+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:27:59.879524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591347+0000","last_change":"2023-08-25T03:34:02.591347+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591347+0000","last_undegraded":"2023-08-25T03:34:02.591347+0000","last_fullsized":"2023-08-25T03:34:02.591347+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.603129+0000","last_change":"2023-08-25T03:34:02.599366+0000","last_active":"2023-08-25T03:34:02.603129+0000","last_peered":"2023-08-25T03:34:02.603129+0000","last_clean":"2023-08-25T03:34:02.603129+0000","last_became_active":"2023-08-25T03:34:02.599105+0000","last_became_peered":"2023-08-25T03:34:02.599105+0000","last_unstale":"2023-08-25T03:34:02.603129+0000","last_undegraded":"2023-08-25T03:34:02.603129+0000","last_fullsized":"2023-08-25T03:34:02.603129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:38:11.493391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591521+0000","last_change":"2023-08-25T03:34:02.591521+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591521+0000","last_undegraded":"2023-08-25T03:34:02.591521+0000","last_fullsized":"2023-08-25T03:34:02.591521+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.609089+0000","last_change":"2023-08-25T03:34:02.608397+0000","last_active":"2023-08-25T03:34:02.609089+0000","last_peered":"2023-08-25T03:34:02.609089+0000","last_clean":"2023-08-25T03:34:02.609089+0000","last_became_active":"2023-08-25T03:34:02.608067+0000","last_became_peered":"2023-08-25T03:34:02.608067+0000","last_unstale":"2023-08-25T03:34:02.609089+0000","last_undegraded":"2023-08-25T03:34:02.609089+0000","last_fullsized":"2023-08-25T03:34:02.609089+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:14:44.918756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.616644+0000","last_change":"2023-08-25T03:34:03.613614+0000","last_active":"2023-08-25T03:34:03.616644+0000","last_peered":"2023-08-25T03:34:03.616644+0000","last_clean":"2023-08-25T03:34:03.616644+0000","last_became_active":"2023-08-25T03:34:03.613291+0000","last_became_peered":"2023-08-25T03:34:03.613291+0000","last_unstale":"2023-08-25T03:34:03.616644+0000","last_undegraded":"2023-08-25T03:34:03.616644+0000","last_fullsized":"2023-08-25T03:34:03.616644+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:43:35.780927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.605932+0000","last_change":"2023-08-25T03:34:02.603475+0000","last_active":"2023-08-25T03:34:02.605932+0000","last_peered":"2023-08-25T03:34:02.605932+0000","last_clean":"2023-08-25T03:34:02.605932+0000","last_became_active":"2023-08-25T03:34:02.603129+0000","last_became_peered":"2023-08-25T03:34:02.603129+0000","last_unstale":"2023-08-25T03:34:02.605932+0000","last_undegraded":"2023-08-25T03:34:02.605932+0000","last_fullsized":"2023-08-25T03:34:02.605932+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:10:58.540490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.590564+0000","last_change":"2023-08-25T03:34:02.590564+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.590564+0000","last_undegraded":"2023-08-25T03:34:02.590564+0000","last_fullsized":"2023-08-25T03:34:02.590564+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610665+0000","last_change":"2023-08-25T03:34:02.597247+0000","last_active":"2023-08-25T03:34:03.610665+0000","last_peered":"2023-08-25T03:34:03.610665+0000","last_clean":"2023-08-25T03:34:03.610665+0000","last_became_active":"2023-08-25T03:34:02.596968+0000","last_became_peered":"2023-08-25T03:34:02.596968+0000","last_unstale":"2023-08-25T03:34:03.610665+0000","last_undegraded":"2023-08-25T03:34:03.610665+0000","last_fullsized":"2023-08-25T03:34:03.610665+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:51:52.093732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.592296+0000","last_change":"2023-08-25T03:34:02.592296+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.592296+0000","last_undegraded":"2023-08-25T03:34:02.592296+0000","last_fullsized":"2023-08-25T03:34:02.592296+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.633428+0000","last_change":"2023-08-25T03:34:02.603880+0000","last_active":"2023-08-25T03:34:03.633428+0000","last_peered":"2023-08-25T03:34:03.633428+0000","last_clean":"2023-08-25T03:34:03.633428+0000","last_became_active":"2023-08-25T03:34:02.603502+0000","last_became_peered":"2023-08-25T03:34:02.603502+0000","last_unstale":"2023-08-25T03:34:03.633428+0000","last_undegraded":"2023-08-25T03:34:03.633428+0000","last_fullsized":"2023-08-25T03:34:03.633428+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:05:24.650744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.586286+0000","last_change":"2023-08-25T03:34:02.586286+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.586286+0000","last_undegraded":"2023-08-25T03:34:02.586286+0000","last_fullsized":"2023-08-25T03:34:02.586286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'6","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.632748+0000","last_change":"2023-08-25T03:34:02.606875+0000","last_active":"2023-08-25T03:34:03.632748+0000","last_peered":"2023-08-25T03:34:03.632748+0000","last_clean":"2023-08-25T03:34:03.632748+0000","last_became_active":"2023-08-25T03:34:02.606567+0000","last_became_peered":"2023-08-25T03:34:02.606567+0000","last_unstale":"2023-08-25T03:34:03.632748+0000","last_undegraded":"2023-08-25T03:34:03.632748+0000","last_fullsized":"2023-08-25T03:34:03.632748+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:51:52.093732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627947+0000","last_change":"2023-08-25T03:34:03.619847+0000","last_active":"2023-08-25T03:34:03.627947+0000","last_peered":"2023-08-25T03:34:03.627947+0000","last_clean":"2023-08-25T03:34:03.627947+0000","last_became_active":"2023-08-25T03:34:03.619046+0000","last_became_peered":"2023-08-25T03:34:03.619046+0000","last_unstale":"2023-08-25T03:34:03.627947+0000","last_undegraded":"2023-08-25T03:34:03.627947+0000","last_fullsized":"2023-08-25T03:34:03.627947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:52:42.938242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628937+0000","last_change":"2023-08-25T03:34:02.604424+0000","last_active":"2023-08-25T03:34:03.628937+0000","last_peered":"2023-08-25T03:34:03.628937+0000","last_clean":"2023-08-25T03:34:03.628937+0000","last_became_active":"2023-08-25T03:34:02.604175+0000","last_became_peered":"2023-08-25T03:34:02.604175+0000","last_unstale":"2023-08-25T03:34:03.628937+0000","last_undegraded":"2023-08-25T03:34:03.628937+0000","last_fullsized":"2023-08-25T03:34:03.628937+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:57.997213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591307+0000","last_change":"2023-08-25T03:34:02.591307+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591307+0000","last_undegraded":"2023-08-25T03:34:02.591307+0000","last_fullsized":"2023-08-25T03:34:02.591307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.607623+0000","last_change":"2023-08-25T03:34:02.606889+0000","last_active":"2023-08-25T03:34:02.607623+0000","last_peered":"2023-08-25T03:34:02.607623+0000","last_clean":"2023-08-25T03:34:02.607623+0000","last_became_active":"2023-08-25T03:34:02.606424+0000","last_became_peered":"2023-08-25T03:34:02.606424+0000","last_unstale":"2023-08-25T03:34:02.607623+0000","last_undegraded":"2023-08-25T03:34:02.607623+0000","last_fullsized":"2023-08-25T03:34:02.607623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:04:02.000324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.616388+0000","last_change":"2023-08-25T03:34:03.613888+0000","last_active":"2023-08-25T03:34:03.616388+0000","last_peered":"2023-08-25T03:34:03.616388+0000","last_clean":"2023-08-25T03:34:03.616388+0000","last_became_active":"2023-08-25T03:34:03.612829+0000","last_became_peered":"2023-08-25T03:34:03.612829+0000","last_unstale":"2023-08-25T03:34:03.616388+0000","last_undegraded":"2023-08-25T03:34:03.616388+0000","last_fullsized":"2023-08-25T03:34:03.616388+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:23:49.310992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610728+0000","last_change":"2023-08-25T03:34:02.606279+0000","last_active":"2023-08-25T03:34:03.610728+0000","last_peered":"2023-08-25T03:34:03.610728+0000","last_clean":"2023-08-25T03:34:03.610728+0000","last_became_active":"2023-08-25T03:34:02.606005+0000","last_became_peered":"2023-08-25T03:34:02.606005+0000","last_unstale":"2023-08-25T03:34:03.610728+0000","last_undegraded":"2023-08-25T03:34:03.610728+0000","last_fullsized":"2023-08-25T03:34:03.610728+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:02:01.407883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.592567+0000","last_change":"2023-08-25T03:34:02.592567+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.592567+0000","last_undegraded":"2023-08-25T03:34:02.592567+0000","last_fullsized":"2023-08-25T03:34:02.592567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.633646+0000","last_change":"2023-08-25T03:34:02.608430+0000","last_active":"2023-08-25T03:34:03.633646+0000","last_peered":"2023-08-25T03:34:03.633646+0000","last_clean":"2023-08-25T03:34:03.633646+0000","last_became_active":"2023-08-25T03:34:02.608086+0000","last_became_peered":"2023-08-25T03:34:02.608086+0000","last_unstale":"2023-08-25T03:34:03.633646+0000","last_undegraded":"2023-08-25T03:34:03.633646+0000","last_fullsized":"2023-08-25T03:34:03.633646+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:52:41.930828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591676+0000","last_change":"2023-08-25T03:34:02.591676+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591676+0000","last_undegraded":"2023-08-25T03:34:02.591676+0000","last_fullsized":"2023-08-25T03:34:02.591676+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628287+0000","last_change":"2023-08-25T03:34:02.604587+0000","last_active":"2023-08-25T03:34:03.628287+0000","last_peered":"2023-08-25T03:34:03.628287+0000","last_clean":"2023-08-25T03:34:03.628287+0000","last_became_active":"2023-08-25T03:34:02.604290+0000","last_became_peered":"2023-08-25T03:34:02.604290+0000","last_unstale":"2023-08-25T03:34:03.628287+0000","last_undegraded":"2023-08-25T03:34:03.628287+0000","last_fullsized":"2023-08-25T03:34:03.628287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:52:25.303751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.615963+0000","last_change":"2023-08-25T03:34:03.613959+0000","last_active":"2023-08-25T03:34:03.615963+0000","last_peered":"2023-08-25T03:34:03.615963+0000","last_clean":"2023-08-25T03:34:03.615963+0000","last_became_active":"2023-08-25T03:34:03.613667+0000","last_became_peered":"2023-08-25T03:34:03.613667+0000","last_unstale":"2023-08-25T03:34:03.615963+0000","last_undegraded":"2023-08-25T03:34:03.615963+0000","last_fullsized":"2023-08-25T03:34:03.615963+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:25:49.895284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628771+0000","last_change":"2023-08-25T03:34:02.604585+0000","last_active":"2023-08-25T03:34:03.628771+0000","last_peered":"2023-08-25T03:34:03.628771+0000","last_clean":"2023-08-25T03:34:03.628771+0000","last_became_active":"2023-08-25T03:34:02.604255+0000","last_became_peered":"2023-08-25T03:34:02.604255+0000","last_unstale":"2023-08-25T03:34:03.628771+0000","last_undegraded":"2023-08-25T03:34:03.628771+0000","last_fullsized":"2023-08-25T03:34:03.628771+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:07.999174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627819+0000","last_change":"2023-08-25T03:34:03.624731+0000","last_active":"2023-08-25T03:34:03.627819+0000","last_peered":"2023-08-25T03:34:03.627819+0000","last_clean":"2023-08-25T03:34:03.627819+0000","last_became_active":"2023-08-25T03:34:03.619454+0000","last_became_peered":"2023-08-25T03:34:03.619454+0000","last_unstale":"2023-08-25T03:34:03.627819+0000","last_undegraded":"2023-08-25T03:34:03.627819+0000","last_fullsized":"2023-08-25T03:34:03.627819+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:25:43.507515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.629966+0000","last_change":"2023-08-25T03:34:02.605137+0000","last_active":"2023-08-25T03:34:03.629966+0000","last_peered":"2023-08-25T03:34:03.629966+0000","last_clean":"2023-08-25T03:34:03.629966+0000","last_became_active":"2023-08-25T03:34:02.604846+0000","last_became_peered":"2023-08-25T03:34:02.604846+0000","last_unstale":"2023-08-25T03:34:03.629966+0000","last_undegraded":"2023-08-25T03:34:03.629966+0000","last_fullsized":"2023-08-25T03:34:03.629966+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:07.243584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.616584+0000","last_change":"2023-08-25T03:34:03.614885+0000","last_active":"2023-08-25T03:34:03.616584+0000","last_peered":"2023-08-25T03:34:03.616584+0000","last_clean":"2023-08-25T03:34:03.616584+0000","last_became_active":"2023-08-25T03:34:03.614420+0000","last_became_peered":"2023-08-25T03:34:03.614420+0000","last_unstale":"2023-08-25T03:34:03.616584+0000","last_undegraded":"2023-08-25T03:34:03.616584+0000","last_fullsized":"2023-08-25T03:34:03.616584+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:30:21.048549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610137+0000","last_change":"2023-08-25T03:34:02.598840+0000","last_active":"2023-08-25T03:34:03.610137+0000","last_peered":"2023-08-25T03:34:03.610137+0000","last_clean":"2023-08-25T03:34:03.610137+0000","last_became_active":"2023-08-25T03:34:02.598544+0000","last_became_peered":"2023-08-25T03:34:02.598544+0000","last_unstale":"2023-08-25T03:34:03.610137+0000","last_undegraded":"2023-08-25T03:34:03.610137+0000","last_fullsized":"2023-08-25T03:34:03.610137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:43:28.361208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591069+0000","last_change":"2023-08-25T03:34:02.591069+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591069+0000","last_undegraded":"2023-08-25T03:34:02.591069+0000","last_fullsized":"2023-08-25T03:34:02.591069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.605947+0000","last_change":"2023-08-25T03:34:02.605001+0000","last_active":"2023-08-25T03:34:02.605947+0000","last_peered":"2023-08-25T03:34:02.605947+0000","last_clean":"2023-08-25T03:34:02.605947+0000","last_became_active":"2023-08-25T03:34:02.604686+0000","last_became_peered":"2023-08-25T03:34:02.604686+0000","last_unstale":"2023-08-25T03:34:02.605947+0000","last_undegraded":"2023-08-25T03:34:02.605947+0000","last_fullsized":"2023-08-25T03:34:02.605947+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:04:02.000324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.629507+0000","last_change":"2023-08-25T03:34:03.626817+0000","last_active":"2023-08-25T03:34:03.629507+0000","last_peered":"2023-08-25T03:34:03.629507+0000","last_clean":"2023-08-25T03:34:03.629507+0000","last_became_active":"2023-08-25T03:34:03.626483+0000","last_became_peered":"2023-08-25T03:34:03.626483+0000","last_unstale":"2023-08-25T03:34:03.629507+0000","last_undegraded":"2023-08-25T03:34:03.629507+0000","last_fullsized":"2023-08-25T03:34:03.629507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:30:21.048549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.600756+0000","last_change":"2023-08-25T03:34:02.601925+0000","last_active":"2023-08-25T03:34:03.600756+0000","last_peered":"2023-08-25T03:34:03.600756+0000","last_clean":"2023-08-25T03:34:03.600756+0000","last_became_active":"2023-08-25T03:34:02.601552+0000","last_became_peered":"2023-08-25T03:34:02.601552+0000","last_unstale":"2023-08-25T03:34:03.600756+0000","last_undegraded":"2023-08-25T03:34:03.600756+0000","last_fullsized":"2023-08-25T03:34:03.600756+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:25:48.887870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.630046+0000","last_change":"2023-08-25T03:34:03.627975+0000","last_active":"2023-08-25T03:34:03.630046+0000","last_peered":"2023-08-25T03:34:03.630046+0000","last_clean":"2023-08-25T03:34:03.630046+0000","last_became_active":"2023-08-25T03:34:03.627489+0000","last_became_peered":"2023-08-25T03:34:03.627489+0000","last_unstale":"2023-08-25T03:34:03.630046+0000","last_undegraded":"2023-08-25T03:34:03.630046+0000","last_fullsized":"2023-08-25T03:34:03.630046+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:23:49.310992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.605294+0000","last_change":"2023-08-25T03:34:02.603347+0000","last_active":"2023-08-25T03:34:03.605294+0000","last_peered":"2023-08-25T03:34:03.605294+0000","last_clean":"2023-08-25T03:34:03.605294+0000","last_became_active":"2023-08-25T03:34:02.603010+0000","last_became_peered":"2023-08-25T03:34:02.603010+0000","last_unstale":"2023-08-25T03:34:03.605294+0000","last_undegraded":"2023-08-25T03:34:03.605294+0000","last_fullsized":"2023-08-25T03:34:03.605294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:38:55.054764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.626681+0000","last_change":"2023-08-25T03:34:03.616962+0000","last_active":"2023-08-25T03:34:03.626681+0000","last_peered":"2023-08-25T03:34:03.626681+0000","last_clean":"2023-08-25T03:34:03.626681+0000","last_became_active":"2023-08-25T03:34:03.616656+0000","last_became_peered":"2023-08-25T03:34:03.616656+0000","last_unstale":"2023-08-25T03:34:03.626681+0000","last_undegraded":"2023-08-25T03:34:03.626681+0000","last_fullsized":"2023-08-25T03:34:03.626681+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:44:42.715336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.606366+0000","last_change":"2023-08-25T03:34:02.603820+0000","last_active":"2023-08-25T03:34:02.606366+0000","last_peered":"2023-08-25T03:34:02.606366+0000","last_clean":"2023-08-25T03:34:02.606366+0000","last_became_active":"2023-08-25T03:34:02.603493+0000","last_became_peered":"2023-08-25T03:34:02.603493+0000","last_unstale":"2023-08-25T03:34:02.606366+0000","last_undegraded":"2023-08-25T03:34:02.606366+0000","last_fullsized":"2023-08-25T03:34:02.606366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:10:58.540490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.586628+0000","last_change":"2023-08-25T03:34:02.586628+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.586628+0000","last_undegraded":"2023-08-25T03:34:02.586628+0000","last_fullsized":"2023-08-25T03:34:02.586628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.590381+0000","last_change":"2023-08-25T03:33:57.570267+0000","last_active":"2023-08-25T03:34:02.590381+0000","last_peered":"2023-08-25T03:34:02.590381+0000","last_clean":"2023-08-25T03:34:02.590381+0000","last_became_active":"2023-08-25T03:33:57.569882+0000","last_became_peered":"2023-08-25T03:33:57.569882+0000","last_unstale":"2023-08-25T03:34:02.590381+0000","last_undegraded":"2023-08-25T03:34:02.590381+0000","last_fullsized":"2023-08-25T03:34:02.590381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:52:36.914859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610822+0000","last_change":"2023-08-25T03:34:02.600129+0000","last_active":"2023-08-25T03:34:03.610822+0000","last_peered":"2023-08-25T03:34:03.610822+0000","last_clean":"2023-08-25T03:34:03.610822+0000","last_became_active":"2023-08-25T03:34:02.599762+0000","last_became_peered":"2023-08-25T03:34:02.599762+0000","last_unstale":"2023-08-25T03:34:03.610822+0000","last_undegraded":"2023-08-25T03:34:03.610822+0000","last_fullsized":"2023-08-25T03:34:03.610822+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:57.997213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.584740+0000","last_change":"2023-08-25T03:34:02.584740+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.584740+0000","last_undegraded":"2023-08-25T03:34:02.584740+0000","last_fullsized":"2023-08-25T03:34:02.584740+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.609973+0000","last_change":"2023-08-25T03:33:57.569595+0000","last_active":"2023-08-25T03:34:03.609973+0000","last_peered":"2023-08-25T03:34:03.609973+0000","last_clean":"2023-08-25T03:34:03.609973+0000","last_became_active":"2023-08-25T03:33:57.569307+0000","last_became_peered":"2023-08-25T03:33:57.569307+0000","last_unstale":"2023-08-25T03:34:03.609973+0000","last_undegraded":"2023-08-25T03:34:03.609973+0000","last_fullsized":"2023-08-25T03:34:03.609973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:38:50.038795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.605192+0000","last_change":"2023-08-25T03:34:02.605404+0000","last_active":"2023-08-25T03:34:03.605192+0000","last_peered":"2023-08-25T03:34:03.605192+0000","last_clean":"2023-08-25T03:34:03.605192+0000","last_became_active":"2023-08-25T03:34:02.604806+0000","last_became_peered":"2023-08-25T03:34:02.604806+0000","last_unstale":"2023-08-25T03:34:03.605192+0000","last_undegraded":"2023-08-25T03:34:03.605192+0000","last_fullsized":"2023-08-25T03:34:03.605192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:41:01.441055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.585339+0000","last_change":"2023-08-25T03:34:02.585339+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.585339+0000","last_undegraded":"2023-08-25T03:34:02.585339+0000","last_fullsized":"2023-08-25T03:34:02.585339+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.585854+0000","last_change":"2023-08-25T03:33:57.886704+0000","last_active":"2023-08-25T03:34:02.585854+0000","last_peered":"2023-08-25T03:34:02.585854+0000","last_clean":"2023-08-25T03:34:02.585854+0000","last_became_active":"2023-08-25T03:33:57.886294+0000","last_became_peered":"2023-08-25T03:33:57.886294+0000","last_unstale":"2023-08-25T03:34:02.585854+0000","last_undegraded":"2023-08-25T03:34:02.585854+0000","last_fullsized":"2023-08-25T03:34:02.585854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:03:56.984355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.598074+0000","last_change":"2023-08-25T03:34:02.589214+0000","last_active":"2023-08-25T03:34:02.598074+0000","last_peered":"2023-08-25T03:34:02.598074+0000","last_clean":"2023-08-25T03:34:02.598074+0000","last_became_active":"2023-08-25T03:34:02.588607+0000","last_became_peered":"2023-08-25T03:34:02.588607+0000","last_unstale":"2023-08-25T03:34:02.598074+0000","last_undegraded":"2023-08-25T03:34:02.598074+0000","last_fullsized":"2023-08-25T03:34:02.598074+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:43:34.773513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.587185+0000","last_change":"2023-08-25T03:34:02.587185+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.587185+0000","last_undegraded":"2023-08-25T03:34:02.587185+0000","last_fullsized":"2023-08-25T03:34:02.587185+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610764+0000","last_change":"2023-08-25T03:33:57.570657+0000","last_active":"2023-08-25T03:34:03.610764+0000","last_peered":"2023-08-25T03:34:03.610764+0000","last_clean":"2023-08-25T03:34:03.610764+0000","last_became_active":"2023-08-25T03:33:57.570425+0000","last_became_peered":"2023-08-25T03:33:57.570425+0000","last_unstale":"2023-08-25T03:34:03.610764+0000","last_undegraded":"2023-08-25T03:34:03.610764+0000","last_fullsized":"2023-08-25T03:34:03.610764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:59:31.618299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610198+0000","last_change":"2023-08-25T03:33:51.259898+0000","last_active":"2023-08-25T03:34:03.610198+0000","last_peered":"2023-08-25T03:34:03.610198+0000","last_clean":"2023-08-25T03:34:03.610198+0000","last_became_active":"2023-08-25T03:33:51.258780+0000","last_became_peered":"2023-08-25T03:33:51.258780+0000","last_unstale":"2023-08-25T03:34:03.610198+0000","last_undegraded":"2023-08-25T03:34:03.610198+0000","last_fullsized":"2023-08-25T03:34:03.610198+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:49.666858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:49.666858+0000","last_clean_scrub_stamp":"2023-08-25T03:33:49.666858+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:51:40.190283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.605411+0000","last_change":"2023-08-25T03:34:02.598738+0000","last_active":"2023-08-25T03:34:02.605411+0000","last_peered":"2023-08-25T03:34:02.605411+0000","last_clean":"2023-08-25T03:34:02.605411+0000","last_became_active":"2023-08-25T03:34:02.598219+0000","last_became_peered":"2023-08-25T03:34:02.598219+0000","last_unstale":"2023-08-25T03:34:02.605411+0000","last_undegraded":"2023-08-25T03:34:02.605411+0000","last_fullsized":"2023-08-25T03:34:02.605411+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:54:25.036156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.626310+0000","last_change":"2023-08-25T03:34:03.624616+0000","last_active":"2023-08-25T03:34:03.626310+0000","last_peered":"2023-08-25T03:34:03.626310+0000","last_clean":"2023-08-25T03:34:03.626310+0000","last_became_active":"2023-08-25T03:34:03.624330+0000","last_became_peered":"2023-08-25T03:34:03.624330+0000","last_unstale":"2023-08-25T03:34:03.626310+0000","last_undegraded":"2023-08-25T03:34:03.626310+0000","last_fullsized":"2023-08-25T03:34:03.626310+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:44:42.715336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610373+0000","last_change":"2023-08-25T03:33:57.571821+0000","last_active":"2023-08-25T03:34:03.610373+0000","last_peered":"2023-08-25T03:34:03.610373+0000","last_clean":"2023-08-25T03:34:03.610373+0000","last_became_active":"2023-08-25T03:33:57.571528+0000","last_became_peered":"2023-08-25T03:33:57.571528+0000","last_unstale":"2023-08-25T03:34:03.610373+0000","last_undegraded":"2023-08-25T03:34:03.610373+0000","last_fullsized":"2023-08-25T03:34:03.610373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:59:31.618299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.604252+0000","last_change":"2023-08-25T03:34:02.602275+0000","last_active":"2023-08-25T03:34:02.604252+0000","last_peered":"2023-08-25T03:34:02.604252+0000","last_clean":"2023-08-25T03:34:02.604252+0000","last_became_active":"2023-08-25T03:34:02.601960+0000","last_became_peered":"2023-08-25T03:34:02.601960+0000","last_unstale":"2023-08-25T03:34:02.604252+0000","last_undegraded":"2023-08-25T03:34:02.604252+0000","last_fullsized":"2023-08-25T03:34:02.604252+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:14:44.918756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.590059+0000","last_change":"2023-08-25T03:34:02.590059+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.590059+0000","last_undegraded":"2023-08-25T03:34:02.590059+0000","last_fullsized":"2023-08-25T03:34:02.590059+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.611012+0000","last_change":"2023-08-25T03:33:57.571730+0000","last_active":"2023-08-25T03:34:03.611012+0000","last_peered":"2023-08-25T03:34:03.611012+0000","last_clean":"2023-08-25T03:34:03.611012+0000","last_became_active":"2023-08-25T03:33:57.571442+0000","last_became_peered":"2023-08-25T03:33:57.571442+0000","last_unstale":"2023-08-25T03:34:03.611012+0000","last_undegraded":"2023-08-25T03:34:03.611012+0000","last_fullsized":"2023-08-25T03:34:03.611012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:09:58.529844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.603118+0000","last_change":"2023-08-25T03:34:02.596876+0000","last_active":"2023-08-25T03:34:02.603118+0000","last_peered":"2023-08-25T03:34:02.603118+0000","last_clean":"2023-08-25T03:34:02.603118+0000","last_became_active":"2023-08-25T03:34:02.596556+0000","last_became_peered":"2023-08-25T03:34:02.596556+0000","last_unstale":"2023-08-25T03:34:02.603118+0000","last_undegraded":"2023-08-25T03:34:02.603118+0000","last_fullsized":"2023-08-25T03:34:02.603118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:12:37.274627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.589064+0000","last_change":"2023-08-25T03:34:02.589064+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.589064+0000","last_undegraded":"2023-08-25T03:34:02.589064+0000","last_fullsized":"2023-08-25T03:34:02.589064+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.605794+0000","last_change":"2023-08-25T03:34:02.604771+0000","last_active":"2023-08-25T03:34:02.605794+0000","last_peered":"2023-08-25T03:34:02.605794+0000","last_clean":"2023-08-25T03:34:02.605794+0000","last_became_active":"2023-08-25T03:34:02.604509+0000","last_became_peered":"2023-08-25T03:34:02.604509+0000","last_unstale":"2023-08-25T03:34:02.605794+0000","last_undegraded":"2023-08-25T03:34:02.605794+0000","last_fullsized":"2023-08-25T03:34:02.605794+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:16:05.268097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.590154+0000","last_change":"2023-08-25T03:33:57.570506+0000","last_active":"2023-08-25T03:34:02.590154+0000","last_peered":"2023-08-25T03:34:02.590154+0000","last_clean":"2023-08-25T03:34:02.590154+0000","last_became_active":"2023-08-25T03:33:57.570171+0000","last_became_peered":"2023-08-25T03:33:57.570171+0000","last_unstale":"2023-08-25T03:34:02.590154+0000","last_undegraded":"2023-08-25T03:34:02.590154+0000","last_fullsized":"2023-08-25T03:34:02.590154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:30:15.025166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.594873+0000","last_change":"2023-08-25T03:34:02.594873+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.594873+0000","last_undegraded":"2023-08-25T03:34:02.594873+0000","last_fullsized":"2023-08-25T03:34:02.594873+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.600483+0000","last_change":"2023-08-25T03:33:57.570598+0000","last_active":"2023-08-25T03:34:03.600483+0000","last_peered":"2023-08-25T03:34:03.600483+0000","last_clean":"2023-08-25T03:34:03.600483+0000","last_became_active":"2023-08-25T03:33:57.570215+0000","last_became_peered":"2023-08-25T03:33:57.570215+0000","last_unstale":"2023-08-25T03:34:03.600483+0000","last_undegraded":"2023-08-25T03:34:03.600483+0000","last_fullsized":"2023-08-25T03:34:03.600483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:12:32.258657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.606611+0000","last_change":"2023-08-25T03:34:02.605323+0000","last_active":"2023-08-25T03:34:02.606611+0000","last_peered":"2023-08-25T03:34:02.606611+0000","last_clean":"2023-08-25T03:34:02.606611+0000","last_became_active":"2023-08-25T03:34:02.605111+0000","last_became_peered":"2023-08-25T03:34:02.605111+0000","last_unstale":"2023-08-25T03:34:02.606611+0000","last_undegraded":"2023-08-25T03:34:02.606611+0000","last_fullsized":"2023-08-25T03:34:02.606611+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:44:41.707921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627797+0000","last_change":"2023-08-25T03:34:03.624353+0000","last_active":"2023-08-25T03:34:03.627797+0000","last_peered":"2023-08-25T03:34:03.627797+0000","last_clean":"2023-08-25T03:34:03.627797+0000","last_became_active":"2023-08-25T03:34:03.618468+0000","last_became_peered":"2023-08-25T03:34:03.618468+0000","last_unstale":"2023-08-25T03:34:03.627797+0000","last_undegraded":"2023-08-25T03:34:03.627797+0000","last_fullsized":"2023-08-25T03:34:03.627797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:38:12.500805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.605909+0000","last_change":"2023-08-25T03:34:02.603419+0000","last_active":"2023-08-25T03:34:02.605909+0000","last_peered":"2023-08-25T03:34:02.605909+0000","last_clean":"2023-08-25T03:34:02.605909+0000","last_became_active":"2023-08-25T03:34:02.603082+0000","last_became_peered":"2023-08-25T03:34:02.603082+0000","last_unstale":"2023-08-25T03:34:02.605909+0000","last_undegraded":"2023-08-25T03:34:02.605909+0000","last_fullsized":"2023-08-25T03:34:02.605909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:16:05.268097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.583635+0000","last_change":"2023-08-25T03:34:02.583635+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.583635+0000","last_undegraded":"2023-08-25T03:34:02.583635+0000","last_fullsized":"2023-08-25T03:34:02.583635+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.629772+0000","last_change":"2023-08-25T03:34:02.601201+0000","last_active":"2023-08-25T03:34:03.629772+0000","last_peered":"2023-08-25T03:34:03.629772+0000","last_clean":"2023-08-25T03:34:03.629772+0000","last_became_active":"2023-08-25T03:34:02.600904+0000","last_became_peered":"2023-08-25T03:34:02.600904+0000","last_unstale":"2023-08-25T03:34:03.629772+0000","last_undegraded":"2023-08-25T03:34:03.629772+0000","last_fullsized":"2023-08-25T03:34:03.629772+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:41:01.441055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.629645+0000","last_change":"2023-08-25T03:34:03.626366+0000","last_active":"2023-08-25T03:34:03.629645+0000","last_peered":"2023-08-25T03:34:03.629645+0000","last_clean":"2023-08-25T03:34:03.629645+0000","last_became_active":"2023-08-25T03:34:03.625496+0000","last_became_peered":"2023-08-25T03:34:03.625496+0000","last_unstale":"2023-08-25T03:34:03.629645+0000","last_undegraded":"2023-08-25T03:34:03.629645+0000","last_fullsized":"2023-08-25T03:34:03.629645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:01:15.550047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.629783+0000","last_change":"2023-08-25T03:34:02.603423+0000","last_active":"2023-08-25T03:34:03.629783+0000","last_peered":"2023-08-25T03:34:03.629783+0000","last_clean":"2023-08-25T03:34:03.629783+0000","last_became_active":"2023-08-25T03:34:02.603077+0000","last_became_peered":"2023-08-25T03:34:02.603077+0000","last_unstale":"2023-08-25T03:34:03.629783+0000","last_undegraded":"2023-08-25T03:34:03.629783+0000","last_fullsized":"2023-08-25T03:34:03.629783+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:27:59.879524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628025+0000","last_change":"2023-08-25T03:34:03.622038+0000","last_active":"2023-08-25T03:34:03.628025+0000","last_peered":"2023-08-25T03:34:03.628025+0000","last_clean":"2023-08-25T03:34:03.628025+0000","last_became_active":"2023-08-25T03:34:03.621747+0000","last_became_peered":"2023-08-25T03:34:03.621747+0000","last_unstale":"2023-08-25T03:34:03.628025+0000","last_undegraded":"2023-08-25T03:34:03.628025+0000","last_fullsized":"2023-08-25T03:34:03.628025+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:43:35.780927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.605434+0000","last_change":"2023-08-25T03:34:02.598272+0000","last_active":"2023-08-25T03:34:02.605434+0000","last_peered":"2023-08-25T03:34:02.605434+0000","last_clean":"2023-08-25T03:34:02.605434+0000","last_became_active":"2023-08-25T03:34:02.597630+0000","last_became_peered":"2023-08-25T03:34:02.597630+0000","last_unstale":"2023-08-25T03:34:02.605434+0000","last_undegraded":"2023-08-25T03:34:02.605434+0000","last_fullsized":"2023-08-25T03:34:02.605434+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:02:33.121969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.592797+0000","last_change":"2023-08-25T03:34:02.592797+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.592797+0000","last_undegraded":"2023-08-25T03:34:02.592797+0000","last_fullsized":"2023-08-25T03:34:02.592797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.604249+0000","last_change":"2023-08-25T03:34:02.597748+0000","last_active":"2023-08-25T03:34:02.604249+0000","last_peered":"2023-08-25T03:34:02.604249+0000","last_clean":"2023-08-25T03:34:02.604249+0000","last_became_active":"2023-08-25T03:34:02.597456+0000","last_became_peered":"2023-08-25T03:34:02.597456+0000","last_unstale":"2023-08-25T03:34:02.604249+0000","last_undegraded":"2023-08-25T03:34:02.604249+0000","last_fullsized":"2023-08-25T03:34:02.604249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:02:33.121969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627853+0000","last_change":"2023-08-25T03:34:03.619886+0000","last_active":"2023-08-25T03:34:03.627853+0000","last_peered":"2023-08-25T03:34:03.627853+0000","last_clean":"2023-08-25T03:34:03.627853+0000","last_became_active":"2023-08-25T03:34:03.618445+0000","last_became_peered":"2023-08-25T03:34:03.618445+0000","last_unstale":"2023-08-25T03:34:03.627853+0000","last_undegraded":"2023-08-25T03:34:03.627853+0000","last_fullsized":"2023-08-25T03:34:03.627853+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:10:59.547904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.616143+0000","last_change":"2023-08-25T03:34:03.612902+0000","last_active":"2023-08-25T03:34:03.616143+0000","last_peered":"2023-08-25T03:34:03.616143+0000","last_clean":"2023-08-25T03:34:03.616143+0000","last_became_active":"2023-08-25T03:34:03.612582+0000","last_became_peered":"2023-08-25T03:34:03.612582+0000","last_unstale":"2023-08-25T03:34:03.616143+0000","last_undegraded":"2023-08-25T03:34:03.616143+0000","last_fullsized":"2023-08-25T03:34:03.616143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:38:12.500805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.593258+0000","last_change":"2023-08-25T03:34:02.593258+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.593258+0000","last_undegraded":"2023-08-25T03:34:02.593258+0000","last_fullsized":"2023-08-25T03:34:02.593258+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628003+0000","last_change":"2023-08-25T03:34:03.622024+0000","last_active":"2023-08-25T03:34:03.628003+0000","last_peered":"2023-08-25T03:34:03.628003+0000","last_clean":"2023-08-25T03:34:03.628003+0000","last_became_active":"2023-08-25T03:34:03.619673+0000","last_became_peered":"2023-08-25T03:34:03.619673+0000","last_unstale":"2023-08-25T03:34:03.628003+0000","last_undegraded":"2023-08-25T03:34:03.628003+0000","last_fullsized":"2023-08-25T03:34:03.628003+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:47:27.124952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.589875+0000","last_change":"2023-08-25T03:34:02.589875+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.589875+0000","last_undegraded":"2023-08-25T03:34:02.589875+0000","last_fullsized":"2023-08-25T03:34:02.589875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.586986+0000","last_change":"2023-08-25T03:34:02.586986+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.586986+0000","last_undegraded":"2023-08-25T03:34:02.586986+0000","last_fullsized":"2023-08-25T03:34:02.586986+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":669737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]}]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":86497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":79369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.249}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":669737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":72749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":79369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":88165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":245760,"data_stored":87149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-25T03:34:08.559 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3f is not active+clean 2023-08-25T03:34:08.559 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.584233+0000', 'last_change': '2023-08-25T03:34:02.584233+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.584233+0000', 'last_undegraded': '2023-08-25T03:34:02.584233+0000', 'last_fullsized': '2023-08-25T03:34:02.584233+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-25T03:34:08.560 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3b is not active+clean 2023-08-25T03:34:08.560 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.590306+0000', 'last_change': '2023-08-25T03:34:02.590306+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.590306+0000', 'last_undegraded': '2023-08-25T03:34:02.590306+0000', 'last_fullsized': '2023-08-25T03:34:02.590306+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-25T03:34:08.561 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.37 is not active+clean 2023-08-25T03:34:08.561 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.593242+0000', 'last_change': '2023-08-25T03:34:02.593242+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.593242+0000', 'last_undegraded': '2023-08-25T03:34:02.593242+0000', 'last_fullsized': '2023-08-25T03:34:02.593242+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-25T03:34:08.562 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.34 is not active+clean 2023-08-25T03:34:08.562 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.584558+0000', 'last_change': '2023-08-25T03:34:02.584558+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.584558+0000', 'last_undegraded': '2023-08-25T03:34:02.584558+0000', 'last_fullsized': '2023-08-25T03:34:02.584558+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-25T03:34:08.563 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.33 is not active+clean 2023-08-25T03:34:08.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.584609+0000', 'last_change': '2023-08-25T03:34:02.584609+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.584609+0000', 'last_undegraded': '2023-08-25T03:34:02.584609+0000', 'last_fullsized': '2023-08-25T03:34:02.584609+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-25T03:34:08.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2f is not active+clean 2023-08-25T03:34:08.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.591771+0000', 'last_change': '2023-08-25T03:34:02.591771+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.591771+0000', 'last_undegraded': '2023-08-25T03:34:02.591771+0000', 'last_fullsized': '2023-08-25T03:34:02.591771+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-25T03:34:08.565 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2d is not active+clean 2023-08-25T03:34:08.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.583704+0000', 'last_change': '2023-08-25T03:34:02.583704+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.583704+0000', 'last_undegraded': '2023-08-25T03:34:02.583704+0000', 'last_fullsized': '2023-08-25T03:34:02.583704+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-25T03:34:08.565 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2a is not active+clean 2023-08-25T03:34:08.566 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.589817+0000', 'last_change': '2023-08-25T03:34:02.589817+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.589817+0000', 'last_undegraded': '2023-08-25T03:34:02.589817+0000', 'last_fullsized': '2023-08-25T03:34:02.589817+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-25T03:34:08.566 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.29 is not active+clean 2023-08-25T03:34:08.566 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.591327+0000', 'last_change': '2023-08-25T03:34:02.591327+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.591327+0000', 'last_undegraded': '2023-08-25T03:34:02.591327+0000', 'last_fullsized': '2023-08-25T03:34:02.591327+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-25T03:34:08.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.27 is not active+clean 2023-08-25T03:34:08.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.588252+0000', 'last_change': '2023-08-25T03:34:02.588252+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.588252+0000', 'last_undegraded': '2023-08-25T03:34:02.588252+0000', 'last_fullsized': '2023-08-25T03:34:02.588252+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-25T03:34:08.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.26 is not active+clean 2023-08-25T03:34:08.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.586918+0000', 'last_change': '2023-08-25T03:34:02.586918+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.586918+0000', 'last_undegraded': '2023-08-25T03:34:02.586918+0000', 'last_fullsized': '2023-08-25T03:34:02.586918+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-25T03:34:08.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1d is not active+clean 2023-08-25T03:34:08.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.591347+0000', 'last_change': '2023-08-25T03:34:02.591347+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.591347+0000', 'last_undegraded': '2023-08-25T03:34:02.591347+0000', 'last_fullsized': '2023-08-25T03:34:02.591347+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-25T03:34:08.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1e is not active+clean 2023-08-25T03:34:08.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.591521+0000', 'last_change': '2023-08-25T03:34:02.591521+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.591521+0000', 'last_undegraded': '2023-08-25T03:34:02.591521+0000', 'last_fullsized': '2023-08-25T03:34:02.591521+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-25T03:34:08.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.10 is not active+clean 2023-08-25T03:34:08.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.590564+0000', 'last_change': '2023-08-25T03:34:02.590564+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.590564+0000', 'last_undegraded': '2023-08-25T03:34:02.590564+0000', 'last_fullsized': '2023-08-25T03:34:02.590564+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-25T03:34:08.570 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.11 is not active+clean 2023-08-25T03:34:08.570 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.592296+0000', 'last_change': '2023-08-25T03:34:02.592296+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.592296+0000', 'last_undegraded': '2023-08-25T03:34:02.592296+0000', 'last_fullsized': '2023-08-25T03:34:02.592296+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-25T03:34:08.570 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.12 is not active+clean 2023-08-25T03:34:08.571 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.586286+0000', 'last_change': '2023-08-25T03:34:02.586286+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.586286+0000', 'last_undegraded': '2023-08-25T03:34:02.586286+0000', 'last_fullsized': '2023-08-25T03:34:02.586286+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-25T03:34:08.571 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.14 is not active+clean 2023-08-25T03:34:08.571 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.591307+0000', 'last_change': '2023-08-25T03:34:02.591307+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.591307+0000', 'last_undegraded': '2023-08-25T03:34:02.591307+0000', 'last_fullsized': '2023-08-25T03:34:02.591307+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-25T03:34:08.572 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.16 is not active+clean 2023-08-25T03:34:08.572 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.592567+0000', 'last_change': '2023-08-25T03:34:02.592567+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.592567+0000', 'last_undegraded': '2023-08-25T03:34:02.592567+0000', 'last_fullsized': '2023-08-25T03:34:02.592567+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-25T03:34:08.572 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.17 is not active+clean 2023-08-25T03:34:08.573 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.591676+0000', 'last_change': '2023-08-25T03:34:02.591676+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.591676+0000', 'last_undegraded': '2023-08-25T03:34:02.591676+0000', 'last_fullsized': '2023-08-25T03:34:02.591676+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-25T03:34:08.573 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.b is not active+clean 2023-08-25T03:34:08.573 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.591069+0000', 'last_change': '2023-08-25T03:34:02.591069+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.591069+0000', 'last_undegraded': '2023-08-25T03:34:02.591069+0000', 'last_fullsized': '2023-08-25T03:34:02.591069+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-25T03:34:08.573 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.f is not active+clean 2023-08-25T03:34:08.574 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.586628+0000', 'last_change': '2023-08-25T03:34:02.586628+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.586628+0000', 'last_undegraded': '2023-08-25T03:34:02.586628+0000', 'last_fullsized': '2023-08-25T03:34:02.586628+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-25T03:34:08.574 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2 is not active+clean 2023-08-25T03:34:08.574 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.584740+0000', 'last_change': '2023-08-25T03:34:02.584740+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.584740+0000', 'last_undegraded': '2023-08-25T03:34:02.584740+0000', 'last_fullsized': '2023-08-25T03:34:02.584740+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-25T03:34:08.574 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.4 is not active+clean 2023-08-25T03:34:08.575 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.585339+0000', 'last_change': '2023-08-25T03:34:02.585339+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.585339+0000', 'last_undegraded': '2023-08-25T03:34:02.585339+0000', 'last_fullsized': '2023-08-25T03:34:02.585339+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-25T03:34:08.575 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.7 is not active+clean 2023-08-25T03:34:08.575 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.587185+0000', 'last_change': '2023-08-25T03:34:02.587185+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.587185+0000', 'last_undegraded': '2023-08-25T03:34:02.587185+0000', 'last_fullsized': '2023-08-25T03:34:02.587185+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-25T03:34:08.576 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.6 is not active+clean 2023-08-25T03:34:08.576 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.590059+0000', 'last_change': '2023-08-25T03:34:02.590059+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.590059+0000', 'last_undegraded': '2023-08-25T03:34:02.590059+0000', 'last_fullsized': '2023-08-25T03:34:02.590059+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-25T03:34:08.576 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3 is not active+clean 2023-08-25T03:34:08.577 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.589064+0000', 'last_change': '2023-08-25T03:34:02.589064+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.589064+0000', 'last_undegraded': '2023-08-25T03:34:02.589064+0000', 'last_fullsized': '2023-08-25T03:34:02.589064+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-08-25T03:34:08.577 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.0 is not active+clean 2023-08-25T03:34:08.577 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.594873+0000', 'last_change': '2023-08-25T03:34:02.594873+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.594873+0000', 'last_undegraded': '2023-08-25T03:34:02.594873+0000', 'last_fullsized': '2023-08-25T03:34:02.594873+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-25T03:34:08.577 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1c is not active+clean 2023-08-25T03:34:08.578 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.583635+0000', 'last_change': '2023-08-25T03:34:02.583635+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.583635+0000', 'last_undegraded': '2023-08-25T03:34:02.583635+0000', 'last_fullsized': '2023-08-25T03:34:02.583635+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-08-25T03:34:08.578 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.19 is not active+clean 2023-08-25T03:34:08.578 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.592797+0000', 'last_change': '2023-08-25T03:34:02.592797+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.592797+0000', 'last_undegraded': '2023-08-25T03:34:02.592797+0000', 'last_fullsized': '2023-08-25T03:34:02.592797+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-25T03:34:08.579 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.21 is not active+clean 2023-08-25T03:34:08.579 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.593258+0000', 'last_change': '2023-08-25T03:34:02.593258+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.593258+0000', 'last_undegraded': '2023-08-25T03:34:02.593258+0000', 'last_fullsized': '2023-08-25T03:34:02.593258+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-25T03:34:08.579 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.23 is not active+clean 2023-08-25T03:34:08.580 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.589875+0000', 'last_change': '2023-08-25T03:34:02.589875+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.589875+0000', 'last_undegraded': '2023-08-25T03:34:02.589875+0000', 'last_fullsized': '2023-08-25T03:34:02.589875+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-08-25T03:34:08.580 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.24 is not active+clean 2023-08-25T03:34:08.580 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 22, 'state': 'creating+peering', 'last_fresh': '2023-08-25T03:34:02.586986+0000', 'last_change': '2023-08-25T03:34:02.586986+0000', 'last_active': '2023-08-25T03:34:02.577721+0000', 'last_peered': '2023-08-25T03:34:02.577721+0000', 'last_clean': '2023-08-25T03:34:02.577721+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-08-25T03:34:02.586986+0000', 'last_undegraded': '2023-08-25T03:34:02.586986+0000', 'last_fullsized': '2023-08-25T03:34:02.586986+0000', 'mapping_epoch': 22, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 22, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'last_clean_scrub_stamp': '2023-08-25T03:34:02.577721+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-08-25T03:34:08.581 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-08-25T03:34:08.693 INFO:tasks.ceph.mds.a.smithi002.stderr:2023-08-25T03:34:08.692+0000 7f6fd6ba1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-25T03:34:08.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.711+0000 7f504a374700 1 Processor -- start 2023-08-25T03:34:08.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.712+0000 7f504a374700 1 -- start start 2023-08-25T03:34:08.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.712+0000 7f504a374700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f50440cb5e0 0x7f504415b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:08.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.712+0000 7f504a374700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50440b39e0 con 0x7f504415b020 2023-08-25T03:34:08.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.712+0000 7f504a374700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f504415c650 con 0x7f50440d02f0 2023-08-25T03:34:08.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.712+0000 7f504a374700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504415bad0 con 0x7f50440cb5e0 2023-08-25T03:34:08.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.712+0000 7f5043fff700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f504415b020 0x7f50440cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45914/0 (socket says 172.21.15.2:45914) 2023-08-25T03:34:08.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.712+0000 7f5043fff700 1 -- 172.21.15.2:0/1341304143 learned_addr learned my addr 172.21.15.2:0/1341304143 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:08.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.713+0000 7f50437fe700 1 --2- 172.21.15.2:0/1341304143 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f50440cb5e0 0x7f504415b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:08.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.713+0000 7f5042ffd700 1 -- 172.21.15.2:0/1341304143 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 576255919 0 0) 0x7f50440b39e0 con 0x7f504415b020 2023-08-25T03:34:08.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.713+0000 7f5042ffd700 1 -- 172.21.15.2:0/1341304143 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50300035b0 con 0x7f504415b020 2023-08-25T03:34:08.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.713+0000 7f5042ffd700 1 -- 172.21.15.2:0/1341304143 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3489441467 0 0) 0x7f504415c650 con 0x7f50440d02f0 2023-08-25T03:34:08.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.713+0000 7f5042ffd700 1 -- 172.21.15.2:0/1341304143 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50440b39e0 con 0x7f50440d02f0 2023-08-25T03:34:08.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.713+0000 7f5042ffd700 1 -- 172.21.15.2:0/1341304143 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 675340150 0 0) 0x7f50300035b0 con 0x7f504415b020 2023-08-25T03:34:08.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.713+0000 7f5042ffd700 1 -- 172.21.15.2:0/1341304143 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f50440cb5e0 msgr2=0x7f504415b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:08.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.713+0000 7f5042ffd700 1 --2- 172.21.15.2:0/1341304143 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f50440cb5e0 0x7f504415b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:08.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.713+0000 7f5042ffd700 1 -- 172.21.15.2:0/1341304143 >> v1:172.21.15.31:6789/0 conn(0x7f50440d02f0 legacy=0x7f50440d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:08.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.713+0000 7f5042ffd700 1 -- 172.21.15.2:0/1341304143 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504415bdb0 con 0x7f504415b020 2023-08-25T03:34:08.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.713+0000 7f5042ffd700 1 -- 172.21.15.2:0/1341304143 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f502c001670 con 0x7f504415b020 2023-08-25T03:34:08.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.713+0000 7f5042ffd700 1 -- 172.21.15.2:0/1341304143 >> v1:172.21.15.2:6789/0 conn(0x7f504415b020 legacy=0x7f50440cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:08.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.713+0000 7f5042ffd700 1 --2- 172.21.15.2:0/1341304143 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f50440cb5e0 0x7f5030003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:08.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.713+0000 7f5042ffd700 1 --2- 172.21.15.2:0/1341304143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5030004170 0x7f5030004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:08.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.713+0000 7f5042ffd700 1 --2- 172.21.15.2:0/1341304143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5030004d90 0x7f5030005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:08.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.713+0000 7f5042ffd700 1 -- 172.21.15.2:0/1341304143 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504415bad0 con 0x7f5030004d90 2023-08-25T03:34:08.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.713+0000 7f5042ffd700 1 -- 172.21.15.2:0/1341304143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503000bb50 con 0x7f5030004170 2023-08-25T03:34:08.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.713+0000 7f5042ffd700 1 -- 172.21.15.2:0/1341304143 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f503000bd10 con 0x7f50440cb5e0 2023-08-25T03:34:08.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.713+0000 7f5048911700 1 --2- 172.21.15.2:0/1341304143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5030004d90 0x7f5030005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:08.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.713+0000 7f5043fff700 1 --2- 172.21.15.2:0/1341304143 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f50440cb5e0 0x7f5030003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:08.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.713+0000 7f50437fe700 1 --2- 172.21.15.2:0/1341304143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5030004170 0x7f5030004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:08.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.714+0000 7f5048911700 1 -- 172.21.15.2:0/1341304143 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f50440cb5e0 msgr2=0x7f5030003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:08.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.714+0000 7f5048911700 1 --2- 172.21.15.2:0/1341304143 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f50440cb5e0 0x7f5030003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:08.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.714+0000 7f5048911700 1 -- 172.21.15.2:0/1341304143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5030004170 msgr2=0x7f5030004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:08.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.714+0000 7f5048911700 1 --2- 172.21.15.2:0/1341304143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5030004170 0x7f5030004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:08.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.714+0000 7f5048911700 1 -- 172.21.15.2:0/1341304143 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50300035b0 con 0x7f5030004d90 2023-08-25T03:34:08.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.714+0000 7f5048911700 1 --2- 172.21.15.2:0/1341304143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5030004d90 0x7f5030005210 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f503c00c2f0 tx=0x7f5030005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=542a1d0de10becfa server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:08.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.714+0000 7f5042ffd700 1 -- 172.21.15.2:0/1341304143 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c001f30 con 0x7f5030004d90 2023-08-25T03:34:08.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.714+0000 7f5042ffd700 1 -- 172.21.15.2:0/1341304143 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503c00ce90 con 0x7f5030004d90 2023-08-25T03:34:08.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.714+0000 7f5042ffd700 1 -- 172.21.15.2:0/1341304143 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c003c40 con 0x7f5030004d90 2023-08-25T03:34:08.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.714+0000 7f504a374700 1 -- 172.21.15.2:0/1341304143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5030004d90 msgr2=0x7f5030005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:08.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.714+0000 7f504a374700 1 --2- 172.21.15.2:0/1341304143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5030004d90 0x7f5030005210 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f503c00c2f0 tx=0x7f5030005880 comp rx=0 tx=0).stop 2023-08-25T03:34:08.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.714+0000 7f5048911700 1 -- 172.21.15.2:0/1341304143 reap_dead start 2023-08-25T03:34:08.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.714+0000 7f504a374700 1 -- 172.21.15.2:0/1341304143 shutdown_connections 2023-08-25T03:34:08.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.714+0000 7f504a374700 1 -- 172.21.15.2:0/1341304143 >> 172.21.15.2:0/1341304143 conn(0x7f5044065fe0 msgr2=0x7f50440663c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:08.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.714+0000 7f504a374700 1 -- 172.21.15.2:0/1341304143 shutdown_connections 2023-08-25T03:34:08.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.715+0000 7f504a374700 1 -- 172.21.15.2:0/1341304143 wait complete. 2023-08-25T03:34:08.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.716+0000 7f504a374700 1 Processor -- start 2023-08-25T03:34:08.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.716+0000 7f504a374700 1 -- start start 2023-08-25T03:34:08.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.717+0000 7f504a374700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f504414a960 0x7f50440c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:08.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.717+0000 7f504a374700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50440c70a0 0x7f50440b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:08.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.717+0000 7f504a374700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50440b1fd0 0x7f504414e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:08.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.717+0000 7f504a374700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50440c5db0 con 0x7f50440b1fd0 2023-08-25T03:34:08.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.717+0000 7f504a374700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50440b2450 con 0x7f50440c70a0 2023-08-25T03:34:08.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.717+0000 7f504a374700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50440c4ac0 con 0x7f504414a960 2023-08-25T03:34:08.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.717+0000 7f5048911700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50440b1fd0 0x7f504414e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:08.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.717+0000 7f5048911700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50440b1fd0 0x7f504414e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59264/0 (socket says 172.21.15.2:59264) 2023-08-25T03:34:08.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.717+0000 7f5048911700 1 -- 172.21.15.2:0/1287236035 learned_addr learned my addr 172.21.15.2:0/1287236035 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:08.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.717+0000 7f5043fff700 1 --2- 172.21.15.2:0/1287236035 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f504414a960 0x7f50440c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:08.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.717+0000 7f50437fe700 1 --2- 172.21.15.2:0/1287236035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50440c70a0 0x7f50440b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:08.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.717+0000 7f5048911700 1 -- 172.21.15.2:0/1287236035 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f504414a960 msgr2=0x7f50440c6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:08.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.717+0000 7f5048911700 1 --2- 172.21.15.2:0/1287236035 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f504414a960 0x7f50440c6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:08.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.717+0000 7f5048911700 1 -- 172.21.15.2:0/1287236035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50440c70a0 msgr2=0x7f50440b1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:08.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.717+0000 7f5048911700 1 --2- 172.21.15.2:0/1287236035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50440c70a0 0x7f50440b1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:08.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.717+0000 7f5048911700 1 -- 172.21.15.2:0/1287236035 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50440c4d10 con 0x7f50440b1fd0 2023-08-25T03:34:08.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.717+0000 7f5048911700 1 --2- 172.21.15.2:0/1287236035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50440b1fd0 0x7f504414e7c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f503c004000 tx=0x7f503c001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:08.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.717+0000 7f5040ff9700 1 -- 172.21.15.2:0/1287236035 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c0050d0 con 0x7f50440b1fd0 2023-08-25T03:34:08.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.717+0000 7f5040ff9700 1 -- 172.21.15.2:0/1287236035 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f503c005250 con 0x7f50440b1fd0 2023-08-25T03:34:08.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.717+0000 7f504a374700 1 -- 172.21.15.2:0/1287236035 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50440c4ff0 con 0x7f50440b1fd0 2023-08-25T03:34:08.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.717+0000 7f5040ff9700 1 -- 172.21.15.2:0/1287236035 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f503c005600 con 0x7f50440b1fd0 2023-08-25T03:34:08.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.718+0000 7f504a374700 1 -- 172.21.15.2:0/1287236035 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50440c55b0 con 0x7f50440b1fd0 2023-08-25T03:34:08.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.718+0000 7f504a374700 1 -- 172.21.15.2:0/1287236035 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50440c7480 con 0x7f50440b1fd0 2023-08-25T03:34:08.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.719+0000 7f5040ff9700 1 -- 172.21.15.2:0/1287236035 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f503c019040 con 0x7f50440b1fd0 2023-08-25T03:34:08.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.719+0000 7f5040ff9700 1 --2- 172.21.15.2:0/1287236035 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f502406a0e0 0x7f502406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:08.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.719+0000 7f5043fff700 1 --2- 172.21.15.2:0/1287236035 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f502406a0e0 0x7f502406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:08.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.719+0000 7f5040ff9700 1 -- 172.21.15.2:0/1287236035 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f503c089430 con 0x7f50440b1fd0 2023-08-25T03:34:08.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.720+0000 7f5043fff700 1 --2- 172.21.15.2:0/1287236035 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f502406a0e0 0x7f502406c5a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f502c000c00 tx=0x7f502c001af0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:08.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:08.724+0000 7f5040ff9700 1 -- 172.21.15.2:0/1287236035 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f503c054d90 con 0x7f50440b1fd0 2023-08-25T03:34:09.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.006+0000 7f504a374700 1 -- 172.21.15.2:0/1287236035 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f504406f4a0 con 0x7f50440b1fd0 2023-08-25T03:34:09.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.007+0000 7f5040ff9700 1 -- 172.21.15.2:0/1287236035 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1097 (secure 0 0 0) 0x7f503c0011c0 con 0x7f50440b1fd0 2023-08-25T03:34:09.008 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:09.008 INFO:teuthology.orchestra.run.smithi002.stdout:{"fsid":"dc54136d-3683-431b-8a68-994ec618505c","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":31,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":24,"num_osds":8,"num_up_osds":8,"osd_up_since":1692934428,"num_in_osds":8,"osd_in_since":1692934420,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":73},{"state_name":"creating+peering","count":32}],"num_pgs":105,"num_pools":4,"num_objects":19,"data_bytes":592196,"bytes_used":224423936,"bytes_avail":772869689344,"bytes_total":773094113280,"inactive_pgs_ratio":0.30476191639900208,"write_bytes_sec":2250,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":8,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"a","status":"up:active","gid":4299}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2023-08-25T03:33:37.641111+0000","services":{}},"progress_events":{}} 2023-08-25T03:34:09.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.009+0000 7f504a374700 1 -- 172.21.15.2:0/1287236035 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f502406a0e0 msgr2=0x7f502406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:09.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.009+0000 7f504a374700 1 --2- 172.21.15.2:0/1287236035 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f502406a0e0 0x7f502406c5a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f502c000c00 tx=0x7f502c001af0 comp rx=0 tx=0).stop 2023-08-25T03:34:09.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.009+0000 7f504a374700 1 -- 172.21.15.2:0/1287236035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50440b1fd0 msgr2=0x7f504414e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:09.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.009+0000 7f504a374700 1 --2- 172.21.15.2:0/1287236035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50440b1fd0 0x7f504414e7c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f503c004000 tx=0x7f503c001650 comp rx=0 tx=0).stop 2023-08-25T03:34:09.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.009+0000 7f504a374700 1 -- 172.21.15.2:0/1287236035 shutdown_connections 2023-08-25T03:34:09.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.009+0000 7f504a374700 1 --2- 172.21.15.2:0/1287236035 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f502406a0e0 0x7f502406c5a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:09.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.009+0000 7f504a374700 1 --2- 172.21.15.2:0/1287236035 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50440b1fd0 0x7f504414e7c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:09.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.009+0000 7f504a374700 1 --2- 172.21.15.2:0/1287236035 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f504414a960 0x7f50440c6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:09.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.009+0000 7f504a374700 1 --2- 172.21.15.2:0/1287236035 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50440c70a0 0x7f50440b1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:09.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.009+0000 7f504a374700 1 -- 172.21.15.2:0/1287236035 >> 172.21.15.2:0/1287236035 conn(0x7f5044065fe0 msgr2=0x7f50440cd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:09.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.009+0000 7f504a374700 1 -- 172.21.15.2:0/1287236035 shutdown_connections 2023-08-25T03:34:09.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.009+0000 7f504a374700 1 -- 172.21.15.2:0/1287236035 wait complete. 2023-08-25T03:34:09.022 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-08-25T03:34:09.022 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T03:34:09.146 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.145+0000 7f3326023700 1 Processor -- start 2023-08-25T03:34:09.147 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.145+0000 7f3326023700 1 -- start start 2023-08-25T03:34:09.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.146+0000 7f3326023700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f33200cb5a0 0x7f33200ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:09.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.146+0000 7f3326023700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33200b36a0 con 0x7f33200cd480 2023-08-25T03:34:09.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.146+0000 7f3326023700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33200b2a90 con 0x7f332015b060 2023-08-25T03:34:09.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.146+0000 7f3326023700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33200b6b30 con 0x7f33200cb5a0 2023-08-25T03:34:09.150 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.146+0000 7f331ffff700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f33200cd480 0x7f332006dff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45938/0 (socket says 172.21.15.2:45938) 2023-08-25T03:34:09.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.146+0000 7f331ffff700 1 -- 172.21.15.2:0/2998521675 learned_addr learned my addr 172.21.15.2:0/2998521675 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:09.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.147+0000 7f331effd700 1 --2- 172.21.15.2:0/2998521675 >> v2:172.21.15.31:3300/0 conn(0x7f33200cb5a0 0x7f33200ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:09.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.147+0000 7f331e7fc700 1 -- 172.21.15.2:0/2998521675 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1644293676 0 0) 0x7f33200b36a0 con 0x7f33200cd480 2023-08-25T03:34:09.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.147+0000 7f331e7fc700 1 -- 172.21.15.2:0/2998521675 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f330c0035b0 con 0x7f33200cd480 2023-08-25T03:34:09.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.147+0000 7f331e7fc700 1 -- 172.21.15.2:0/2998521675 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2518437622 0 0) 0x7f33200b2a90 con 0x7f332015b060 2023-08-25T03:34:09.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.147+0000 7f331e7fc700 1 -- 172.21.15.2:0/2998521675 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33200b36a0 con 0x7f332015b060 2023-08-25T03:34:09.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.147+0000 7f331e7fc700 1 -- 172.21.15.2:0/2998521675 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3584584087 0 0) 0x7f330c0035b0 con 0x7f33200cd480 2023-08-25T03:34:09.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.147+0000 7f331e7fc700 1 -- 172.21.15.2:0/2998521675 >> v2:172.21.15.31:3300/0 conn(0x7f33200cb5a0 msgr2=0x7f33200ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:09.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.147+0000 7f331e7fc700 1 --2- 172.21.15.2:0/2998521675 >> v2:172.21.15.31:3300/0 conn(0x7f33200cb5a0 0x7f33200ccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:09.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.147+0000 7f331e7fc700 1 -- 172.21.15.2:0/2998521675 >> v1:172.21.15.31:6789/0 conn(0x7f332015b060 legacy=0x7f332015b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:09.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.147+0000 7f331e7fc700 1 -- 172.21.15.2:0/2998521675 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33200b2d70 con 0x7f33200cd480 2023-08-25T03:34:09.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.147+0000 7f331e7fc700 1 -- 172.21.15.2:0/2998521675 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f33180019f0 con 0x7f33200cd480 2023-08-25T03:34:09.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.147+0000 7f331effd700 1 --2- 172.21.15.2:0/2998521675 >> v2:172.21.15.31:3300/0 conn(0x7f33200cb5a0 0x7f33200ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:34:09.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.147+0000 7f331e7fc700 1 -- 172.21.15.2:0/2998521675 >> v1:172.21.15.2:6789/0 conn(0x7f33200cd480 legacy=0x7f332006dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:09.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.147+0000 7f331e7fc700 1 --2- 172.21.15.2:0/2998521675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f330c003b90 0x7f330c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:09.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.147+0000 7f331e7fc700 1 --2- 172.21.15.2:0/2998521675 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f330c004850 0x7f330c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:09.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.147+0000 7f331e7fc700 1 --2- 172.21.15.2:0/2998521675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f330c005210 0x7f330c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:09.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.147+0000 7f331e7fc700 1 -- 172.21.15.2:0/2998521675 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33200b6b30 con 0x7f330c005210 2023-08-25T03:34:09.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.147+0000 7f331e7fc700 1 -- 172.21.15.2:0/2998521675 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330c00c140 con 0x7f330c003b90 2023-08-25T03:34:09.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.147+0000 7f331e7fc700 1 -- 172.21.15.2:0/2998521675 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f330c00c300 con 0x7f330c004850 2023-08-25T03:34:09.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.147+0000 7f331ffff700 1 --2- 172.21.15.2:0/2998521675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f330c005210 0x7f330c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:09.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.147+0000 7f331f7fe700 1 --2- 172.21.15.2:0/2998521675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f330c003b90 0x7f330c003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:09.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.147+0000 7f331effd700 1 --2- 172.21.15.2:0/2998521675 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f330c004850 0x7f330c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:09.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.147+0000 7f331ffff700 1 -- 172.21.15.2:0/2998521675 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f330c004850 msgr2=0x7f330c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:09.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.147+0000 7f331ffff700 1 --2- 172.21.15.2:0/2998521675 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f330c004850 0x7f330c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:09.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.147+0000 7f331ffff700 1 -- 172.21.15.2:0/2998521675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f330c003b90 msgr2=0x7f330c003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:09.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.147+0000 7f331ffff700 1 --2- 172.21.15.2:0/2998521675 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f330c003b90 0x7f330c003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:09.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.148+0000 7f331ffff700 1 -- 172.21.15.2:0/2998521675 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f330c0035b0 con 0x7f330c005210 2023-08-25T03:34:09.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.148+0000 7f331ffff700 1 -- 172.21.15.2:0/2998521675 reap_dead start 2023-08-25T03:34:09.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.148+0000 7f331ffff700 1 --2- 172.21.15.2:0/2998521675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f330c005210 0x7f330c00ba10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f331800a7a0 tx=0x7f330c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835f633f495e3f82 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:09.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.148+0000 7f331e7fc700 1 -- 172.21.15.2:0/2998521675 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3318012600 con 0x7f330c005210 2023-08-25T03:34:09.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.148+0000 7f331e7fc700 1 -- 172.21.15.2:0/2998521675 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3318012780 con 0x7f330c005210 2023-08-25T03:34:09.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.148+0000 7f331e7fc700 1 -- 172.21.15.2:0/2998521675 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3318012b30 con 0x7f330c005210 2023-08-25T03:34:09.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.148+0000 7f3326023700 1 -- 172.21.15.2:0/2998521675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f330c005210 msgr2=0x7f330c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:09.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.148+0000 7f3326023700 1 --2- 172.21.15.2:0/2998521675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f330c005210 0x7f330c00ba10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f331800a7a0 tx=0x7f330c005880 comp rx=0 tx=0).stop 2023-08-25T03:34:09.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.148+0000 7f3326023700 1 -- 172.21.15.2:0/2998521675 shutdown_connections 2023-08-25T03:34:09.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.148+0000 7f3326023700 1 --2- 172.21.15.2:0/2998521675 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f330c005210 0x7f330c00ba10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:09.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.148+0000 7f3326023700 1 -- 172.21.15.2:0/2998521675 >> 172.21.15.2:0/2998521675 conn(0x7f33200d0f70 msgr2=0x7f33200d1350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:09.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.148+0000 7f3326023700 1 -- 172.21.15.2:0/2998521675 shutdown_connections 2023-08-25T03:34:09.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.148+0000 7f3326023700 1 -- 172.21.15.2:0/2998521675 wait complete. 2023-08-25T03:34:09.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.150+0000 7f3326023700 1 Processor -- start 2023-08-25T03:34:09.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.150+0000 7f3326023700 1 -- start start 2023-08-25T03:34:09.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.151+0000 7f3326023700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f330c005210 0x7f33200b3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:09.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.151+0000 7f3326023700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33200afd40 0x7f33200ced20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:09.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.151+0000 7f3326023700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f332006c630 0x7f332006cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:09.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.151+0000 7f3326023700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33200ca9a0 con 0x7f330c005210 2023-08-25T03:34:09.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.151+0000 7f3326023700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f332006c1a0 con 0x7f33200afd40 2023-08-25T03:34:09.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.151+0000 7f3326023700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33200ca3e0 con 0x7f332006c630 2023-08-25T03:34:09.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.151+0000 7f331f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f330c005210 0x7f33200b3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:09.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.151+0000 7f331f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f330c005210 0x7f33200b3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59280/0 (socket says 172.21.15.2:59280) 2023-08-25T03:34:09.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.151+0000 7f331f7fe700 1 -- 172.21.15.2:0/2796717642 learned_addr learned my addr 172.21.15.2:0/2796717642 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:09.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.151+0000 7f331effd700 1 --2- 172.21.15.2:0/2796717642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33200afd40 0x7f33200ced20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:09.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.151+0000 7f331ffff700 1 --2- 172.21.15.2:0/2796717642 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f332006c630 0x7f332006cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:09.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.152+0000 7f331f7fe700 1 -- 172.21.15.2:0/2796717642 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f332006c630 msgr2=0x7f332006cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:09.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.152+0000 7f331f7fe700 1 --2- 172.21.15.2:0/2796717642 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f332006c630 0x7f332006cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:09.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.152+0000 7f331f7fe700 1 -- 172.21.15.2:0/2796717642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33200afd40 msgr2=0x7f33200ced20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:09.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.152+0000 7f331f7fe700 1 --2- 172.21.15.2:0/2796717642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33200afd40 0x7f33200ced20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:09.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.152+0000 7f331f7fe700 1 -- 172.21.15.2:0/2796717642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33200ca6c0 con 0x7f330c005210 2023-08-25T03:34:09.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.152+0000 7f331ffff700 1 --2- 172.21.15.2:0/2796717642 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f332006c630 0x7f332006cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:09.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.152+0000 7f331f7fe700 1 --2- 172.21.15.2:0/2796717642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f330c005210 0x7f33200b3b30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f331000f7a0 tx=0x7f331000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:09.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.152+0000 7f3317fff700 1 -- 172.21.15.2:0/2796717642 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3310005320 con 0x7f330c005210 2023-08-25T03:34:09.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.152+0000 7f3317fff700 1 -- 172.21.15.2:0/2796717642 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33100054a0 con 0x7f330c005210 2023-08-25T03:34:09.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.152+0000 7f3317fff700 1 -- 172.21.15.2:0/2796717642 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3310006aa0 con 0x7f330c005210 2023-08-25T03:34:09.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.152+0000 7f3326023700 1 -- 172.21.15.2:0/2796717642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f332006be30 con 0x7f330c005210 2023-08-25T03:34:09.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.152+0000 7f3326023700 1 -- 172.21.15.2:0/2796717642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33200b01c0 con 0x7f330c005210 2023-08-25T03:34:09.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.153+0000 7f3326023700 1 -- 172.21.15.2:0/2796717642 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3320070120 con 0x7f330c005210 2023-08-25T03:34:09.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.153+0000 7f3317fff700 1 -- 172.21.15.2:0/2796717642 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3310006c00 con 0x7f330c005210 2023-08-25T03:34:09.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.154+0000 7f3317fff700 1 --2- 172.21.15.2:0/2796717642 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f330006a0e0 0x7f330006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:09.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.154+0000 7f331effd700 1 --2- 172.21.15.2:0/2796717642 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f330006a0e0 0x7f330006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:09.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.154+0000 7f3317fff700 1 -- 172.21.15.2:0/2796717642 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f33100879a0 con 0x7f330c005210 2023-08-25T03:34:09.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.154+0000 7f331effd700 1 --2- 172.21.15.2:0/2796717642 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f330006a0e0 0x7f330006c5a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f33201596b0 tx=0x7f3308009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:09.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.159+0000 7f3317fff700 1 -- 172.21.15.2:0/2796717642 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3310053120 con 0x7f330c005210 2023-08-25T03:34:09.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.357+0000 7f3326023700 1 -- 172.21.15.2:0/2796717642 --> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f33200b3330 con 0x7f330006a0e0 2023-08-25T03:34:09.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.365+0000 7f3317fff700 1 -- 172.21.15.2:0/2796717642 <== mgr.4108 v2:172.21.15.2:6832/3528779779 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+284114 (secure 0 0 0) 0x7f3308019070 con 0x7f330006a0e0 2023-08-25T03:34:09.369 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:09.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.368+0000 7f3326023700 1 -- 172.21.15.2:0/2796717642 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f330006a0e0 msgr2=0x7f330006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:09.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.368+0000 7f3326023700 1 --2- 172.21.15.2:0/2796717642 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f330006a0e0 0x7f330006c5a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f33201596b0 tx=0x7f3308009210 comp rx=0 tx=0).stop 2023-08-25T03:34:09.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.368+0000 7f3326023700 1 -- 172.21.15.2:0/2796717642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f330c005210 msgr2=0x7f33200b3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:09.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.368+0000 7f3326023700 1 --2- 172.21.15.2:0/2796717642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f330c005210 0x7f33200b3b30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f331000f7a0 tx=0x7f331000fab0 comp rx=0 tx=0).stop 2023-08-25T03:34:09.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.368+0000 7f3326023700 1 -- 172.21.15.2:0/2796717642 shutdown_connections 2023-08-25T03:34:09.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.368+0000 7f3326023700 1 --2- 172.21.15.2:0/2796717642 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f330006a0e0 0x7f330006c5a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:09.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.368+0000 7f3326023700 1 --2- 172.21.15.2:0/2796717642 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f330c005210 0x7f33200b3b30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:09.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.368+0000 7f3326023700 1 --2- 172.21.15.2:0/2796717642 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f332006c630 0x7f332006cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:09.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.368+0000 7f3326023700 1 --2- 172.21.15.2:0/2796717642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f33200afd40 0x7f33200ced20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:09.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.368+0000 7f3326023700 1 -- 172.21.15.2:0/2796717642 >> 172.21.15.2:0/2796717642 conn(0x7f33200d0f70 msgr2=0x7f33200b20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:09.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.369+0000 7f3326023700 1 -- 172.21.15.2:0/2796717642 shutdown_connections 2023-08-25T03:34:09.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:09.369+0000 7f3326023700 1 -- 172.21.15.2:0/2796717642 wait complete. 2023-08-25T03:34:09.372 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-08-25T03:34:09.381 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-08-25T03:34:07.504648+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592196,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":195,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219164,"kb_used_data":2972,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754755556,"statfs":{"total":773094113280,"available":772869689344,"internally_reserved":0,"allocated":3043328,"data_stored":1832772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1828,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.915745"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.584233+0000","last_change":"2023-08-25T03:34:02.584233+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.584233+0000","last_undegraded":"2023-08-25T03:34:02.584233+0000","last_fullsized":"2023-08-25T03:34:02.584233+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.620161+0000","last_change":"2023-08-25T03:34:03.616304+0000","last_active":"2023-08-25T03:34:03.620161+0000","last_peered":"2023-08-25T03:34:03.620161+0000","last_clean":"2023-08-25T03:34:03.620161+0000","last_became_active":"2023-08-25T03:34:03.615917+0000","last_became_peered":"2023-08-25T03:34:03.615917+0000","last_unstale":"2023-08-25T03:34:03.620161+0000","last_undegraded":"2023-08-25T03:34:03.620161+0000","last_fullsized":"2023-08-25T03:34:03.620161+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:25:43.507515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627607+0000","last_change":"2023-08-25T03:34:03.622799+0000","last_active":"2023-08-25T03:34:03.627607+0000","last_peered":"2023-08-25T03:34:03.627607+0000","last_clean":"2023-08-25T03:34:03.627607+0000","last_became_active":"2023-08-25T03:34:03.622602+0000","last_became_peered":"2023-08-25T03:34:03.622602+0000","last_unstale":"2023-08-25T03:34:03.627607+0000","last_undegraded":"2023-08-25T03:34:03.627607+0000","last_fullsized":"2023-08-25T03:34:03.627607+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:43:29.368622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.616911+0000","last_change":"2023-08-25T03:34:03.615254+0000","last_active":"2023-08-25T03:34:03.616911+0000","last_peered":"2023-08-25T03:34:03.616911+0000","last_clean":"2023-08-25T03:34:03.616911+0000","last_became_active":"2023-08-25T03:34:03.614917+0000","last_became_peered":"2023-08-25T03:34:03.614917+0000","last_unstale":"2023-08-25T03:34:03.616911+0000","last_undegraded":"2023-08-25T03:34:03.616911+0000","last_fullsized":"2023-08-25T03:34:03.616911+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:10:59.547904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.590306+0000","last_change":"2023-08-25T03:34:02.590306+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.590306+0000","last_undegraded":"2023-08-25T03:34:02.590306+0000","last_fullsized":"2023-08-25T03:34:02.590306+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627201+0000","last_change":"2023-08-25T03:34:03.619323+0000","last_active":"2023-08-25T03:34:03.627201+0000","last_peered":"2023-08-25T03:34:03.627201+0000","last_clean":"2023-08-25T03:34:03.627201+0000","last_became_active":"2023-08-25T03:34:03.618985+0000","last_became_peered":"2023-08-25T03:34:03.618985+0000","last_unstale":"2023-08-25T03:34:03.627201+0000","last_undegraded":"2023-08-25T03:34:03.627201+0000","last_fullsized":"2023-08-25T03:34:03.627201+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:47:39.963816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.622126+0000","last_change":"2023-08-25T03:34:03.617493+0000","last_active":"2023-08-25T03:34:03.622126+0000","last_peered":"2023-08-25T03:34:03.622126+0000","last_clean":"2023-08-25T03:34:03.622126+0000","last_became_active":"2023-08-25T03:34:03.617167+0000","last_became_peered":"2023-08-25T03:34:03.617167+0000","last_unstale":"2023-08-25T03:34:03.622126+0000","last_undegraded":"2023-08-25T03:34:03.622126+0000","last_fullsized":"2023-08-25T03:34:03.622126+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:41:33.001585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.629535+0000","last_change":"2023-08-25T03:34:03.625710+0000","last_active":"2023-08-25T03:34:03.629535+0000","last_peered":"2023-08-25T03:34:03.629535+0000","last_clean":"2023-08-25T03:34:03.629535+0000","last_became_active":"2023-08-25T03:34:03.625404+0000","last_became_peered":"2023-08-25T03:34:03.625404+0000","last_unstale":"2023-08-25T03:34:03.629535+0000","last_undegraded":"2023-08-25T03:34:03.629535+0000","last_fullsized":"2023-08-25T03:34:03.629535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:15:37.568529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.593242+0000","last_change":"2023-08-25T03:34:02.593242+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.593242+0000","last_undegraded":"2023-08-25T03:34:02.593242+0000","last_fullsized":"2023-08-25T03:34:02.593242+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.620872+0000","last_change":"2023-08-25T03:34:03.618247+0000","last_active":"2023-08-25T03:34:03.620872+0000","last_peered":"2023-08-25T03:34:03.620872+0000","last_clean":"2023-08-25T03:34:03.620872+0000","last_became_active":"2023-08-25T03:34:03.617830+0000","last_became_peered":"2023-08-25T03:34:03.617830+0000","last_unstale":"2023-08-25T03:34:03.620872+0000","last_undegraded":"2023-08-25T03:34:03.620872+0000","last_fullsized":"2023-08-25T03:34:03.620872+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:47:39.963816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627422+0000","last_change":"2023-08-25T03:34:03.622380+0000","last_active":"2023-08-25T03:34:03.627422+0000","last_peered":"2023-08-25T03:34:03.627422+0000","last_clean":"2023-08-25T03:34:03.627422+0000","last_became_active":"2023-08-25T03:34:03.619691+0000","last_became_peered":"2023-08-25T03:34:03.619691+0000","last_unstale":"2023-08-25T03:34:03.627422+0000","last_undegraded":"2023-08-25T03:34:03.627422+0000","last_fullsized":"2023-08-25T03:34:03.627422+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:25:49.895284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.584558+0000","last_change":"2023-08-25T03:34:02.584558+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.584558+0000","last_undegraded":"2023-08-25T03:34:02.584558+0000","last_fullsized":"2023-08-25T03:34:02.584558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.584609+0000","last_change":"2023-08-25T03:34:02.584609+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.584609+0000","last_undegraded":"2023-08-25T03:34:02.584609+0000","last_fullsized":"2023-08-25T03:34:02.584609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.620519+0000","last_change":"2023-08-25T03:34:03.616557+0000","last_active":"2023-08-25T03:34:03.620519+0000","last_peered":"2023-08-25T03:34:03.620519+0000","last_clean":"2023-08-25T03:34:03.620519+0000","last_became_active":"2023-08-25T03:34:03.615850+0000","last_became_peered":"2023-08-25T03:34:03.615850+0000","last_unstale":"2023-08-25T03:34:03.620519+0000","last_undegraded":"2023-08-25T03:34:03.620519+0000","last_fullsized":"2023-08-25T03:34:03.620519+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:01:15.550047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.621835+0000","last_change":"2023-08-25T03:34:03.617319+0000","last_active":"2023-08-25T03:34:03.621835+0000","last_peered":"2023-08-25T03:34:03.621835+0000","last_clean":"2023-08-25T03:34:03.621835+0000","last_became_active":"2023-08-25T03:34:03.617012+0000","last_became_peered":"2023-08-25T03:34:03.617012+0000","last_unstale":"2023-08-25T03:34:03.621835+0000","last_undegraded":"2023-08-25T03:34:03.621835+0000","last_fullsized":"2023-08-25T03:34:03.621835+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:22:25.881167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.624247+0000","last_change":"2023-08-25T03:34:03.615370+0000","last_active":"2023-08-25T03:34:03.624247+0000","last_peered":"2023-08-25T03:34:03.624247+0000","last_clean":"2023-08-25T03:34:03.624247+0000","last_became_active":"2023-08-25T03:34:03.615076+0000","last_became_peered":"2023-08-25T03:34:03.615076+0000","last_unstale":"2023-08-25T03:34:03.624247+0000","last_undegraded":"2023-08-25T03:34:03.624247+0000","last_fullsized":"2023-08-25T03:34:03.624247+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:16:06.275512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591771+0000","last_change":"2023-08-25T03:34:02.591771+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591771+0000","last_undegraded":"2023-08-25T03:34:02.591771+0000","last_fullsized":"2023-08-25T03:34:02.591771+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627116+0000","last_change":"2023-08-25T03:34:03.618966+0000","last_active":"2023-08-25T03:34:03.627116+0000","last_peered":"2023-08-25T03:34:03.627116+0000","last_clean":"2023-08-25T03:34:03.627116+0000","last_became_active":"2023-08-25T03:34:03.618636+0000","last_became_peered":"2023-08-25T03:34:03.618636+0000","last_unstale":"2023-08-25T03:34:03.627116+0000","last_undegraded":"2023-08-25T03:34:03.627116+0000","last_fullsized":"2023-08-25T03:34:03.627116+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:15:37.568529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.583704+0000","last_change":"2023-08-25T03:34:02.583704+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.583704+0000","last_undegraded":"2023-08-25T03:34:02.583704+0000","last_fullsized":"2023-08-25T03:34:02.583704+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627305+0000","last_change":"2023-08-25T03:34:03.621319+0000","last_active":"2023-08-25T03:34:03.627305+0000","last_peered":"2023-08-25T03:34:03.627305+0000","last_clean":"2023-08-25T03:34:03.627305+0000","last_became_active":"2023-08-25T03:34:03.619115+0000","last_became_peered":"2023-08-25T03:34:03.619115+0000","last_unstale":"2023-08-25T03:34:03.627305+0000","last_undegraded":"2023-08-25T03:34:03.627305+0000","last_fullsized":"2023-08-25T03:34:03.627305+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:12:38.282041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627798+0000","last_change":"2023-08-25T03:34:03.622785+0000","last_active":"2023-08-25T03:34:03.627798+0000","last_peered":"2023-08-25T03:34:03.627798+0000","last_clean":"2023-08-25T03:34:03.627798+0000","last_became_active":"2023-08-25T03:34:03.622469+0000","last_became_peered":"2023-08-25T03:34:03.622469+0000","last_unstale":"2023-08-25T03:34:03.627798+0000","last_undegraded":"2023-08-25T03:34:03.627798+0000","last_fullsized":"2023-08-25T03:34:03.627798+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:47:13.833805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.589817+0000","last_change":"2023-08-25T03:34:02.589817+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.589817+0000","last_undegraded":"2023-08-25T03:34:02.589817+0000","last_fullsized":"2023-08-25T03:34:02.589817+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591327+0000","last_change":"2023-08-25T03:34:02.591327+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591327+0000","last_undegraded":"2023-08-25T03:34:02.591327+0000","last_fullsized":"2023-08-25T03:34:02.591327+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628155+0000","last_change":"2023-08-25T03:34:03.623150+0000","last_active":"2023-08-25T03:34:03.628155+0000","last_peered":"2023-08-25T03:34:03.628155+0000","last_clean":"2023-08-25T03:34:03.628155+0000","last_became_active":"2023-08-25T03:34:03.622796+0000","last_became_peered":"2023-08-25T03:34:03.622796+0000","last_unstale":"2023-08-25T03:34:03.628155+0000","last_undegraded":"2023-08-25T03:34:03.628155+0000","last_fullsized":"2023-08-25T03:34:03.628155+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:49:11.917056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.588252+0000","last_change":"2023-08-25T03:34:02.588252+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.588252+0000","last_undegraded":"2023-08-25T03:34:02.588252+0000","last_fullsized":"2023-08-25T03:34:02.588252+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.586918+0000","last_change":"2023-08-25T03:34:02.586918+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.586918+0000","last_undegraded":"2023-08-25T03:34:02.586918+0000","last_fullsized":"2023-08-25T03:34:02.586918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.616368+0000","last_change":"2023-08-25T03:34:03.613385+0000","last_active":"2023-08-25T03:34:03.616368+0000","last_peered":"2023-08-25T03:34:03.616368+0000","last_clean":"2023-08-25T03:34:03.616368+0000","last_became_active":"2023-08-25T03:34:03.613110+0000","last_became_peered":"2023-08-25T03:34:03.613110+0000","last_unstale":"2023-08-25T03:34:03.616368+0000","last_undegraded":"2023-08-25T03:34:03.616368+0000","last_fullsized":"2023-08-25T03:34:03.616368+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:47:27.124952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628064+0000","last_change":"2023-08-25T03:34:02.608606+0000","last_active":"2023-08-25T03:34:03.628064+0000","last_peered":"2023-08-25T03:34:03.628064+0000","last_clean":"2023-08-25T03:34:03.628064+0000","last_became_active":"2023-08-25T03:34:02.608298+0000","last_became_peered":"2023-08-25T03:34:02.608298+0000","last_unstale":"2023-08-25T03:34:03.628064+0000","last_undegraded":"2023-08-25T03:34:03.628064+0000","last_fullsized":"2023-08-25T03:34:03.628064+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:27:59.879524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591347+0000","last_change":"2023-08-25T03:34:02.591347+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591347+0000","last_undegraded":"2023-08-25T03:34:02.591347+0000","last_fullsized":"2023-08-25T03:34:02.591347+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.603129+0000","last_change":"2023-08-25T03:34:02.599366+0000","last_active":"2023-08-25T03:34:02.603129+0000","last_peered":"2023-08-25T03:34:02.603129+0000","last_clean":"2023-08-25T03:34:02.603129+0000","last_became_active":"2023-08-25T03:34:02.599105+0000","last_became_peered":"2023-08-25T03:34:02.599105+0000","last_unstale":"2023-08-25T03:34:02.603129+0000","last_undegraded":"2023-08-25T03:34:02.603129+0000","last_fullsized":"2023-08-25T03:34:02.603129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:38:11.493391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591521+0000","last_change":"2023-08-25T03:34:02.591521+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591521+0000","last_undegraded":"2023-08-25T03:34:02.591521+0000","last_fullsized":"2023-08-25T03:34:02.591521+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.609089+0000","last_change":"2023-08-25T03:34:02.608397+0000","last_active":"2023-08-25T03:34:02.609089+0000","last_peered":"2023-08-25T03:34:02.609089+0000","last_clean":"2023-08-25T03:34:02.609089+0000","last_became_active":"2023-08-25T03:34:02.608067+0000","last_became_peered":"2023-08-25T03:34:02.608067+0000","last_unstale":"2023-08-25T03:34:02.609089+0000","last_undegraded":"2023-08-25T03:34:02.609089+0000","last_fullsized":"2023-08-25T03:34:02.609089+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:14:44.918756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.616644+0000","last_change":"2023-08-25T03:34:03.613614+0000","last_active":"2023-08-25T03:34:03.616644+0000","last_peered":"2023-08-25T03:34:03.616644+0000","last_clean":"2023-08-25T03:34:03.616644+0000","last_became_active":"2023-08-25T03:34:03.613291+0000","last_became_peered":"2023-08-25T03:34:03.613291+0000","last_unstale":"2023-08-25T03:34:03.616644+0000","last_undegraded":"2023-08-25T03:34:03.616644+0000","last_fullsized":"2023-08-25T03:34:03.616644+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:43:35.780927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.605932+0000","last_change":"2023-08-25T03:34:02.603475+0000","last_active":"2023-08-25T03:34:02.605932+0000","last_peered":"2023-08-25T03:34:02.605932+0000","last_clean":"2023-08-25T03:34:02.605932+0000","last_became_active":"2023-08-25T03:34:02.603129+0000","last_became_peered":"2023-08-25T03:34:02.603129+0000","last_unstale":"2023-08-25T03:34:02.605932+0000","last_undegraded":"2023-08-25T03:34:02.605932+0000","last_fullsized":"2023-08-25T03:34:02.605932+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:10:58.540490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.590564+0000","last_change":"2023-08-25T03:34:02.590564+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.590564+0000","last_undegraded":"2023-08-25T03:34:02.590564+0000","last_fullsized":"2023-08-25T03:34:02.590564+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610665+0000","last_change":"2023-08-25T03:34:02.597247+0000","last_active":"2023-08-25T03:34:03.610665+0000","last_peered":"2023-08-25T03:34:03.610665+0000","last_clean":"2023-08-25T03:34:03.610665+0000","last_became_active":"2023-08-25T03:34:02.596968+0000","last_became_peered":"2023-08-25T03:34:02.596968+0000","last_unstale":"2023-08-25T03:34:03.610665+0000","last_undegraded":"2023-08-25T03:34:03.610665+0000","last_fullsized":"2023-08-25T03:34:03.610665+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:51:52.093732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.592296+0000","last_change":"2023-08-25T03:34:02.592296+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.592296+0000","last_undegraded":"2023-08-25T03:34:02.592296+0000","last_fullsized":"2023-08-25T03:34:02.592296+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":14,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.633428+0000","last_change":"2023-08-25T03:34:02.603880+0000","last_active":"2023-08-25T03:34:03.633428+0000","last_peered":"2023-08-25T03:34:03.633428+0000","last_clean":"2023-08-25T03:34:03.633428+0000","last_became_active":"2023-08-25T03:34:02.603502+0000","last_became_peered":"2023-08-25T03:34:02.603502+0000","last_unstale":"2023-08-25T03:34:03.633428+0000","last_undegraded":"2023-08-25T03:34:03.633428+0000","last_fullsized":"2023-08-25T03:34:03.633428+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:05:24.650744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.586286+0000","last_change":"2023-08-25T03:34:02.586286+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.586286+0000","last_undegraded":"2023-08-25T03:34:02.586286+0000","last_fullsized":"2023-08-25T03:34:02.586286+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'6","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.632748+0000","last_change":"2023-08-25T03:34:02.606875+0000","last_active":"2023-08-25T03:34:03.632748+0000","last_peered":"2023-08-25T03:34:03.632748+0000","last_clean":"2023-08-25T03:34:03.632748+0000","last_became_active":"2023-08-25T03:34:02.606567+0000","last_became_peered":"2023-08-25T03:34:02.606567+0000","last_unstale":"2023-08-25T03:34:03.632748+0000","last_undegraded":"2023-08-25T03:34:03.632748+0000","last_fullsized":"2023-08-25T03:34:03.632748+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:51:52.093732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627947+0000","last_change":"2023-08-25T03:34:03.619847+0000","last_active":"2023-08-25T03:34:03.627947+0000","last_peered":"2023-08-25T03:34:03.627947+0000","last_clean":"2023-08-25T03:34:03.627947+0000","last_became_active":"2023-08-25T03:34:03.619046+0000","last_became_peered":"2023-08-25T03:34:03.619046+0000","last_unstale":"2023-08-25T03:34:03.627947+0000","last_undegraded":"2023-08-25T03:34:03.627947+0000","last_fullsized":"2023-08-25T03:34:03.627947+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:52:42.938242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628937+0000","last_change":"2023-08-25T03:34:02.604424+0000","last_active":"2023-08-25T03:34:03.628937+0000","last_peered":"2023-08-25T03:34:03.628937+0000","last_clean":"2023-08-25T03:34:03.628937+0000","last_became_active":"2023-08-25T03:34:02.604175+0000","last_became_peered":"2023-08-25T03:34:02.604175+0000","last_unstale":"2023-08-25T03:34:03.628937+0000","last_undegraded":"2023-08-25T03:34:03.628937+0000","last_fullsized":"2023-08-25T03:34:03.628937+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:57.997213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591307+0000","last_change":"2023-08-25T03:34:02.591307+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591307+0000","last_undegraded":"2023-08-25T03:34:02.591307+0000","last_fullsized":"2023-08-25T03:34:02.591307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.607623+0000","last_change":"2023-08-25T03:34:02.606889+0000","last_active":"2023-08-25T03:34:02.607623+0000","last_peered":"2023-08-25T03:34:02.607623+0000","last_clean":"2023-08-25T03:34:02.607623+0000","last_became_active":"2023-08-25T03:34:02.606424+0000","last_became_peered":"2023-08-25T03:34:02.606424+0000","last_unstale":"2023-08-25T03:34:02.607623+0000","last_undegraded":"2023-08-25T03:34:02.607623+0000","last_fullsized":"2023-08-25T03:34:02.607623+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:04:02.000324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.616388+0000","last_change":"2023-08-25T03:34:03.613888+0000","last_active":"2023-08-25T03:34:03.616388+0000","last_peered":"2023-08-25T03:34:03.616388+0000","last_clean":"2023-08-25T03:34:03.616388+0000","last_became_active":"2023-08-25T03:34:03.612829+0000","last_became_peered":"2023-08-25T03:34:03.612829+0000","last_unstale":"2023-08-25T03:34:03.616388+0000","last_undegraded":"2023-08-25T03:34:03.616388+0000","last_fullsized":"2023-08-25T03:34:03.616388+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:23:49.310992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610728+0000","last_change":"2023-08-25T03:34:02.606279+0000","last_active":"2023-08-25T03:34:03.610728+0000","last_peered":"2023-08-25T03:34:03.610728+0000","last_clean":"2023-08-25T03:34:03.610728+0000","last_became_active":"2023-08-25T03:34:02.606005+0000","last_became_peered":"2023-08-25T03:34:02.606005+0000","last_unstale":"2023-08-25T03:34:03.610728+0000","last_undegraded":"2023-08-25T03:34:03.610728+0000","last_fullsized":"2023-08-25T03:34:03.610728+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:02:01.407883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.592567+0000","last_change":"2023-08-25T03:34:02.592567+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.592567+0000","last_undegraded":"2023-08-25T03:34:02.592567+0000","last_fullsized":"2023-08-25T03:34:02.592567+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.633646+0000","last_change":"2023-08-25T03:34:02.608430+0000","last_active":"2023-08-25T03:34:03.633646+0000","last_peered":"2023-08-25T03:34:03.633646+0000","last_clean":"2023-08-25T03:34:03.633646+0000","last_became_active":"2023-08-25T03:34:02.608086+0000","last_became_peered":"2023-08-25T03:34:02.608086+0000","last_unstale":"2023-08-25T03:34:03.633646+0000","last_undegraded":"2023-08-25T03:34:03.633646+0000","last_fullsized":"2023-08-25T03:34:03.633646+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:52:41.930828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591676+0000","last_change":"2023-08-25T03:34:02.591676+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591676+0000","last_undegraded":"2023-08-25T03:34:02.591676+0000","last_fullsized":"2023-08-25T03:34:02.591676+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628287+0000","last_change":"2023-08-25T03:34:02.604587+0000","last_active":"2023-08-25T03:34:03.628287+0000","last_peered":"2023-08-25T03:34:03.628287+0000","last_clean":"2023-08-25T03:34:03.628287+0000","last_became_active":"2023-08-25T03:34:02.604290+0000","last_became_peered":"2023-08-25T03:34:02.604290+0000","last_unstale":"2023-08-25T03:34:03.628287+0000","last_undegraded":"2023-08-25T03:34:03.628287+0000","last_fullsized":"2023-08-25T03:34:03.628287+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:52:25.303751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.615963+0000","last_change":"2023-08-25T03:34:03.613959+0000","last_active":"2023-08-25T03:34:03.615963+0000","last_peered":"2023-08-25T03:34:03.615963+0000","last_clean":"2023-08-25T03:34:03.615963+0000","last_became_active":"2023-08-25T03:34:03.613667+0000","last_became_peered":"2023-08-25T03:34:03.613667+0000","last_unstale":"2023-08-25T03:34:03.615963+0000","last_undegraded":"2023-08-25T03:34:03.615963+0000","last_fullsized":"2023-08-25T03:34:03.615963+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:25:49.895284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628771+0000","last_change":"2023-08-25T03:34:02.604585+0000","last_active":"2023-08-25T03:34:03.628771+0000","last_peered":"2023-08-25T03:34:03.628771+0000","last_clean":"2023-08-25T03:34:03.628771+0000","last_became_active":"2023-08-25T03:34:02.604255+0000","last_became_peered":"2023-08-25T03:34:02.604255+0000","last_unstale":"2023-08-25T03:34:03.628771+0000","last_undegraded":"2023-08-25T03:34:03.628771+0000","last_fullsized":"2023-08-25T03:34:03.628771+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:07.999174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627819+0000","last_change":"2023-08-25T03:34:03.624731+0000","last_active":"2023-08-25T03:34:03.627819+0000","last_peered":"2023-08-25T03:34:03.627819+0000","last_clean":"2023-08-25T03:34:03.627819+0000","last_became_active":"2023-08-25T03:34:03.619454+0000","last_became_peered":"2023-08-25T03:34:03.619454+0000","last_unstale":"2023-08-25T03:34:03.627819+0000","last_undegraded":"2023-08-25T03:34:03.627819+0000","last_fullsized":"2023-08-25T03:34:03.627819+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:25:43.507515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":12,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.629966+0000","last_change":"2023-08-25T03:34:02.605137+0000","last_active":"2023-08-25T03:34:03.629966+0000","last_peered":"2023-08-25T03:34:03.629966+0000","last_clean":"2023-08-25T03:34:03.629966+0000","last_became_active":"2023-08-25T03:34:02.604846+0000","last_became_peered":"2023-08-25T03:34:02.604846+0000","last_unstale":"2023-08-25T03:34:03.629966+0000","last_undegraded":"2023-08-25T03:34:03.629966+0000","last_fullsized":"2023-08-25T03:34:03.629966+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:07.243584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.616584+0000","last_change":"2023-08-25T03:34:03.614885+0000","last_active":"2023-08-25T03:34:03.616584+0000","last_peered":"2023-08-25T03:34:03.616584+0000","last_clean":"2023-08-25T03:34:03.616584+0000","last_became_active":"2023-08-25T03:34:03.614420+0000","last_became_peered":"2023-08-25T03:34:03.614420+0000","last_unstale":"2023-08-25T03:34:03.616584+0000","last_undegraded":"2023-08-25T03:34:03.616584+0000","last_fullsized":"2023-08-25T03:34:03.616584+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:30:21.048549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610137+0000","last_change":"2023-08-25T03:34:02.598840+0000","last_active":"2023-08-25T03:34:03.610137+0000","last_peered":"2023-08-25T03:34:03.610137+0000","last_clean":"2023-08-25T03:34:03.610137+0000","last_became_active":"2023-08-25T03:34:02.598544+0000","last_became_peered":"2023-08-25T03:34:02.598544+0000","last_unstale":"2023-08-25T03:34:03.610137+0000","last_undegraded":"2023-08-25T03:34:03.610137+0000","last_fullsized":"2023-08-25T03:34:03.610137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:43:28.361208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.591069+0000","last_change":"2023-08-25T03:34:02.591069+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.591069+0000","last_undegraded":"2023-08-25T03:34:02.591069+0000","last_fullsized":"2023-08-25T03:34:02.591069+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.605947+0000","last_change":"2023-08-25T03:34:02.605001+0000","last_active":"2023-08-25T03:34:02.605947+0000","last_peered":"2023-08-25T03:34:02.605947+0000","last_clean":"2023-08-25T03:34:02.605947+0000","last_became_active":"2023-08-25T03:34:02.604686+0000","last_became_peered":"2023-08-25T03:34:02.604686+0000","last_unstale":"2023-08-25T03:34:02.605947+0000","last_undegraded":"2023-08-25T03:34:02.605947+0000","last_fullsized":"2023-08-25T03:34:02.605947+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:04:02.000324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.629507+0000","last_change":"2023-08-25T03:34:03.626817+0000","last_active":"2023-08-25T03:34:03.629507+0000","last_peered":"2023-08-25T03:34:03.629507+0000","last_clean":"2023-08-25T03:34:03.629507+0000","last_became_active":"2023-08-25T03:34:03.626483+0000","last_became_peered":"2023-08-25T03:34:03.626483+0000","last_unstale":"2023-08-25T03:34:03.629507+0000","last_undegraded":"2023-08-25T03:34:03.629507+0000","last_fullsized":"2023-08-25T03:34:03.629507+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:30:21.048549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.600756+0000","last_change":"2023-08-25T03:34:02.601925+0000","last_active":"2023-08-25T03:34:03.600756+0000","last_peered":"2023-08-25T03:34:03.600756+0000","last_clean":"2023-08-25T03:34:03.600756+0000","last_became_active":"2023-08-25T03:34:02.601552+0000","last_became_peered":"2023-08-25T03:34:02.601552+0000","last_unstale":"2023-08-25T03:34:03.600756+0000","last_undegraded":"2023-08-25T03:34:03.600756+0000","last_fullsized":"2023-08-25T03:34:03.600756+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:25:48.887870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.630046+0000","last_change":"2023-08-25T03:34:03.627975+0000","last_active":"2023-08-25T03:34:03.630046+0000","last_peered":"2023-08-25T03:34:03.630046+0000","last_clean":"2023-08-25T03:34:03.630046+0000","last_became_active":"2023-08-25T03:34:03.627489+0000","last_became_peered":"2023-08-25T03:34:03.627489+0000","last_unstale":"2023-08-25T03:34:03.630046+0000","last_undegraded":"2023-08-25T03:34:03.630046+0000","last_fullsized":"2023-08-25T03:34:03.630046+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:23:49.310992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.605294+0000","last_change":"2023-08-25T03:34:02.603347+0000","last_active":"2023-08-25T03:34:03.605294+0000","last_peered":"2023-08-25T03:34:03.605294+0000","last_clean":"2023-08-25T03:34:03.605294+0000","last_became_active":"2023-08-25T03:34:02.603010+0000","last_became_peered":"2023-08-25T03:34:02.603010+0000","last_unstale":"2023-08-25T03:34:03.605294+0000","last_undegraded":"2023-08-25T03:34:03.605294+0000","last_fullsized":"2023-08-25T03:34:03.605294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:38:55.054764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.626681+0000","last_change":"2023-08-25T03:34:03.616962+0000","last_active":"2023-08-25T03:34:03.626681+0000","last_peered":"2023-08-25T03:34:03.626681+0000","last_clean":"2023-08-25T03:34:03.626681+0000","last_became_active":"2023-08-25T03:34:03.616656+0000","last_became_peered":"2023-08-25T03:34:03.616656+0000","last_unstale":"2023-08-25T03:34:03.626681+0000","last_undegraded":"2023-08-25T03:34:03.626681+0000","last_fullsized":"2023-08-25T03:34:03.626681+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:44:42.715336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.606366+0000","last_change":"2023-08-25T03:34:02.603820+0000","last_active":"2023-08-25T03:34:02.606366+0000","last_peered":"2023-08-25T03:34:02.606366+0000","last_clean":"2023-08-25T03:34:02.606366+0000","last_became_active":"2023-08-25T03:34:02.603493+0000","last_became_peered":"2023-08-25T03:34:02.603493+0000","last_unstale":"2023-08-25T03:34:02.606366+0000","last_undegraded":"2023-08-25T03:34:02.606366+0000","last_fullsized":"2023-08-25T03:34:02.606366+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:10:58.540490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.586628+0000","last_change":"2023-08-25T03:34:02.586628+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.586628+0000","last_undegraded":"2023-08-25T03:34:02.586628+0000","last_fullsized":"2023-08-25T03:34:02.586628+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.590381+0000","last_change":"2023-08-25T03:33:57.570267+0000","last_active":"2023-08-25T03:34:02.590381+0000","last_peered":"2023-08-25T03:34:02.590381+0000","last_clean":"2023-08-25T03:34:02.590381+0000","last_became_active":"2023-08-25T03:33:57.569882+0000","last_became_peered":"2023-08-25T03:33:57.569882+0000","last_unstale":"2023-08-25T03:34:02.590381+0000","last_undegraded":"2023-08-25T03:34:02.590381+0000","last_fullsized":"2023-08-25T03:34:02.590381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:52:36.914859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610822+0000","last_change":"2023-08-25T03:34:02.600129+0000","last_active":"2023-08-25T03:34:03.610822+0000","last_peered":"2023-08-25T03:34:03.610822+0000","last_clean":"2023-08-25T03:34:03.610822+0000","last_became_active":"2023-08-25T03:34:02.599762+0000","last_became_peered":"2023-08-25T03:34:02.599762+0000","last_unstale":"2023-08-25T03:34:03.610822+0000","last_undegraded":"2023-08-25T03:34:03.610822+0000","last_fullsized":"2023-08-25T03:34:03.610822+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:57.997213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.584740+0000","last_change":"2023-08-25T03:34:02.584740+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.584740+0000","last_undegraded":"2023-08-25T03:34:02.584740+0000","last_fullsized":"2023-08-25T03:34:02.584740+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.609973+0000","last_change":"2023-08-25T03:33:57.569595+0000","last_active":"2023-08-25T03:34:03.609973+0000","last_peered":"2023-08-25T03:34:03.609973+0000","last_clean":"2023-08-25T03:34:03.609973+0000","last_became_active":"2023-08-25T03:33:57.569307+0000","last_became_peered":"2023-08-25T03:33:57.569307+0000","last_unstale":"2023-08-25T03:34:03.609973+0000","last_undegraded":"2023-08-25T03:34:03.609973+0000","last_fullsized":"2023-08-25T03:34:03.609973+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:38:50.038795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.605192+0000","last_change":"2023-08-25T03:34:02.605404+0000","last_active":"2023-08-25T03:34:03.605192+0000","last_peered":"2023-08-25T03:34:03.605192+0000","last_clean":"2023-08-25T03:34:03.605192+0000","last_became_active":"2023-08-25T03:34:02.604806+0000","last_became_peered":"2023-08-25T03:34:02.604806+0000","last_unstale":"2023-08-25T03:34:03.605192+0000","last_undegraded":"2023-08-25T03:34:03.605192+0000","last_fullsized":"2023-08-25T03:34:03.605192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:41:01.441055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.585339+0000","last_change":"2023-08-25T03:34:02.585339+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.585339+0000","last_undegraded":"2023-08-25T03:34:02.585339+0000","last_fullsized":"2023-08-25T03:34:02.585339+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.585854+0000","last_change":"2023-08-25T03:33:57.886704+0000","last_active":"2023-08-25T03:34:02.585854+0000","last_peered":"2023-08-25T03:34:02.585854+0000","last_clean":"2023-08-25T03:34:02.585854+0000","last_became_active":"2023-08-25T03:33:57.886294+0000","last_became_peered":"2023-08-25T03:33:57.886294+0000","last_unstale":"2023-08-25T03:34:02.585854+0000","last_undegraded":"2023-08-25T03:34:02.585854+0000","last_fullsized":"2023-08-25T03:34:02.585854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:03:56.984355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.598074+0000","last_change":"2023-08-25T03:34:02.589214+0000","last_active":"2023-08-25T03:34:02.598074+0000","last_peered":"2023-08-25T03:34:02.598074+0000","last_clean":"2023-08-25T03:34:02.598074+0000","last_became_active":"2023-08-25T03:34:02.588607+0000","last_became_peered":"2023-08-25T03:34:02.588607+0000","last_unstale":"2023-08-25T03:34:02.598074+0000","last_undegraded":"2023-08-25T03:34:02.598074+0000","last_fullsized":"2023-08-25T03:34:02.598074+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:43:34.773513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.587185+0000","last_change":"2023-08-25T03:34:02.587185+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.587185+0000","last_undegraded":"2023-08-25T03:34:02.587185+0000","last_fullsized":"2023-08-25T03:34:02.587185+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610764+0000","last_change":"2023-08-25T03:33:57.570657+0000","last_active":"2023-08-25T03:34:03.610764+0000","last_peered":"2023-08-25T03:34:03.610764+0000","last_clean":"2023-08-25T03:34:03.610764+0000","last_became_active":"2023-08-25T03:33:57.570425+0000","last_became_peered":"2023-08-25T03:33:57.570425+0000","last_unstale":"2023-08-25T03:34:03.610764+0000","last_undegraded":"2023-08-25T03:34:03.610764+0000","last_fullsized":"2023-08-25T03:34:03.610764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:59:31.618299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":173,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610198+0000","last_change":"2023-08-25T03:33:51.259898+0000","last_active":"2023-08-25T03:34:03.610198+0000","last_peered":"2023-08-25T03:34:03.610198+0000","last_clean":"2023-08-25T03:34:03.610198+0000","last_became_active":"2023-08-25T03:33:51.258780+0000","last_became_peered":"2023-08-25T03:33:51.258780+0000","last_unstale":"2023-08-25T03:34:03.610198+0000","last_undegraded":"2023-08-25T03:34:03.610198+0000","last_fullsized":"2023-08-25T03:34:03.610198+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:49.666858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:49.666858+0000","last_clean_scrub_stamp":"2023-08-25T03:33:49.666858+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:51:40.190283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.605411+0000","last_change":"2023-08-25T03:34:02.598738+0000","last_active":"2023-08-25T03:34:02.605411+0000","last_peered":"2023-08-25T03:34:02.605411+0000","last_clean":"2023-08-25T03:34:02.605411+0000","last_became_active":"2023-08-25T03:34:02.598219+0000","last_became_peered":"2023-08-25T03:34:02.598219+0000","last_unstale":"2023-08-25T03:34:02.605411+0000","last_undegraded":"2023-08-25T03:34:02.605411+0000","last_fullsized":"2023-08-25T03:34:02.605411+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:54:25.036156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.626310+0000","last_change":"2023-08-25T03:34:03.624616+0000","last_active":"2023-08-25T03:34:03.626310+0000","last_peered":"2023-08-25T03:34:03.626310+0000","last_clean":"2023-08-25T03:34:03.626310+0000","last_became_active":"2023-08-25T03:34:03.624330+0000","last_became_peered":"2023-08-25T03:34:03.624330+0000","last_unstale":"2023-08-25T03:34:03.626310+0000","last_undegraded":"2023-08-25T03:34:03.626310+0000","last_fullsized":"2023-08-25T03:34:03.626310+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:44:42.715336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.610373+0000","last_change":"2023-08-25T03:33:57.571821+0000","last_active":"2023-08-25T03:34:03.610373+0000","last_peered":"2023-08-25T03:34:03.610373+0000","last_clean":"2023-08-25T03:34:03.610373+0000","last_became_active":"2023-08-25T03:33:57.571528+0000","last_became_peered":"2023-08-25T03:33:57.571528+0000","last_unstale":"2023-08-25T03:34:03.610373+0000","last_undegraded":"2023-08-25T03:34:03.610373+0000","last_fullsized":"2023-08-25T03:34:03.610373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:59:31.618299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.604252+0000","last_change":"2023-08-25T03:34:02.602275+0000","last_active":"2023-08-25T03:34:02.604252+0000","last_peered":"2023-08-25T03:34:02.604252+0000","last_clean":"2023-08-25T03:34:02.604252+0000","last_became_active":"2023-08-25T03:34:02.601960+0000","last_became_peered":"2023-08-25T03:34:02.601960+0000","last_unstale":"2023-08-25T03:34:02.604252+0000","last_undegraded":"2023-08-25T03:34:02.604252+0000","last_fullsized":"2023-08-25T03:34:02.604252+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:14:44.918756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.590059+0000","last_change":"2023-08-25T03:34:02.590059+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.590059+0000","last_undegraded":"2023-08-25T03:34:02.590059+0000","last_fullsized":"2023-08-25T03:34:02.590059+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.611012+0000","last_change":"2023-08-25T03:33:57.571730+0000","last_active":"2023-08-25T03:34:03.611012+0000","last_peered":"2023-08-25T03:34:03.611012+0000","last_clean":"2023-08-25T03:34:03.611012+0000","last_became_active":"2023-08-25T03:33:57.571442+0000","last_became_peered":"2023-08-25T03:33:57.571442+0000","last_unstale":"2023-08-25T03:34:03.611012+0000","last_undegraded":"2023-08-25T03:34:03.611012+0000","last_fullsized":"2023-08-25T03:34:03.611012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:09:58.529844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.603118+0000","last_change":"2023-08-25T03:34:02.596876+0000","last_active":"2023-08-25T03:34:02.603118+0000","last_peered":"2023-08-25T03:34:02.603118+0000","last_clean":"2023-08-25T03:34:02.603118+0000","last_became_active":"2023-08-25T03:34:02.596556+0000","last_became_peered":"2023-08-25T03:34:02.596556+0000","last_unstale":"2023-08-25T03:34:02.603118+0000","last_undegraded":"2023-08-25T03:34:02.603118+0000","last_fullsized":"2023-08-25T03:34:02.603118+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:12:37.274627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.589064+0000","last_change":"2023-08-25T03:34:02.589064+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.589064+0000","last_undegraded":"2023-08-25T03:34:02.589064+0000","last_fullsized":"2023-08-25T03:34:02.589064+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.605794+0000","last_change":"2023-08-25T03:34:02.604771+0000","last_active":"2023-08-25T03:34:02.605794+0000","last_peered":"2023-08-25T03:34:02.605794+0000","last_clean":"2023-08-25T03:34:02.605794+0000","last_became_active":"2023-08-25T03:34:02.604509+0000","last_became_peered":"2023-08-25T03:34:02.604509+0000","last_unstale":"2023-08-25T03:34:02.605794+0000","last_undegraded":"2023-08-25T03:34:02.605794+0000","last_fullsized":"2023-08-25T03:34:02.605794+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:16:05.268097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.590154+0000","last_change":"2023-08-25T03:33:57.570506+0000","last_active":"2023-08-25T03:34:02.590154+0000","last_peered":"2023-08-25T03:34:02.590154+0000","last_clean":"2023-08-25T03:34:02.590154+0000","last_became_active":"2023-08-25T03:33:57.570171+0000","last_became_peered":"2023-08-25T03:33:57.570171+0000","last_unstale":"2023-08-25T03:34:02.590154+0000","last_undegraded":"2023-08-25T03:34:02.590154+0000","last_fullsized":"2023-08-25T03:34:02.590154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:30:15.025166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.594873+0000","last_change":"2023-08-25T03:34:02.594873+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.594873+0000","last_undegraded":"2023-08-25T03:34:02.594873+0000","last_fullsized":"2023-08-25T03:34:02.594873+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.600483+0000","last_change":"2023-08-25T03:33:57.570598+0000","last_active":"2023-08-25T03:34:03.600483+0000","last_peered":"2023-08-25T03:34:03.600483+0000","last_clean":"2023-08-25T03:34:03.600483+0000","last_became_active":"2023-08-25T03:33:57.570215+0000","last_became_peered":"2023-08-25T03:33:57.570215+0000","last_unstale":"2023-08-25T03:34:03.600483+0000","last_undegraded":"2023-08-25T03:34:03.600483+0000","last_fullsized":"2023-08-25T03:34:03.600483+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:12:32.258657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.606611+0000","last_change":"2023-08-25T03:34:02.605323+0000","last_active":"2023-08-25T03:34:02.606611+0000","last_peered":"2023-08-25T03:34:02.606611+0000","last_clean":"2023-08-25T03:34:02.606611+0000","last_became_active":"2023-08-25T03:34:02.605111+0000","last_became_peered":"2023-08-25T03:34:02.605111+0000","last_unstale":"2023-08-25T03:34:02.606611+0000","last_undegraded":"2023-08-25T03:34:02.606611+0000","last_fullsized":"2023-08-25T03:34:02.606611+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:44:41.707921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627797+0000","last_change":"2023-08-25T03:34:03.624353+0000","last_active":"2023-08-25T03:34:03.627797+0000","last_peered":"2023-08-25T03:34:03.627797+0000","last_clean":"2023-08-25T03:34:03.627797+0000","last_became_active":"2023-08-25T03:34:03.618468+0000","last_became_peered":"2023-08-25T03:34:03.618468+0000","last_unstale":"2023-08-25T03:34:03.627797+0000","last_undegraded":"2023-08-25T03:34:03.627797+0000","last_fullsized":"2023-08-25T03:34:03.627797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:38:12.500805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.605909+0000","last_change":"2023-08-25T03:34:02.603419+0000","last_active":"2023-08-25T03:34:02.605909+0000","last_peered":"2023-08-25T03:34:02.605909+0000","last_clean":"2023-08-25T03:34:02.605909+0000","last_became_active":"2023-08-25T03:34:02.603082+0000","last_became_peered":"2023-08-25T03:34:02.603082+0000","last_unstale":"2023-08-25T03:34:02.605909+0000","last_undegraded":"2023-08-25T03:34:02.605909+0000","last_fullsized":"2023-08-25T03:34:02.605909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:16:05.268097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.583635+0000","last_change":"2023-08-25T03:34:02.583635+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.583635+0000","last_undegraded":"2023-08-25T03:34:02.583635+0000","last_fullsized":"2023-08-25T03:34:02.583635+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.629772+0000","last_change":"2023-08-25T03:34:02.601201+0000","last_active":"2023-08-25T03:34:03.629772+0000","last_peered":"2023-08-25T03:34:03.629772+0000","last_clean":"2023-08-25T03:34:03.629772+0000","last_became_active":"2023-08-25T03:34:02.600904+0000","last_became_peered":"2023-08-25T03:34:02.600904+0000","last_unstale":"2023-08-25T03:34:03.629772+0000","last_undegraded":"2023-08-25T03:34:03.629772+0000","last_fullsized":"2023-08-25T03:34:03.629772+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:41:01.441055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.629645+0000","last_change":"2023-08-25T03:34:03.626366+0000","last_active":"2023-08-25T03:34:03.629645+0000","last_peered":"2023-08-25T03:34:03.629645+0000","last_clean":"2023-08-25T03:34:03.629645+0000","last_became_active":"2023-08-25T03:34:03.625496+0000","last_became_peered":"2023-08-25T03:34:03.625496+0000","last_unstale":"2023-08-25T03:34:03.629645+0000","last_undegraded":"2023-08-25T03:34:03.629645+0000","last_fullsized":"2023-08-25T03:34:03.629645+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:01:15.550047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.629783+0000","last_change":"2023-08-25T03:34:02.603423+0000","last_active":"2023-08-25T03:34:03.629783+0000","last_peered":"2023-08-25T03:34:03.629783+0000","last_clean":"2023-08-25T03:34:03.629783+0000","last_became_active":"2023-08-25T03:34:02.603077+0000","last_became_peered":"2023-08-25T03:34:02.603077+0000","last_unstale":"2023-08-25T03:34:03.629783+0000","last_undegraded":"2023-08-25T03:34:03.629783+0000","last_fullsized":"2023-08-25T03:34:03.629783+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:27:59.879524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628025+0000","last_change":"2023-08-25T03:34:03.622038+0000","last_active":"2023-08-25T03:34:03.628025+0000","last_peered":"2023-08-25T03:34:03.628025+0000","last_clean":"2023-08-25T03:34:03.628025+0000","last_became_active":"2023-08-25T03:34:03.621747+0000","last_became_peered":"2023-08-25T03:34:03.621747+0000","last_unstale":"2023-08-25T03:34:03.628025+0000","last_undegraded":"2023-08-25T03:34:03.628025+0000","last_fullsized":"2023-08-25T03:34:03.628025+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:43:35.780927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.605434+0000","last_change":"2023-08-25T03:34:02.598272+0000","last_active":"2023-08-25T03:34:02.605434+0000","last_peered":"2023-08-25T03:34:02.605434+0000","last_clean":"2023-08-25T03:34:02.605434+0000","last_became_active":"2023-08-25T03:34:02.597630+0000","last_became_peered":"2023-08-25T03:34:02.597630+0000","last_unstale":"2023-08-25T03:34:02.605434+0000","last_undegraded":"2023-08-25T03:34:02.605434+0000","last_fullsized":"2023-08-25T03:34:02.605434+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:02:33.121969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.592797+0000","last_change":"2023-08-25T03:34:02.592797+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.592797+0000","last_undegraded":"2023-08-25T03:34:02.592797+0000","last_fullsized":"2023-08-25T03:34:02.592797+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-08-25T03:34:02.604249+0000","last_change":"2023-08-25T03:34:02.597748+0000","last_active":"2023-08-25T03:34:02.604249+0000","last_peered":"2023-08-25T03:34:02.604249+0000","last_clean":"2023-08-25T03:34:02.604249+0000","last_became_active":"2023-08-25T03:34:02.597456+0000","last_became_peered":"2023-08-25T03:34:02.597456+0000","last_unstale":"2023-08-25T03:34:02.604249+0000","last_undegraded":"2023-08-25T03:34:02.604249+0000","last_fullsized":"2023-08-25T03:34:02.604249+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:02:33.121969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.627853+0000","last_change":"2023-08-25T03:34:03.619886+0000","last_active":"2023-08-25T03:34:03.627853+0000","last_peered":"2023-08-25T03:34:03.627853+0000","last_clean":"2023-08-25T03:34:03.627853+0000","last_became_active":"2023-08-25T03:34:03.618445+0000","last_became_peered":"2023-08-25T03:34:03.618445+0000","last_unstale":"2023-08-25T03:34:03.627853+0000","last_undegraded":"2023-08-25T03:34:03.627853+0000","last_fullsized":"2023-08-25T03:34:03.627853+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:10:59.547904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.616143+0000","last_change":"2023-08-25T03:34:03.612902+0000","last_active":"2023-08-25T03:34:03.616143+0000","last_peered":"2023-08-25T03:34:03.616143+0000","last_clean":"2023-08-25T03:34:03.616143+0000","last_became_active":"2023-08-25T03:34:03.612582+0000","last_became_peered":"2023-08-25T03:34:03.612582+0000","last_unstale":"2023-08-25T03:34:03.616143+0000","last_undegraded":"2023-08-25T03:34:03.616143+0000","last_fullsized":"2023-08-25T03:34:03.616143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:38:12.500805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.593258+0000","last_change":"2023-08-25T03:34:02.593258+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.593258+0000","last_undegraded":"2023-08-25T03:34:02.593258+0000","last_fullsized":"2023-08-25T03:34:02.593258+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-08-25T03:34:03.628003+0000","last_change":"2023-08-25T03:34:03.622024+0000","last_active":"2023-08-25T03:34:03.628003+0000","last_peered":"2023-08-25T03:34:03.628003+0000","last_clean":"2023-08-25T03:34:03.628003+0000","last_became_active":"2023-08-25T03:34:03.619673+0000","last_became_peered":"2023-08-25T03:34:03.619673+0000","last_unstale":"2023-08-25T03:34:03.628003+0000","last_undegraded":"2023-08-25T03:34:03.628003+0000","last_fullsized":"2023-08-25T03:34:03.628003+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:47:27.124952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.589875+0000","last_change":"2023-08-25T03:34:02.589875+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.589875+0000","last_undegraded":"2023-08-25T03:34:02.589875+0000","last_fullsized":"2023-08-25T03:34:02.589875+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":1,"reported_epoch":22,"state":"creating+peering","last_fresh":"2023-08-25T03:34:02.586986+0000","last_change":"2023-08-25T03:34:02.586986+0000","last_active":"2023-08-25T03:34:02.577721+0000","last_peered":"2023-08-25T03:34:02.577721+0000","last_clean":"2023-08-25T03:34:02.577721+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-08-25T03:34:02.586986+0000","last_undegraded":"2023-08-25T03:34:02.586986+0000","last_fullsized":"2023-08-25T03:34:02.586986+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1828,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":4148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":669737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]}]},{"osd":6,"up_from":14,"seq":60129542151,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":86497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":79369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.249}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":819200,"data_stored":669737,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":212992,"data_stored":72749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41899999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":79369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":88165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":245760,"data_stored":87149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-25T03:34:12.389 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T03:34:12.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.528+0000 7fd370e92700 1 Processor -- start 2023-08-25T03:34:12.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.528+0000 7fd370e92700 1 -- start start 2023-08-25T03:34:12.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.529+0000 7fd370e92700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fd36c15d1f0 0x7fd36c0ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:12.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.529+0000 7fd370e92700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd36c0cf120 con 0x7fd36c0d0720 2023-08-25T03:34:12.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.529+0000 7fd370e92700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd36c0b29f0 con 0x7fd36c06cfe0 2023-08-25T03:34:12.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.529+0000 7fd370e92700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd36c0d0bb0 con 0x7fd36c15d1f0 2023-08-25T03:34:12.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.529+0000 7fd36a59c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fd36c15d1f0 0x7fd36c0ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:12.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.529+0000 7fd369d9b700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fd36c0d0720 0x7fd36c06c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45954/0 (socket says 172.21.15.2:45954) 2023-08-25T03:34:12.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.529+0000 7fd369d9b700 1 -- 172.21.15.2:0/1393647952 learned_addr learned my addr 172.21.15.2:0/1393647952 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:12.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.530+0000 7fd36a59c700 1 -- 172.21.15.2:0/1393647952 >> v1:172.21.15.31:6789/0 conn(0x7fd36c06cfe0 legacy=0x7fd36c15d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:12.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.530+0000 7fd36a59c700 1 -- 172.21.15.2:0/1393647952 >> v1:172.21.15.2:6789/0 conn(0x7fd36c0d0720 legacy=0x7fd36c06c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:12.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.530+0000 7fd36a59c700 1 -- 172.21.15.2:0/1393647952 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36c0b2cd0 con 0x7fd36c15d1f0 2023-08-25T03:34:12.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.530+0000 7fd36959a700 1 -- 172.21.15.2:0/1393647952 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2689466034 0 0) 0x7fd36c0cf120 con 0x7fd36c0d0720 2023-08-25T03:34:12.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.530+0000 7fd36a59c700 1 --2- 172.21.15.2:0/1393647952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd36c15d1f0 0x7fd36c0ceab0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd360009840 tx=0x7fd360009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df9342bcfbbb7774 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:12.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.530+0000 7fd36959a700 1 -- 172.21.15.2:0/1393647952 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd360011830 con 0x7fd36c15d1f0 2023-08-25T03:34:12.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.530+0000 7fd36959a700 1 -- 172.21.15.2:0/1393647952 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd360011990 con 0x7fd36c15d1f0 2023-08-25T03:34:12.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.530+0000 7fd36959a700 1 -- 172.21.15.2:0/1393647952 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd360011d20 con 0x7fd36c15d1f0 2023-08-25T03:34:12.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.531+0000 7fd370e92700 1 -- 172.21.15.2:0/1393647952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd36c15d1f0 msgr2=0x7fd36c0ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:12.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.531+0000 7fd370e92700 1 --2- 172.21.15.2:0/1393647952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd36c15d1f0 0x7fd36c0ceab0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd360009840 tx=0x7fd360009b50 comp rx=0 tx=0).stop 2023-08-25T03:34:12.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.531+0000 7fd370e92700 1 -- 172.21.15.2:0/1393647952 shutdown_connections 2023-08-25T03:34:12.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.531+0000 7fd370e92700 1 --2- 172.21.15.2:0/1393647952 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd36c15d1f0 0x7fd36c0ceab0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:12.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.531+0000 7fd370e92700 1 -- 172.21.15.2:0/1393647952 >> 172.21.15.2:0/1393647952 conn(0x7fd36c0d0f70 msgr2=0x7fd36c0d1350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:12.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.531+0000 7fd370e92700 1 -- 172.21.15.2:0/1393647952 shutdown_connections 2023-08-25T03:34:12.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.531+0000 7fd370e92700 1 -- 172.21.15.2:0/1393647952 wait complete. 2023-08-25T03:34:12.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.533+0000 7fd370e92700 1 Processor -- start 2023-08-25T03:34:12.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.533+0000 7fd370e92700 1 -- start start 2023-08-25T03:34:12.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.534+0000 7fd370e92700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd36c06cfe0 0x7fd36c160440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:12.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.534+0000 7fd370e92700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd36c0d0720 0x7fd36c0af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:12.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.534+0000 7fd370e92700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd36c15d1f0 0x7fd36c1f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:12.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.534+0000 7fd370e92700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36c1f5820 con 0x7fd36c15d1f0 2023-08-25T03:34:12.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.534+0000 7fd370e92700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36c0ae2e0 con 0x7fd36c0d0720 2023-08-25T03:34:12.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.534+0000 7fd370e92700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd36c0ae5c0 con 0x7fd36c06cfe0 2023-08-25T03:34:12.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.534+0000 7fd36ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd36c15d1f0 0x7fd36c1f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:12.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.534+0000 7fd36ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd36c15d1f0 0x7fd36c1f52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59298/0 (socket says 172.21.15.2:59298) 2023-08-25T03:34:12.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.534+0000 7fd36ad9d700 1 -- 172.21.15.2:0/2140955959 learned_addr learned my addr 172.21.15.2:0/2140955959 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:12.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.534+0000 7fd36a59c700 1 --2- 172.21.15.2:0/2140955959 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd36c06cfe0 0x7fd36c160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:12.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.534+0000 7fd369d9b700 1 --2- 172.21.15.2:0/2140955959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd36c0d0720 0x7fd36c0af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:12.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.534+0000 7fd36ad9d700 1 -- 172.21.15.2:0/2140955959 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd36c06cfe0 msgr2=0x7fd36c160440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:12.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.534+0000 7fd36ad9d700 1 --2- 172.21.15.2:0/2140955959 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd36c06cfe0 0x7fd36c160440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:12.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.534+0000 7fd36ad9d700 1 -- 172.21.15.2:0/2140955959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd36c0d0720 msgr2=0x7fd36c0af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:12.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.534+0000 7fd36ad9d700 1 --2- 172.21.15.2:0/2140955959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd36c0d0720 0x7fd36c0af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:12.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.534+0000 7fd36ad9d700 1 -- 172.21.15.2:0/2140955959 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3600094a0 con 0x7fd36c15d1f0 2023-08-25T03:34:12.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.534+0000 7fd36ad9d700 1 --2- 172.21.15.2:0/2140955959 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd36c15d1f0 0x7fd36c1f52e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd36400cc50 tx=0x7fd36400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:12.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.534+0000 7fd35affd700 1 -- 172.21.15.2:0/2140955959 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd364003c00 con 0x7fd36c15d1f0 2023-08-25T03:34:12.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.534+0000 7fd35affd700 1 -- 172.21.15.2:0/2140955959 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd364003d60 con 0x7fd36c15d1f0 2023-08-25T03:34:12.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.534+0000 7fd370e92700 1 -- 172.21.15.2:0/2140955959 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd36c0ae8a0 con 0x7fd36c15d1f0 2023-08-25T03:34:12.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.534+0000 7fd35affd700 1 -- 172.21.15.2:0/2140955959 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36400aaf0 con 0x7fd36c15d1f0 2023-08-25T03:34:12.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.535+0000 7fd370e92700 1 -- 172.21.15.2:0/2140955959 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd36c0aee60 con 0x7fd36c15d1f0 2023-08-25T03:34:12.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.535+0000 7fd370e92700 1 -- 172.21.15.2:0/2140955959 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd36c0b2f90 con 0x7fd36c15d1f0 2023-08-25T03:34:12.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.536+0000 7fd35affd700 1 -- 172.21.15.2:0/2140955959 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd36400ac50 con 0x7fd36c15d1f0 2023-08-25T03:34:12.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.536+0000 7fd35affd700 1 --2- 172.21.15.2:0/2140955959 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd34c06a0e0 0x7fd34c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:12.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.536+0000 7fd35affd700 1 -- 172.21.15.2:0/2140955959 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd364088570 con 0x7fd36c15d1f0 2023-08-25T03:34:12.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.536+0000 7fd36a59c700 1 --2- 172.21.15.2:0/2140955959 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd34c06a0e0 0x7fd34c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:12.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.537+0000 7fd36a59c700 1 --2- 172.21.15.2:0/2140955959 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd34c06a0e0 0x7fd34c06c5a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd3600090b0 tx=0x7fd36001c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:12.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.541+0000 7fd35affd700 1 -- 172.21.15.2:0/2140955959 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd36c0b2f90 con 0x7fd36c15d1f0 2023-08-25T03:34:12.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.736+0000 7fd370e92700 1 -- 172.21.15.2:0/2140955959 --> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd36c0b3290 con 0x7fd34c06a0e0 2023-08-25T03:34:12.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.744+0000 7fd35affd700 1 -- 172.21.15.2:0/2140955959 <== mgr.4108 v2:172.21.15.2:6832/3528779779 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286793 (secure 0 0 0) 0x7fd360011c00 con 0x7fd34c06a0e0 2023-08-25T03:34:12.750 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:12.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.748+0000 7fd370e92700 1 -- 172.21.15.2:0/2140955959 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd34c06a0e0 msgr2=0x7fd34c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:12.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.748+0000 7fd370e92700 1 --2- 172.21.15.2:0/2140955959 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd34c06a0e0 0x7fd34c06c5a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd3600090b0 tx=0x7fd36001c040 comp rx=0 tx=0).stop 2023-08-25T03:34:12.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.748+0000 7fd370e92700 1 -- 172.21.15.2:0/2140955959 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd36c15d1f0 msgr2=0x7fd36c1f52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:12.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.748+0000 7fd370e92700 1 --2- 172.21.15.2:0/2140955959 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd36c15d1f0 0x7fd36c1f52e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd36400cc50 tx=0x7fd36400cf60 comp rx=0 tx=0).stop 2023-08-25T03:34:12.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.748+0000 7fd370e92700 1 -- 172.21.15.2:0/2140955959 shutdown_connections 2023-08-25T03:34:12.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.748+0000 7fd370e92700 1 --2- 172.21.15.2:0/2140955959 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd34c06a0e0 0x7fd34c06c5a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:12.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.748+0000 7fd370e92700 1 --2- 172.21.15.2:0/2140955959 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd36c15d1f0 0x7fd36c1f52e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:12.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.748+0000 7fd370e92700 1 --2- 172.21.15.2:0/2140955959 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd36c06cfe0 0x7fd36c160440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:12.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.748+0000 7fd370e92700 1 --2- 172.21.15.2:0/2140955959 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd36c0d0720 0x7fd36c0af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:12.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.748+0000 7fd370e92700 1 -- 172.21.15.2:0/2140955959 >> 172.21.15.2:0/2140955959 conn(0x7fd36c0d0f70 msgr2=0x7fd36c0b3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:12.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.748+0000 7fd370e92700 1 -- 172.21.15.2:0/2140955959 shutdown_connections 2023-08-25T03:34:12.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.748+0000 7fd370e92700 1 -- 172.21.15.2:0/2140955959 wait complete. 2023-08-25T03:34:12.756 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-08-25T03:34:12.760 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2023-08-25T03:34:11.505546+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":203,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219352,"kb_used_data":3176,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754755368,"statfs":{"total":773094113280,"available":772869496832,"internally_reserved":0,"allocated":3252224,"data_stored":1930756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001913"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611410+0000","last_change":"2023-08-25T03:34:03.628457+0000","last_active":"2023-08-25T03:34:04.611410+0000","last_peered":"2023-08-25T03:34:04.611410+0000","last_clean":"2023-08-25T03:34:04.611410+0000","last_became_active":"2023-08-25T03:34:03.627977+0000","last_became_peered":"2023-08-25T03:34:03.627977+0000","last_unstale":"2023-08-25T03:34:04.611410+0000","last_undegraded":"2023-08-25T03:34:04.611410+0000","last_fullsized":"2023-08-25T03:34:04.611410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:01:15.550047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611582+0000","last_change":"2023-08-25T03:34:03.616304+0000","last_active":"2023-08-25T03:34:04.611582+0000","last_peered":"2023-08-25T03:34:04.611582+0000","last_clean":"2023-08-25T03:34:04.611582+0000","last_became_active":"2023-08-25T03:34:03.615917+0000","last_became_peered":"2023-08-25T03:34:03.615917+0000","last_unstale":"2023-08-25T03:34:04.611582+0000","last_undegraded":"2023-08-25T03:34:04.611582+0000","last_fullsized":"2023-08-25T03:34:04.611582+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:08.250998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348968+0000","last_change":"2023-08-25T03:34:03.622799+0000","last_active":"2023-08-25T03:34:05.348968+0000","last_peered":"2023-08-25T03:34:05.348968+0000","last_clean":"2023-08-25T03:34:05.348968+0000","last_became_active":"2023-08-25T03:34:03.622602+0000","last_became_peered":"2023-08-25T03:34:03.622602+0000","last_unstale":"2023-08-25T03:34:05.348968+0000","last_undegraded":"2023-08-25T03:34:05.348968+0000","last_fullsized":"2023-08-25T03:34:05.348968+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:45:56.260357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.613438+0000","last_change":"2023-08-25T03:34:03.615254+0000","last_active":"2023-08-25T03:34:04.613438+0000","last_peered":"2023-08-25T03:34:04.613438+0000","last_clean":"2023-08-25T03:34:04.613438+0000","last_became_active":"2023-08-25T03:34:03.614917+0000","last_became_peered":"2023-08-25T03:34:03.614917+0000","last_unstale":"2023-08-25T03:34:04.613438+0000","last_undegraded":"2023-08-25T03:34:04.613438+0000","last_fullsized":"2023-08-25T03:34:04.613438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:43:29.368622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169198+0000","last_change":"2023-08-25T03:34:03.626638+0000","last_active":"2023-08-25T03:34:05.169198+0000","last_peered":"2023-08-25T03:34:05.169198+0000","last_clean":"2023-08-25T03:34:05.169198+0000","last_became_active":"2023-08-25T03:34:03.626317+0000","last_became_peered":"2023-08-25T03:34:03.626317+0000","last_unstale":"2023-08-25T03:34:05.169198+0000","last_undegraded":"2023-08-25T03:34:05.169198+0000","last_fullsized":"2023-08-25T03:34:05.169198+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:10:04.553228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348619+0000","last_change":"2023-08-25T03:34:03.619323+0000","last_active":"2023-08-25T03:34:05.348619+0000","last_peered":"2023-08-25T03:34:05.348619+0000","last_clean":"2023-08-25T03:34:05.348619+0000","last_became_active":"2023-08-25T03:34:03.618985+0000","last_became_peered":"2023-08-25T03:34:03.618985+0000","last_unstale":"2023-08-25T03:34:05.348619+0000","last_undegraded":"2023-08-25T03:34:05.348619+0000","last_fullsized":"2023-08-25T03:34:05.348619+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:02:02.415298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.263861+0000","last_change":"2023-08-25T03:34:03.617493+0000","last_active":"2023-08-25T03:34:05.263861+0000","last_peered":"2023-08-25T03:34:05.263861+0000","last_clean":"2023-08-25T03:34:05.263861+0000","last_became_active":"2023-08-25T03:34:03.617167+0000","last_became_peered":"2023-08-25T03:34:03.617167+0000","last_unstale":"2023-08-25T03:34:05.263861+0000","last_undegraded":"2023-08-25T03:34:05.263861+0000","last_fullsized":"2023-08-25T03:34:05.263861+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:57:59.924993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611074+0000","last_change":"2023-08-25T03:34:03.625710+0000","last_active":"2023-08-25T03:34:04.611074+0000","last_peered":"2023-08-25T03:34:04.611074+0000","last_clean":"2023-08-25T03:34:04.611074+0000","last_became_active":"2023-08-25T03:34:03.625404+0000","last_became_peered":"2023-08-25T03:34:03.625404+0000","last_unstale":"2023-08-25T03:34:04.611074+0000","last_undegraded":"2023-08-25T03:34:04.611074+0000","last_fullsized":"2023-08-25T03:34:04.611074+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:02:02.415298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169221+0000","last_change":"2023-08-25T03:34:03.626723+0000","last_active":"2023-08-25T03:34:05.169221+0000","last_peered":"2023-08-25T03:34:05.169221+0000","last_clean":"2023-08-25T03:34:05.169221+0000","last_became_active":"2023-08-25T03:34:03.626419+0000","last_became_peered":"2023-08-25T03:34:03.626419+0000","last_unstale":"2023-08-25T03:34:05.169221+0000","last_undegraded":"2023-08-25T03:34:05.169221+0000","last_fullsized":"2023-08-25T03:34:05.169221+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:09.006588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611787+0000","last_change":"2023-08-25T03:34:03.618247+0000","last_active":"2023-08-25T03:34:04.611787+0000","last_peered":"2023-08-25T03:34:04.611787+0000","last_clean":"2023-08-25T03:34:04.611787+0000","last_became_active":"2023-08-25T03:34:03.617830+0000","last_became_peered":"2023-08-25T03:34:03.617830+0000","last_unstale":"2023-08-25T03:34:04.611787+0000","last_undegraded":"2023-08-25T03:34:04.611787+0000","last_fullsized":"2023-08-25T03:34:04.611787+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:01:41.454184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349005+0000","last_change":"2023-08-25T03:34:03.622380+0000","last_active":"2023-08-25T03:34:05.349005+0000","last_peered":"2023-08-25T03:34:05.349005+0000","last_clean":"2023-08-25T03:34:05.349005+0000","last_became_active":"2023-08-25T03:34:03.619691+0000","last_became_peered":"2023-08-25T03:34:03.619691+0000","last_unstale":"2023-08-25T03:34:05.349005+0000","last_undegraded":"2023-08-25T03:34:05.349005+0000","last_fullsized":"2023-08-25T03:34:05.349005+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:08.250998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611500+0000","last_change":"2023-08-25T03:34:03.624204+0000","last_active":"2023-08-25T03:34:04.611500+0000","last_peered":"2023-08-25T03:34:04.611500+0000","last_clean":"2023-08-25T03:34:04.611500+0000","last_became_active":"2023-08-25T03:34:03.623945+0000","last_became_peered":"2023-08-25T03:34:03.623945+0000","last_unstale":"2023-08-25T03:34:04.611500+0000","last_undegraded":"2023-08-25T03:34:04.611500+0000","last_fullsized":"2023-08-25T03:34:04.611500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:25:43.507515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611336+0000","last_change":"2023-08-25T03:34:03.622941+0000","last_active":"2023-08-25T03:34:04.611336+0000","last_peered":"2023-08-25T03:34:04.611336+0000","last_clean":"2023-08-25T03:34:04.611336+0000","last_became_active":"2023-08-25T03:34:03.622627+0000","last_became_peered":"2023-08-25T03:34:03.622627+0000","last_unstale":"2023-08-25T03:34:04.611336+0000","last_undegraded":"2023-08-25T03:34:04.611336+0000","last_fullsized":"2023-08-25T03:34:04.611336+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:23:49.310992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611493+0000","last_change":"2023-08-25T03:34:03.616557+0000","last_active":"2023-08-25T03:34:04.611493+0000","last_peered":"2023-08-25T03:34:04.611493+0000","last_clean":"2023-08-25T03:34:04.611493+0000","last_became_active":"2023-08-25T03:34:03.615850+0000","last_became_peered":"2023-08-25T03:34:03.615850+0000","last_unstale":"2023-08-25T03:34:04.611493+0000","last_undegraded":"2023-08-25T03:34:04.611493+0000","last_fullsized":"2023-08-25T03:34:04.611493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:10:04.553228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264238+0000","last_change":"2023-08-25T03:34:03.617319+0000","last_active":"2023-08-25T03:34:05.264238+0000","last_peered":"2023-08-25T03:34:05.264238+0000","last_clean":"2023-08-25T03:34:05.264238+0000","last_became_active":"2023-08-25T03:34:03.617012+0000","last_became_peered":"2023-08-25T03:34:03.617012+0000","last_unstale":"2023-08-25T03:34:05.264238+0000","last_undegraded":"2023-08-25T03:34:05.264238+0000","last_fullsized":"2023-08-25T03:34:05.264238+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:00:42.442989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348569+0000","last_change":"2023-08-25T03:34:03.615370+0000","last_active":"2023-08-25T03:34:05.348569+0000","last_peered":"2023-08-25T03:34:05.348569+0000","last_clean":"2023-08-25T03:34:05.348569+0000","last_became_active":"2023-08-25T03:34:03.615076+0000","last_became_peered":"2023-08-25T03:34:03.615076+0000","last_unstale":"2023-08-25T03:34:05.348569+0000","last_undegraded":"2023-08-25T03:34:05.348569+0000","last_fullsized":"2023-08-25T03:34:05.348569+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:49:11.917056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169218+0000","last_change":"2023-08-25T03:34:03.627117+0000","last_active":"2023-08-25T03:34:05.169218+0000","last_peered":"2023-08-25T03:34:05.169218+0000","last_clean":"2023-08-25T03:34:05.169218+0000","last_became_active":"2023-08-25T03:34:03.626473+0000","last_became_peered":"2023-08-25T03:34:03.626473+0000","last_unstale":"2023-08-25T03:34:05.169218+0000","last_undegraded":"2023-08-25T03:34:05.169218+0000","last_fullsized":"2023-08-25T03:34:05.169218+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:52:26.311165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349494+0000","last_change":"2023-08-25T03:34:03.618966+0000","last_active":"2023-08-25T03:34:05.349494+0000","last_peered":"2023-08-25T03:34:05.349494+0000","last_clean":"2023-08-25T03:34:05.349494+0000","last_became_active":"2023-08-25T03:34:03.618636+0000","last_became_peered":"2023-08-25T03:34:03.618636+0000","last_unstale":"2023-08-25T03:34:05.349494+0000","last_undegraded":"2023-08-25T03:34:05.349494+0000","last_fullsized":"2023-08-25T03:34:05.349494+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:33:29.470663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611519+0000","last_change":"2023-08-25T03:34:03.605314+0000","last_active":"2023-08-25T03:34:04.611519+0000","last_peered":"2023-08-25T03:34:04.611519+0000","last_clean":"2023-08-25T03:34:04.611519+0000","last_became_active":"2023-08-25T03:34:03.605008+0000","last_became_peered":"2023-08-25T03:34:03.605008+0000","last_unstale":"2023-08-25T03:34:04.611519+0000","last_undegraded":"2023-08-25T03:34:04.611519+0000","last_fullsized":"2023-08-25T03:34:04.611519+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:47:39.963816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349001+0000","last_change":"2023-08-25T03:34:03.621319+0000","last_active":"2023-08-25T03:34:05.349001+0000","last_peered":"2023-08-25T03:34:05.349001+0000","last_clean":"2023-08-25T03:34:05.349001+0000","last_became_active":"2023-08-25T03:34:03.619115+0000","last_became_peered":"2023-08-25T03:34:03.619115+0000","last_unstale":"2023-08-25T03:34:05.349001+0000","last_undegraded":"2023-08-25T03:34:05.349001+0000","last_fullsized":"2023-08-25T03:34:05.349001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:23:05.976771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264207+0000","last_change":"2023-08-25T03:34:03.622785+0000","last_active":"2023-08-25T03:34:05.264207+0000","last_peered":"2023-08-25T03:34:05.264207+0000","last_clean":"2023-08-25T03:34:05.264207+0000","last_became_active":"2023-08-25T03:34:03.622469+0000","last_became_peered":"2023-08-25T03:34:03.622469+0000","last_unstale":"2023-08-25T03:34:05.264207+0000","last_undegraded":"2023-08-25T03:34:05.264207+0000","last_fullsized":"2023-08-25T03:34:05.264207+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:35:42.993608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.168950+0000","last_change":"2023-08-25T03:34:03.620545+0000","last_active":"2023-08-25T03:34:05.168950+0000","last_peered":"2023-08-25T03:34:05.168950+0000","last_clean":"2023-08-25T03:34:05.168950+0000","last_became_active":"2023-08-25T03:34:03.619698+0000","last_became_peered":"2023-08-25T03:34:03.619698+0000","last_unstale":"2023-08-25T03:34:05.168950+0000","last_undegraded":"2023-08-25T03:34:05.168950+0000","last_fullsized":"2023-08-25T03:34:05.168950+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:41:33.001585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349094+0000","last_change":"2023-08-25T03:34:03.631189+0000","last_active":"2023-08-25T03:34:05.349094+0000","last_peered":"2023-08-25T03:34:05.349094+0000","last_clean":"2023-08-25T03:34:05.349094+0000","last_became_active":"2023-08-25T03:34:03.630807+0000","last_became_peered":"2023-08-25T03:34:03.630807+0000","last_unstale":"2023-08-25T03:34:05.349094+0000","last_undegraded":"2023-08-25T03:34:05.349094+0000","last_fullsized":"2023-08-25T03:34:05.349094+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:41:33.001585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.263844+0000","last_change":"2023-08-25T03:34:03.623150+0000","last_active":"2023-08-25T03:34:05.263844+0000","last_peered":"2023-08-25T03:34:05.263844+0000","last_clean":"2023-08-25T03:34:05.263844+0000","last_became_active":"2023-08-25T03:34:03.622796+0000","last_became_peered":"2023-08-25T03:34:03.622796+0000","last_unstale":"2023-08-25T03:34:05.263844+0000","last_undegraded":"2023-08-25T03:34:05.263844+0000","last_fullsized":"2023-08-25T03:34:05.263844+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:01:41.454184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349917+0000","last_change":"2023-08-25T03:34:03.620031+0000","last_active":"2023-08-25T03:34:05.349917+0000","last_peered":"2023-08-25T03:34:05.349917+0000","last_clean":"2023-08-25T03:34:05.349917+0000","last_became_active":"2023-08-25T03:34:03.619738+0000","last_became_peered":"2023-08-25T03:34:03.619738+0000","last_unstale":"2023-08-25T03:34:05.349917+0000","last_undegraded":"2023-08-25T03:34:05.349917+0000","last_fullsized":"2023-08-25T03:34:05.349917+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:08.250998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349705+0000","last_change":"2023-08-25T03:34:03.632252+0000","last_active":"2023-08-25T03:34:05.349705+0000","last_peered":"2023-08-25T03:34:05.349705+0000","last_clean":"2023-08-25T03:34:05.349705+0000","last_became_active":"2023-08-25T03:34:03.631975+0000","last_became_peered":"2023-08-25T03:34:03.631975+0000","last_unstale":"2023-08-25T03:34:05.349705+0000","last_undegraded":"2023-08-25T03:34:05.349705+0000","last_fullsized":"2023-08-25T03:34:05.349705+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:22:25.881167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611728+0000","last_change":"2023-08-25T03:34:03.613385+0000","last_active":"2023-08-25T03:34:04.611728+0000","last_peered":"2023-08-25T03:34:04.611728+0000","last_clean":"2023-08-25T03:34:04.611728+0000","last_became_active":"2023-08-25T03:34:03.613110+0000","last_became_peered":"2023-08-25T03:34:03.613110+0000","last_unstale":"2023-08-25T03:34:04.611728+0000","last_undegraded":"2023-08-25T03:34:04.611728+0000","last_fullsized":"2023-08-25T03:34:04.611728+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:03:06.448800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348876+0000","last_change":"2023-08-25T03:34:02.608606+0000","last_active":"2023-08-25T03:34:05.348876+0000","last_peered":"2023-08-25T03:34:05.348876+0000","last_clean":"2023-08-25T03:34:05.348876+0000","last_became_active":"2023-08-25T03:34:02.608298+0000","last_became_peered":"2023-08-25T03:34:02.608298+0000","last_unstale":"2023-08-25T03:34:05.348876+0000","last_undegraded":"2023-08-25T03:34:05.348876+0000","last_fullsized":"2023-08-25T03:34:05.348876+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:52:25.303751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.350009+0000","last_change":"2023-08-25T03:34:03.619019+0000","last_active":"2023-08-25T03:34:05.350009+0000","last_peered":"2023-08-25T03:34:05.350009+0000","last_clean":"2023-08-25T03:34:05.350009+0000","last_became_active":"2023-08-25T03:34:03.618719+0000","last_became_peered":"2023-08-25T03:34:03.618719+0000","last_unstale":"2023-08-25T03:34:05.350009+0000","last_undegraded":"2023-08-25T03:34:05.350009+0000","last_fullsized":"2023-08-25T03:34:05.350009+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:23:05.976771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349898+0000","last_change":"2023-08-25T03:34:02.599366+0000","last_active":"2023-08-25T03:34:05.349898+0000","last_peered":"2023-08-25T03:34:05.349898+0000","last_clean":"2023-08-25T03:34:05.349898+0000","last_became_active":"2023-08-25T03:34:02.599105+0000","last_became_peered":"2023-08-25T03:34:02.599105+0000","last_unstale":"2023-08-25T03:34:05.349898+0000","last_undegraded":"2023-08-25T03:34:05.349898+0000","last_fullsized":"2023-08-25T03:34:05.349898+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:03:05.441386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169298+0000","last_change":"2023-08-25T03:34:03.615711+0000","last_active":"2023-08-25T03:34:05.169298+0000","last_peered":"2023-08-25T03:34:05.169298+0000","last_clean":"2023-08-25T03:34:05.169298+0000","last_became_active":"2023-08-25T03:34:03.615315+0000","last_became_peered":"2023-08-25T03:34:03.615315+0000","last_unstale":"2023-08-25T03:34:05.169298+0000","last_undegraded":"2023-08-25T03:34:05.169298+0000","last_fullsized":"2023-08-25T03:34:05.169298+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:03:06.448800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349079+0000","last_change":"2023-08-25T03:34:02.608397+0000","last_active":"2023-08-25T03:34:05.349079+0000","last_peered":"2023-08-25T03:34:05.349079+0000","last_clean":"2023-08-25T03:34:05.349079+0000","last_became_active":"2023-08-25T03:34:02.608067+0000","last_became_peered":"2023-08-25T03:34:02.608067+0000","last_unstale":"2023-08-25T03:34:05.349079+0000","last_undegraded":"2023-08-25T03:34:05.349079+0000","last_fullsized":"2023-08-25T03:34:05.349079+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:07.243584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612775+0000","last_change":"2023-08-25T03:34:03.613614+0000","last_active":"2023-08-25T03:34:04.612775+0000","last_peered":"2023-08-25T03:34:04.612775+0000","last_clean":"2023-08-25T03:34:04.612775+0000","last_became_active":"2023-08-25T03:34:03.613291+0000","last_became_peered":"2023-08-25T03:34:03.613291+0000","last_unstale":"2023-08-25T03:34:04.612775+0000","last_undegraded":"2023-08-25T03:34:04.612775+0000","last_fullsized":"2023-08-25T03:34:04.612775+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:38:56.062178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169421+0000","last_change":"2023-08-25T03:34:02.603475+0000","last_active":"2023-08-25T03:34:05.169421+0000","last_peered":"2023-08-25T03:34:05.169421+0000","last_clean":"2023-08-25T03:34:05.169421+0000","last_became_active":"2023-08-25T03:34:02.603129+0000","last_became_peered":"2023-08-25T03:34:02.603129+0000","last_unstale":"2023-08-25T03:34:05.169421+0000","last_undegraded":"2023-08-25T03:34:05.169421+0000","last_fullsized":"2023-08-25T03:34:05.169421+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:27:14.892441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349004+0000","last_change":"2023-08-25T03:34:03.615004+0000","last_active":"2023-08-25T03:34:05.349004+0000","last_peered":"2023-08-25T03:34:05.349004+0000","last_clean":"2023-08-25T03:34:05.349004+0000","last_became_active":"2023-08-25T03:34:03.614699+0000","last_became_peered":"2023-08-25T03:34:03.614699+0000","last_unstale":"2023-08-25T03:34:05.349004+0000","last_undegraded":"2023-08-25T03:34:05.349004+0000","last_fullsized":"2023-08-25T03:34:05.349004+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:02:02.415298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611685+0000","last_change":"2023-08-25T03:34:02.597247+0000","last_active":"2023-08-25T03:34:04.611685+0000","last_peered":"2023-08-25T03:34:04.611685+0000","last_clean":"2023-08-25T03:34:04.611685+0000","last_became_active":"2023-08-25T03:34:02.596968+0000","last_became_peered":"2023-08-25T03:34:02.596968+0000","last_unstale":"2023-08-25T03:34:04.611685+0000","last_undegraded":"2023-08-25T03:34:04.611685+0000","last_fullsized":"2023-08-25T03:34:04.611685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:23:04.969357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349570+0000","last_change":"2023-08-25T03:34:03.623532+0000","last_active":"2023-08-25T03:34:05.349570+0000","last_peered":"2023-08-25T03:34:05.349570+0000","last_clean":"2023-08-25T03:34:05.349570+0000","last_became_active":"2023-08-25T03:34:03.622546+0000","last_became_peered":"2023-08-25T03:34:03.622546+0000","last_unstale":"2023-08-25T03:34:05.349570+0000","last_undegraded":"2023-08-25T03:34:05.349570+0000","last_fullsized":"2023-08-25T03:34:05.349570+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:22:25.881167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264179+0000","last_change":"2023-08-25T03:34:02.603880+0000","last_active":"2023-08-25T03:34:05.264179+0000","last_peered":"2023-08-25T03:34:05.264179+0000","last_clean":"2023-08-25T03:34:05.264179+0000","last_became_active":"2023-08-25T03:34:02.603502+0000","last_became_peered":"2023-08-25T03:34:02.603502+0000","last_unstale":"2023-08-25T03:34:05.264179+0000","last_undegraded":"2023-08-25T03:34:05.264179+0000","last_fullsized":"2023-08-25T03:34:05.264179+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:39:33.908387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.350317+0000","last_change":"2023-08-25T03:34:03.614556+0000","last_active":"2023-08-25T03:34:05.350317+0000","last_peered":"2023-08-25T03:34:05.350317+0000","last_clean":"2023-08-25T03:34:05.350317+0000","last_became_active":"2023-08-25T03:34:03.614260+0000","last_became_peered":"2023-08-25T03:34:03.614260+0000","last_unstale":"2023-08-25T03:34:05.350317+0000","last_undegraded":"2023-08-25T03:34:05.350317+0000","last_fullsized":"2023-08-25T03:34:05.350317+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T03:41:42.153100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'6","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349211+0000","last_change":"2023-08-25T03:34:02.606875+0000","last_active":"2023-08-25T03:34:05.349211+0000","last_peered":"2023-08-25T03:34:05.349211+0000","last_clean":"2023-08-25T03:34:05.349211+0000","last_became_active":"2023-08-25T03:34:02.606567+0000","last_became_peered":"2023-08-25T03:34:02.606567+0000","last_unstale":"2023-08-25T03:34:05.349211+0000","last_undegraded":"2023-08-25T03:34:05.349211+0000","last_fullsized":"2023-08-25T03:34:05.349211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:27:14.892441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348768+0000","last_change":"2023-08-25T03:34:03.619847+0000","last_active":"2023-08-25T03:34:05.348768+0000","last_peered":"2023-08-25T03:34:05.348768+0000","last_clean":"2023-08-25T03:34:05.348768+0000","last_became_active":"2023-08-25T03:34:03.619046+0000","last_became_peered":"2023-08-25T03:34:03.619046+0000","last_unstale":"2023-08-25T03:34:05.348768+0000","last_undegraded":"2023-08-25T03:34:05.348768+0000","last_fullsized":"2023-08-25T03:34:05.348768+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:22:25.881167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264047+0000","last_change":"2023-08-25T03:34:02.604424+0000","last_active":"2023-08-25T03:34:05.264047+0000","last_peered":"2023-08-25T03:34:05.264047+0000","last_clean":"2023-08-25T03:34:05.264047+0000","last_became_active":"2023-08-25T03:34:02.604175+0000","last_became_peered":"2023-08-25T03:34:02.604175+0000","last_unstale":"2023-08-25T03:34:05.264047+0000","last_undegraded":"2023-08-25T03:34:05.264047+0000","last_fullsized":"2023-08-25T03:34:05.264047+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:45:55.252943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349790+0000","last_change":"2023-08-25T03:34:03.622054+0000","last_active":"2023-08-25T03:34:05.349790+0000","last_peered":"2023-08-25T03:34:05.349790+0000","last_clean":"2023-08-25T03:34:05.349790+0000","last_became_active":"2023-08-25T03:34:03.619122+0000","last_became_peered":"2023-08-25T03:34:03.619122+0000","last_unstale":"2023-08-25T03:34:05.349790+0000","last_undegraded":"2023-08-25T03:34:05.349790+0000","last_fullsized":"2023-08-25T03:34:05.349790+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:09.006588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.168841+0000","last_change":"2023-08-25T03:34:02.606889+0000","last_active":"2023-08-25T03:34:05.168841+0000","last_peered":"2023-08-25T03:34:05.168841+0000","last_clean":"2023-08-25T03:34:05.168841+0000","last_became_active":"2023-08-25T03:34:02.606424+0000","last_became_peered":"2023-08-25T03:34:02.606424+0000","last_unstale":"2023-08-25T03:34:05.168841+0000","last_undegraded":"2023-08-25T03:34:05.168841+0000","last_fullsized":"2023-08-25T03:34:05.168841+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:02:01.407883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611980+0000","last_change":"2023-08-25T03:34:03.613888+0000","last_active":"2023-08-25T03:34:04.611980+0000","last_peered":"2023-08-25T03:34:04.611980+0000","last_clean":"2023-08-25T03:34:04.611980+0000","last_became_active":"2023-08-25T03:34:03.612829+0000","last_became_peered":"2023-08-25T03:34:03.612829+0000","last_unstale":"2023-08-25T03:34:04.611980+0000","last_undegraded":"2023-08-25T03:34:04.611980+0000","last_fullsized":"2023-08-25T03:34:04.611980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:27:15.899855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.263949+0000","last_change":"2023-08-25T03:34:02.606279+0000","last_active":"2023-08-25T03:34:05.263949+0000","last_peered":"2023-08-25T03:34:05.263949+0000","last_clean":"2023-08-25T03:34:05.263949+0000","last_became_active":"2023-08-25T03:34:02.606005+0000","last_became_peered":"2023-08-25T03:34:02.606005+0000","last_unstale":"2023-08-25T03:34:05.263949+0000","last_undegraded":"2023-08-25T03:34:05.263949+0000","last_fullsized":"2023-08-25T03:34:05.263949+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T03:41:41.145686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169244+0000","last_change":"2023-08-25T03:34:03.627203+0000","last_active":"2023-08-25T03:34:05.169244+0000","last_peered":"2023-08-25T03:34:05.169244+0000","last_clean":"2023-08-25T03:34:05.169244+0000","last_became_active":"2023-08-25T03:34:03.626901+0000","last_became_peered":"2023-08-25T03:34:03.626901+0000","last_unstale":"2023-08-25T03:34:05.169244+0000","last_undegraded":"2023-08-25T03:34:05.169244+0000","last_fullsized":"2023-08-25T03:34:05.169244+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:23:05.976771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612301+0000","last_change":"2023-08-25T03:34:02.608430+0000","last_active":"2023-08-25T03:34:04.612301+0000","last_peered":"2023-08-25T03:34:04.612301+0000","last_clean":"2023-08-25T03:34:04.612301+0000","last_became_active":"2023-08-25T03:34:02.608086+0000","last_became_peered":"2023-08-25T03:34:02.608086+0000","last_unstale":"2023-08-25T03:34:04.612301+0000","last_undegraded":"2023-08-25T03:34:04.612301+0000","last_fullsized":"2023-08-25T03:34:04.612301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:23:48.303578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.350395+0000","last_change":"2023-08-25T03:34:03.624700+0000","last_active":"2023-08-25T03:34:05.350395+0000","last_peered":"2023-08-25T03:34:05.350395+0000","last_clean":"2023-08-25T03:34:05.350395+0000","last_became_active":"2023-08-25T03:34:03.624180+0000","last_became_peered":"2023-08-25T03:34:03.624180+0000","last_unstale":"2023-08-25T03:34:05.350395+0000","last_undegraded":"2023-08-25T03:34:05.350395+0000","last_fullsized":"2023-08-25T03:34:05.350395+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:27:15.899855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264384+0000","last_change":"2023-08-25T03:34:02.604587+0000","last_active":"2023-08-25T03:34:05.264384+0000","last_peered":"2023-08-25T03:34:05.264384+0000","last_clean":"2023-08-25T03:34:05.264384+0000","last_became_active":"2023-08-25T03:34:02.604290+0000","last_became_peered":"2023-08-25T03:34:02.604290+0000","last_unstale":"2023-08-25T03:34:05.264384+0000","last_undegraded":"2023-08-25T03:34:05.264384+0000","last_fullsized":"2023-08-25T03:34:05.264384+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:07:41.306323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611079+0000","last_change":"2023-08-25T03:34:03.613959+0000","last_active":"2023-08-25T03:34:04.611079+0000","last_peered":"2023-08-25T03:34:04.611079+0000","last_clean":"2023-08-25T03:34:04.611079+0000","last_became_active":"2023-08-25T03:34:03.613667+0000","last_became_peered":"2023-08-25T03:34:03.613667+0000","last_unstale":"2023-08-25T03:34:04.611079+0000","last_undegraded":"2023-08-25T03:34:04.611079+0000","last_fullsized":"2023-08-25T03:34:04.611079+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:41:33.001585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264371+0000","last_change":"2023-08-25T03:34:02.604585+0000","last_active":"2023-08-25T03:34:05.264371+0000","last_peered":"2023-08-25T03:34:05.264371+0000","last_clean":"2023-08-25T03:34:05.264371+0000","last_became_active":"2023-08-25T03:34:02.604255+0000","last_became_peered":"2023-08-25T03:34:02.604255+0000","last_unstale":"2023-08-25T03:34:05.264371+0000","last_undegraded":"2023-08-25T03:34:05.264371+0000","last_fullsized":"2023-08-25T03:34:05.264371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:31:14.838181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348736+0000","last_change":"2023-08-25T03:34:03.624731+0000","last_active":"2023-08-25T03:34:05.348736+0000","last_peered":"2023-08-25T03:34:05.348736+0000","last_clean":"2023-08-25T03:34:05.348736+0000","last_became_active":"2023-08-25T03:34:03.619454+0000","last_became_peered":"2023-08-25T03:34:03.619454+0000","last_unstale":"2023-08-25T03:34:05.348736+0000","last_undegraded":"2023-08-25T03:34:05.348736+0000","last_fullsized":"2023-08-25T03:34:05.348736+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:05:25.658159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264522+0000","last_change":"2023-08-25T03:34:02.605137+0000","last_active":"2023-08-25T03:34:05.264522+0000","last_peered":"2023-08-25T03:34:05.264522+0000","last_clean":"2023-08-25T03:34:05.264522+0000","last_became_active":"2023-08-25T03:34:02.604846+0000","last_became_peered":"2023-08-25T03:34:02.604846+0000","last_unstale":"2023-08-25T03:34:05.264522+0000","last_undegraded":"2023-08-25T03:34:05.264522+0000","last_fullsized":"2023-08-25T03:34:05.264522+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:44:39.385392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612781+0000","last_change":"2023-08-25T03:34:03.614885+0000","last_active":"2023-08-25T03:34:04.612781+0000","last_peered":"2023-08-25T03:34:04.612781+0000","last_clean":"2023-08-25T03:34:04.612781+0000","last_became_active":"2023-08-25T03:34:03.614420+0000","last_became_peered":"2023-08-25T03:34:03.614420+0000","last_unstale":"2023-08-25T03:34:04.612781+0000","last_undegraded":"2023-08-25T03:34:04.612781+0000","last_fullsized":"2023-08-25T03:34:04.612781+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:47:27.124952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611601+0000","last_change":"2023-08-25T03:34:02.598840+0000","last_active":"2023-08-25T03:34:04.611601+0000","last_peered":"2023-08-25T03:34:04.611601+0000","last_clean":"2023-08-25T03:34:04.611601+0000","last_became_active":"2023-08-25T03:34:02.598544+0000","last_became_peered":"2023-08-25T03:34:02.598544+0000","last_unstale":"2023-08-25T03:34:04.611601+0000","last_undegraded":"2023-08-25T03:34:04.611601+0000","last_fullsized":"2023-08-25T03:34:04.611601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:52:25.303751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.168983+0000","last_change":"2023-08-25T03:34:03.616738+0000","last_active":"2023-08-25T03:34:05.168983+0000","last_peered":"2023-08-25T03:34:05.168983+0000","last_clean":"2023-08-25T03:34:05.168983+0000","last_became_active":"2023-08-25T03:34:03.616450+0000","last_became_peered":"2023-08-25T03:34:03.616450+0000","last_unstale":"2023-08-25T03:34:05.168983+0000","last_undegraded":"2023-08-25T03:34:05.168983+0000","last_fullsized":"2023-08-25T03:34:05.168983+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:05:25.658159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349607+0000","last_change":"2023-08-25T03:34:02.605001+0000","last_active":"2023-08-25T03:34:05.349607+0000","last_peered":"2023-08-25T03:34:05.349607+0000","last_clean":"2023-08-25T03:34:05.349607+0000","last_became_active":"2023-08-25T03:34:02.604686+0000","last_became_peered":"2023-08-25T03:34:02.604686+0000","last_unstale":"2023-08-25T03:34:05.349607+0000","last_undegraded":"2023-08-25T03:34:05.349607+0000","last_fullsized":"2023-08-25T03:34:05.349607+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:10:03.545814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349133+0000","last_change":"2023-08-25T03:34:03.626817+0000","last_active":"2023-08-25T03:34:05.349133+0000","last_peered":"2023-08-25T03:34:05.349133+0000","last_clean":"2023-08-25T03:34:05.349133+0000","last_became_active":"2023-08-25T03:34:03.626483+0000","last_became_peered":"2023-08-25T03:34:03.626483+0000","last_unstale":"2023-08-25T03:34:05.349133+0000","last_undegraded":"2023-08-25T03:34:05.349133+0000","last_fullsized":"2023-08-25T03:34:05.349133+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:01:41.454184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612643+0000","last_change":"2023-08-25T03:34:02.601925+0000","last_active":"2023-08-25T03:34:04.612643+0000","last_peered":"2023-08-25T03:34:04.612643+0000","last_clean":"2023-08-25T03:34:04.612643+0000","last_became_active":"2023-08-25T03:34:02.601552+0000","last_became_peered":"2023-08-25T03:34:02.601552+0000","last_unstale":"2023-08-25T03:34:04.612643+0000","last_undegraded":"2023-08-25T03:34:04.612643+0000","last_fullsized":"2023-08-25T03:34:04.612643+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:47:38.956402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349296+0000","last_change":"2023-08-25T03:34:03.627975+0000","last_active":"2023-08-25T03:34:05.349296+0000","last_peered":"2023-08-25T03:34:05.349296+0000","last_clean":"2023-08-25T03:34:05.349296+0000","last_became_active":"2023-08-25T03:34:03.627489+0000","last_became_peered":"2023-08-25T03:34:03.627489+0000","last_unstale":"2023-08-25T03:34:05.349296+0000","last_undegraded":"2023-08-25T03:34:05.349296+0000","last_fullsized":"2023-08-25T03:34:05.349296+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:57:59.924993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348882+0000","last_change":"2023-08-25T03:34:02.603347+0000","last_active":"2023-08-25T03:34:05.348882+0000","last_peered":"2023-08-25T03:34:05.348882+0000","last_clean":"2023-08-25T03:34:05.348882+0000","last_became_active":"2023-08-25T03:34:02.603010+0000","last_became_peered":"2023-08-25T03:34:02.603010+0000","last_unstale":"2023-08-25T03:34:05.348882+0000","last_undegraded":"2023-08-25T03:34:05.348882+0000","last_fullsized":"2023-08-25T03:34:05.348882+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:07.999174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349594+0000","last_change":"2023-08-25T03:34:03.616962+0000","last_active":"2023-08-25T03:34:05.349594+0000","last_peered":"2023-08-25T03:34:05.349594+0000","last_clean":"2023-08-25T03:34:05.349594+0000","last_became_active":"2023-08-25T03:34:03.616656+0000","last_became_peered":"2023-08-25T03:34:03.616656+0000","last_unstale":"2023-08-25T03:34:05.349594+0000","last_undegraded":"2023-08-25T03:34:05.349594+0000","last_fullsized":"2023-08-25T03:34:05.349594+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:39:34.915801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348904+0000","last_change":"2023-08-25T03:34:02.603820+0000","last_active":"2023-08-25T03:34:05.348904+0000","last_peered":"2023-08-25T03:34:05.348904+0000","last_clean":"2023-08-25T03:34:05.348904+0000","last_became_active":"2023-08-25T03:34:02.603493+0000","last_became_peered":"2023-08-25T03:34:02.603493+0000","last_unstale":"2023-08-25T03:34:05.348904+0000","last_undegraded":"2023-08-25T03:34:05.348904+0000","last_fullsized":"2023-08-25T03:34:05.348904+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:47:12.826390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.350183+0000","last_change":"2023-08-25T03:34:03.625076+0000","last_active":"2023-08-25T03:34:05.350183+0000","last_peered":"2023-08-25T03:34:05.350183+0000","last_clean":"2023-08-25T03:34:05.350183+0000","last_became_active":"2023-08-25T03:34:03.624812+0000","last_became_peered":"2023-08-25T03:34:03.624812+0000","last_unstale":"2023-08-25T03:34:05.350183+0000","last_undegraded":"2023-08-25T03:34:05.350183+0000","last_fullsized":"2023-08-25T03:34:05.350183+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:27:15.899855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349701+0000","last_change":"2023-08-25T03:33:57.570267+0000","last_active":"2023-08-25T03:34:05.349701+0000","last_peered":"2023-08-25T03:34:05.349701+0000","last_clean":"2023-08-25T03:34:05.349701+0000","last_became_active":"2023-08-25T03:33:57.569882+0000","last_became_peered":"2023-08-25T03:33:57.569882+0000","last_unstale":"2023-08-25T03:34:05.349701+0000","last_undegraded":"2023-08-25T03:34:05.349701+0000","last_fullsized":"2023-08-25T03:34:05.349701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:09:58.529844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612129+0000","last_change":"2023-08-25T03:34:02.600129+0000","last_active":"2023-08-25T03:34:04.612129+0000","last_peered":"2023-08-25T03:34:04.612129+0000","last_clean":"2023-08-25T03:34:04.612129+0000","last_became_active":"2023-08-25T03:34:02.599762+0000","last_became_peered":"2023-08-25T03:34:02.599762+0000","last_unstale":"2023-08-25T03:34:04.612129+0000","last_undegraded":"2023-08-25T03:34:04.612129+0000","last_fullsized":"2023-08-25T03:34:04.612129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T03:41:41.145686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349944+0000","last_change":"2023-08-25T03:34:03.616199+0000","last_active":"2023-08-25T03:34:05.349944+0000","last_peered":"2023-08-25T03:34:05.349944+0000","last_clean":"2023-08-25T03:34:05.349944+0000","last_became_active":"2023-08-25T03:34:03.613500+0000","last_became_peered":"2023-08-25T03:34:03.613500+0000","last_unstale":"2023-08-25T03:34:05.349944+0000","last_undegraded":"2023-08-25T03:34:05.349944+0000","last_fullsized":"2023-08-25T03:34:05.349944+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:23:05.976771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611287+0000","last_change":"2023-08-25T03:33:57.569595+0000","last_active":"2023-08-25T03:34:04.611287+0000","last_peered":"2023-08-25T03:34:04.611287+0000","last_clean":"2023-08-25T03:34:04.611287+0000","last_became_active":"2023-08-25T03:33:57.569307+0000","last_became_peered":"2023-08-25T03:33:57.569307+0000","last_unstale":"2023-08-25T03:34:04.611287+0000","last_undegraded":"2023-08-25T03:34:04.611287+0000","last_fullsized":"2023-08-25T03:34:04.611287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:05:19.634775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349072+0000","last_change":"2023-08-25T03:34:02.605404+0000","last_active":"2023-08-25T03:34:05.349072+0000","last_peered":"2023-08-25T03:34:05.349072+0000","last_clean":"2023-08-25T03:34:05.349072+0000","last_became_active":"2023-08-25T03:34:02.604806+0000","last_became_peered":"2023-08-25T03:34:02.604806+0000","last_unstale":"2023-08-25T03:34:05.349072+0000","last_undegraded":"2023-08-25T03:34:05.349072+0000","last_fullsized":"2023-08-25T03:34:05.349072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:03:05.441386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349865+0000","last_change":"2023-08-25T03:34:03.632131+0000","last_active":"2023-08-25T03:34:05.349865+0000","last_peered":"2023-08-25T03:34:05.349865+0000","last_clean":"2023-08-25T03:34:05.349865+0000","last_became_active":"2023-08-25T03:34:03.631825+0000","last_became_peered":"2023-08-25T03:34:03.631825+0000","last_unstale":"2023-08-25T03:34:05.349865+0000","last_undegraded":"2023-08-25T03:34:05.349865+0000","last_fullsized":"2023-08-25T03:34:05.349865+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:09.006588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611334+0000","last_change":"2023-08-25T03:33:57.886704+0000","last_active":"2023-08-25T03:34:04.611334+0000","last_peered":"2023-08-25T03:34:04.611334+0000","last_clean":"2023-08-25T03:34:04.611334+0000","last_became_active":"2023-08-25T03:33:57.886294+0000","last_became_peered":"2023-08-25T03:33:57.886294+0000","last_unstale":"2023-08-25T03:34:04.611334+0000","last_undegraded":"2023-08-25T03:34:04.611334+0000","last_fullsized":"2023-08-25T03:34:04.611334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:15:31.545146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349162+0000","last_change":"2023-08-25T03:34:02.589214+0000","last_active":"2023-08-25T03:34:05.349162+0000","last_peered":"2023-08-25T03:34:05.349162+0000","last_clean":"2023-08-25T03:34:05.349162+0000","last_became_active":"2023-08-25T03:34:02.588607+0000","last_became_peered":"2023-08-25T03:34:02.588607+0000","last_unstale":"2023-08-25T03:34:05.349162+0000","last_undegraded":"2023-08-25T03:34:05.349162+0000","last_fullsized":"2023-08-25T03:34:05.349162+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:02:01.407883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.350043+0000","last_change":"2023-08-25T03:34:03.613767+0000","last_active":"2023-08-25T03:34:05.350043+0000","last_peered":"2023-08-25T03:34:05.350043+0000","last_clean":"2023-08-25T03:34:05.350043+0000","last_became_active":"2023-08-25T03:34:03.613433+0000","last_became_peered":"2023-08-25T03:34:03.613433+0000","last_unstale":"2023-08-25T03:34:05.350043+0000","last_undegraded":"2023-08-25T03:34:05.350043+0000","last_fullsized":"2023-08-25T03:34:05.350043+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:01:41.454184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611389+0000","last_change":"2023-08-25T03:33:57.570657+0000","last_active":"2023-08-25T03:34:04.611389+0000","last_peered":"2023-08-25T03:34:04.611389+0000","last_clean":"2023-08-25T03:34:04.611389+0000","last_became_active":"2023-08-25T03:33:57.570425+0000","last_became_peered":"2023-08-25T03:33:57.570425+0000","last_unstale":"2023-08-25T03:34:04.611389+0000","last_undegraded":"2023-08-25T03:34:04.611389+0000","last_fullsized":"2023-08-25T03:34:04.611389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:22:19.857784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":175,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.263861+0000","last_change":"2023-08-25T03:33:51.259898+0000","last_active":"2023-08-25T03:34:05.263861+0000","last_peered":"2023-08-25T03:34:05.263861+0000","last_clean":"2023-08-25T03:34:05.263861+0000","last_became_active":"2023-08-25T03:33:51.258780+0000","last_became_peered":"2023-08-25T03:33:51.258780+0000","last_unstale":"2023-08-25T03:34:05.263861+0000","last_undegraded":"2023-08-25T03:34:05.263861+0000","last_fullsized":"2023-08-25T03:34:05.263861+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:49.666858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:49.666858+0000","last_clean_scrub_stamp":"2023-08-25T03:33:49.666858+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:27:02.988993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349202+0000","last_change":"2023-08-25T03:34:02.598738+0000","last_active":"2023-08-25T03:34:05.349202+0000","last_peered":"2023-08-25T03:34:05.349202+0000","last_clean":"2023-08-25T03:34:05.349202+0000","last_became_active":"2023-08-25T03:34:02.598219+0000","last_became_peered":"2023-08-25T03:34:02.598219+0000","last_unstale":"2023-08-25T03:34:05.349202+0000","last_undegraded":"2023-08-25T03:34:05.349202+0000","last_fullsized":"2023-08-25T03:34:05.349202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:05:24.650744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612566+0000","last_change":"2023-08-25T03:34:03.624616+0000","last_active":"2023-08-25T03:34:04.612566+0000","last_peered":"2023-08-25T03:34:04.612566+0000","last_clean":"2023-08-25T03:34:04.612566+0000","last_became_active":"2023-08-25T03:34:03.624330+0000","last_became_peered":"2023-08-25T03:34:03.624330+0000","last_unstale":"2023-08-25T03:34:04.612566+0000","last_undegraded":"2023-08-25T03:34:04.612566+0000","last_fullsized":"2023-08-25T03:34:04.612566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:25:43.507515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264117+0000","last_change":"2023-08-25T03:33:57.571821+0000","last_active":"2023-08-25T03:34:05.264117+0000","last_peered":"2023-08-25T03:34:05.264117+0000","last_clean":"2023-08-25T03:34:05.264117+0000","last_became_active":"2023-08-25T03:33:57.571528+0000","last_became_peered":"2023-08-25T03:33:57.571528+0000","last_unstale":"2023-08-25T03:34:05.264117+0000","last_undegraded":"2023-08-25T03:34:05.264117+0000","last_fullsized":"2023-08-25T03:34:05.264117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:33:23.447280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.168795+0000","last_change":"2023-08-25T03:34:02.602275+0000","last_active":"2023-08-25T03:34:05.168795+0000","last_peered":"2023-08-25T03:34:05.168795+0000","last_clean":"2023-08-25T03:34:05.168795+0000","last_became_active":"2023-08-25T03:34:02.601960+0000","last_became_peered":"2023-08-25T03:34:02.601960+0000","last_unstale":"2023-08-25T03:34:05.168795+0000","last_undegraded":"2023-08-25T03:34:05.168795+0000","last_fullsized":"2023-08-25T03:34:05.168795+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:47:12.826390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169447+0000","last_change":"2023-08-25T03:34:03.612794+0000","last_active":"2023-08-25T03:34:05.169447+0000","last_peered":"2023-08-25T03:34:05.169447+0000","last_clean":"2023-08-25T03:34:05.169447+0000","last_became_active":"2023-08-25T03:34:03.612474+0000","last_became_peered":"2023-08-25T03:34:03.612474+0000","last_unstale":"2023-08-25T03:34:05.169447+0000","last_undegraded":"2023-08-25T03:34:05.169447+0000","last_fullsized":"2023-08-25T03:34:05.169447+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T03:41:42.153100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264364+0000","last_change":"2023-08-25T03:33:57.571730+0000","last_active":"2023-08-25T03:34:05.264364+0000","last_peered":"2023-08-25T03:34:05.264364+0000","last_clean":"2023-08-25T03:34:05.264364+0000","last_became_active":"2023-08-25T03:33:57.571442+0000","last_became_peered":"2023-08-25T03:33:57.571442+0000","last_unstale":"2023-08-25T03:34:05.264364+0000","last_undegraded":"2023-08-25T03:34:05.264364+0000","last_fullsized":"2023-08-25T03:34:05.264364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T03:49:14.356320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349890+0000","last_change":"2023-08-25T03:34:02.596876+0000","last_active":"2023-08-25T03:34:05.349890+0000","last_peered":"2023-08-25T03:34:05.349890+0000","last_clean":"2023-08-25T03:34:05.349890+0000","last_became_active":"2023-08-25T03:34:02.596556+0000","last_became_peered":"2023-08-25T03:34:02.596556+0000","last_unstale":"2023-08-25T03:34:05.349890+0000","last_undegraded":"2023-08-25T03:34:05.349890+0000","last_fullsized":"2023-08-25T03:34:05.349890+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:52:25.303751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169267+0000","last_change":"2023-08-25T03:34:03.616290+0000","last_active":"2023-08-25T03:34:05.169267+0000","last_peered":"2023-08-25T03:34:05.169267+0000","last_clean":"2023-08-25T03:34:05.169267+0000","last_became_active":"2023-08-25T03:34:03.616009+0000","last_became_peered":"2023-08-25T03:34:03.616009+0000","last_unstale":"2023-08-25T03:34:05.169267+0000","last_undegraded":"2023-08-25T03:34:05.169267+0000","last_fullsized":"2023-08-25T03:34:05.169267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:08.250998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.350304+0000","last_change":"2023-08-25T03:34:02.604771+0000","last_active":"2023-08-25T03:34:05.350304+0000","last_peered":"2023-08-25T03:34:05.350304+0000","last_clean":"2023-08-25T03:34:05.350304+0000","last_became_active":"2023-08-25T03:34:02.604509+0000","last_became_peered":"2023-08-25T03:34:02.604509+0000","last_unstale":"2023-08-25T03:34:05.350304+0000","last_undegraded":"2023-08-25T03:34:05.350304+0000","last_fullsized":"2023-08-25T03:34:05.350304+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:01:40.446770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349627+0000","last_change":"2023-08-25T03:33:57.570506+0000","last_active":"2023-08-25T03:34:05.349627+0000","last_peered":"2023-08-25T03:34:05.349627+0000","last_clean":"2023-08-25T03:34:05.349627+0000","last_became_active":"2023-08-25T03:33:57.570171+0000","last_became_peered":"2023-08-25T03:33:57.570171+0000","last_unstale":"2023-08-25T03:34:05.349627+0000","last_undegraded":"2023-08-25T03:34:05.349627+0000","last_fullsized":"2023-08-25T03:34:05.349627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:05:19.634775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348810+0000","last_change":"2023-08-25T03:34:03.622270+0000","last_active":"2023-08-25T03:34:05.348810+0000","last_peered":"2023-08-25T03:34:05.348810+0000","last_clean":"2023-08-25T03:34:05.348810+0000","last_became_active":"2023-08-25T03:34:03.621168+0000","last_became_peered":"2023-08-25T03:34:03.621168+0000","last_unstale":"2023-08-25T03:34:05.348810+0000","last_undegraded":"2023-08-25T03:34:05.348810+0000","last_fullsized":"2023-08-25T03:34:05.348810+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:59:37.641683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612720+0000","last_change":"2023-08-25T03:33:57.570598+0000","last_active":"2023-08-25T03:34:04.612720+0000","last_peered":"2023-08-25T03:34:04.612720+0000","last_clean":"2023-08-25T03:34:04.612720+0000","last_became_active":"2023-08-25T03:33:57.570215+0000","last_became_peered":"2023-08-25T03:33:57.570215+0000","last_unstale":"2023-08-25T03:34:04.612720+0000","last_undegraded":"2023-08-25T03:34:04.612720+0000","last_fullsized":"2023-08-25T03:34:04.612720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:40:56.425085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169511+0000","last_change":"2023-08-25T03:34:02.605323+0000","last_active":"2023-08-25T03:34:05.169511+0000","last_peered":"2023-08-25T03:34:05.169511+0000","last_clean":"2023-08-25T03:34:05.169511+0000","last_became_active":"2023-08-25T03:34:02.605111+0000","last_became_peered":"2023-08-25T03:34:02.605111+0000","last_unstale":"2023-08-25T03:34:05.169511+0000","last_undegraded":"2023-08-25T03:34:05.169511+0000","last_fullsized":"2023-08-25T03:34:05.169511+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:57:58.917579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348697+0000","last_change":"2023-08-25T03:34:03.624353+0000","last_active":"2023-08-25T03:34:05.348697+0000","last_peered":"2023-08-25T03:34:05.348697+0000","last_clean":"2023-08-25T03:34:05.348697+0000","last_became_active":"2023-08-25T03:34:03.618468+0000","last_became_peered":"2023-08-25T03:34:03.618468+0000","last_unstale":"2023-08-25T03:34:05.348697+0000","last_undegraded":"2023-08-25T03:34:05.348697+0000","last_fullsized":"2023-08-25T03:34:05.348697+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:41:33.001585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169012+0000","last_change":"2023-08-25T03:34:02.603419+0000","last_active":"2023-08-25T03:34:05.169012+0000","last_peered":"2023-08-25T03:34:05.169012+0000","last_clean":"2023-08-25T03:34:05.169012+0000","last_became_active":"2023-08-25T03:34:02.603082+0000","last_became_peered":"2023-08-25T03:34:02.603082+0000","last_unstale":"2023-08-25T03:34:05.169012+0000","last_undegraded":"2023-08-25T03:34:05.169012+0000","last_fullsized":"2023-08-25T03:34:05.169012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:22:24.873753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611520+0000","last_change":"2023-08-25T03:34:03.618585+0000","last_active":"2023-08-25T03:34:04.611520+0000","last_peered":"2023-08-25T03:34:04.611520+0000","last_clean":"2023-08-25T03:34:04.611520+0000","last_became_active":"2023-08-25T03:34:03.618291+0000","last_became_peered":"2023-08-25T03:34:03.618291+0000","last_unstale":"2023-08-25T03:34:04.611520+0000","last_undegraded":"2023-08-25T03:34:04.611520+0000","last_fullsized":"2023-08-25T03:34:04.611520+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:41:02.448469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611545+0000","last_change":"2023-08-25T03:34:02.601201+0000","last_active":"2023-08-25T03:34:04.611545+0000","last_peered":"2023-08-25T03:34:04.611545+0000","last_clean":"2023-08-25T03:34:04.611545+0000","last_became_active":"2023-08-25T03:34:02.600904+0000","last_became_peered":"2023-08-25T03:34:02.600904+0000","last_unstale":"2023-08-25T03:34:04.611545+0000","last_undegraded":"2023-08-25T03:34:04.611545+0000","last_fullsized":"2023-08-25T03:34:04.611545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:07.999174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349241+0000","last_change":"2023-08-25T03:34:03.626366+0000","last_active":"2023-08-25T03:34:05.349241+0000","last_peered":"2023-08-25T03:34:05.349241+0000","last_clean":"2023-08-25T03:34:05.349241+0000","last_became_active":"2023-08-25T03:34:03.625496+0000","last_became_peered":"2023-08-25T03:34:03.625496+0000","last_unstale":"2023-08-25T03:34:05.349241+0000","last_undegraded":"2023-08-25T03:34:05.349241+0000","last_fullsized":"2023-08-25T03:34:05.349241+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T03:41:42.153100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612260+0000","last_change":"2023-08-25T03:34:02.603423+0000","last_active":"2023-08-25T03:34:04.612260+0000","last_peered":"2023-08-25T03:34:04.612260+0000","last_clean":"2023-08-25T03:34:04.612260+0000","last_became_active":"2023-08-25T03:34:02.603077+0000","last_became_peered":"2023-08-25T03:34:02.603077+0000","last_unstale":"2023-08-25T03:34:04.612260+0000","last_undegraded":"2023-08-25T03:34:04.612260+0000","last_fullsized":"2023-08-25T03:34:04.612260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:57:58.917579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348880+0000","last_change":"2023-08-25T03:34:03.622038+0000","last_active":"2023-08-25T03:34:05.348880+0000","last_peered":"2023-08-25T03:34:05.348880+0000","last_clean":"2023-08-25T03:34:05.348880+0000","last_became_active":"2023-08-25T03:34:03.621747+0000","last_became_peered":"2023-08-25T03:34:03.621747+0000","last_unstale":"2023-08-25T03:34:05.348880+0000","last_undegraded":"2023-08-25T03:34:05.348880+0000","last_fullsized":"2023-08-25T03:34:05.348880+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:10:04.553228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.350130+0000","last_change":"2023-08-25T03:34:02.598272+0000","last_active":"2023-08-25T03:34:05.350130+0000","last_peered":"2023-08-25T03:34:05.350130+0000","last_clean":"2023-08-25T03:34:05.350130+0000","last_became_active":"2023-08-25T03:34:02.597630+0000","last_became_peered":"2023-08-25T03:34:02.597630+0000","last_unstale":"2023-08-25T03:34:05.350130+0000","last_undegraded":"2023-08-25T03:34:05.350130+0000","last_fullsized":"2023-08-25T03:34:05.350130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:03:05.441386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348872+0000","last_change":"2023-08-25T03:34:03.628379+0000","last_active":"2023-08-25T03:34:05.348872+0000","last_peered":"2023-08-25T03:34:05.348872+0000","last_clean":"2023-08-25T03:34:05.348872+0000","last_became_active":"2023-08-25T03:34:03.627870+0000","last_became_peered":"2023-08-25T03:34:03.627870+0000","last_unstale":"2023-08-25T03:34:05.348872+0000","last_undegraded":"2023-08-25T03:34:05.348872+0000","last_fullsized":"2023-08-25T03:34:05.348872+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:49:11.917056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169428+0000","last_change":"2023-08-25T03:34:02.597748+0000","last_active":"2023-08-25T03:34:05.169428+0000","last_peered":"2023-08-25T03:34:05.169428+0000","last_clean":"2023-08-25T03:34:05.169428+0000","last_became_active":"2023-08-25T03:34:02.597456+0000","last_became_peered":"2023-08-25T03:34:02.597456+0000","last_unstale":"2023-08-25T03:34:05.169428+0000","last_undegraded":"2023-08-25T03:34:05.169428+0000","last_fullsized":"2023-08-25T03:34:05.169428+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:01:40.446770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348450+0000","last_change":"2023-08-25T03:34:03.619886+0000","last_active":"2023-08-25T03:34:05.348450+0000","last_peered":"2023-08-25T03:34:05.348450+0000","last_clean":"2023-08-25T03:34:05.348450+0000","last_became_active":"2023-08-25T03:34:03.618445+0000","last_became_peered":"2023-08-25T03:34:03.618445+0000","last_unstale":"2023-08-25T03:34:05.348450+0000","last_undegraded":"2023-08-25T03:34:05.348450+0000","last_fullsized":"2023-08-25T03:34:05.348450+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:59.004627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612396+0000","last_change":"2023-08-25T03:34:03.612902+0000","last_active":"2023-08-25T03:34:04.612396+0000","last_peered":"2023-08-25T03:34:04.612396+0000","last_clean":"2023-08-25T03:34:04.612396+0000","last_became_active":"2023-08-25T03:34:03.612582+0000","last_became_peered":"2023-08-25T03:34:03.612582+0000","last_unstale":"2023-08-25T03:34:04.612396+0000","last_undegraded":"2023-08-25T03:34:04.612396+0000","last_fullsized":"2023-08-25T03:34:04.612396+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:01:15.550047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348872+0000","last_change":"2023-08-25T03:34:03.623559+0000","last_active":"2023-08-25T03:34:05.348872+0000","last_peered":"2023-08-25T03:34:05.348872+0000","last_clean":"2023-08-25T03:34:05.348872+0000","last_became_active":"2023-08-25T03:34:03.622782+0000","last_became_peered":"2023-08-25T03:34:03.622782+0000","last_unstale":"2023-08-25T03:34:05.348872+0000","last_undegraded":"2023-08-25T03:34:05.348872+0000","last_fullsized":"2023-08-25T03:34:05.348872+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:59.004627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348614+0000","last_change":"2023-08-25T03:34:03.622024+0000","last_active":"2023-08-25T03:34:05.348614+0000","last_peered":"2023-08-25T03:34:05.348614+0000","last_clean":"2023-08-25T03:34:05.348614+0000","last_became_active":"2023-08-25T03:34:03.619673+0000","last_became_peered":"2023-08-25T03:34:03.619673+0000","last_unstale":"2023-08-25T03:34:05.348614+0000","last_undegraded":"2023-08-25T03:34:05.348614+0000","last_fullsized":"2023-08-25T03:34:05.348614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:47:13.833805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349801+0000","last_change":"2023-08-25T03:34:03.621260+0000","last_active":"2023-08-25T03:34:05.349801+0000","last_peered":"2023-08-25T03:34:05.349801+0000","last_clean":"2023-08-25T03:34:05.349801+0000","last_became_active":"2023-08-25T03:34:03.620890+0000","last_became_peered":"2023-08-25T03:34:03.620890+0000","last_unstale":"2023-08-25T03:34:05.349801+0000","last_undegraded":"2023-08-25T03:34:05.349801+0000","last_fullsized":"2023-08-25T03:34:05.349801+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:52:26.311165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349539+0000","last_change":"2023-08-25T03:34:03.615228+0000","last_active":"2023-08-25T03:34:05.349539+0000","last_peered":"2023-08-25T03:34:05.349539+0000","last_clean":"2023-08-25T03:34:05.349539+0000","last_became_active":"2023-08-25T03:34:03.613949+0000","last_became_peered":"2023-08-25T03:34:03.613949+0000","last_unstale":"2023-08-25T03:34:05.349539+0000","last_undegraded":"2023-08-25T03:34:05.349539+0000","last_fullsized":"2023-08-25T03:34:05.349539+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:41:33.001585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542152,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":872448,"data_stored":685346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]}]},{"osd":6,"up_from":14,"seq":60129542152,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":249856,"data_stored":93150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":249856,"data_stored":93150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":847872,"data_stored":684100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77900000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":249856,"data_stored":93150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":93240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.373}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]}]},{"osd":4,"up_from":14,"seq":60129542152,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":94818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66200000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542151,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":93802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-25T03:34:12.767 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-25T03:34:12.768 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-08-25T03:34:12.768 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-08-25T03:34:12.768 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-08-25T03:34:12.769 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-08-25T03:34:12.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.924+0000 7f720cf56700 1 Processor -- start 2023-08-25T03:34:12.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.924+0000 7f720cf56700 1 -- start start 2023-08-25T03:34:12.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.925+0000 7f720cf56700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f720815b130 0x7f720806db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:12.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.925+0000 7f720cf56700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f720806bbe0 0x7f72080b19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:12.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.925+0000 7f720cf56700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72080b2330 con 0x7f72080b1ee0 2023-08-25T03:34:12.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.925+0000 7f720cf56700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f720815fdc0 con 0x7f720815b130 2023-08-25T03:34:12.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.925+0000 7f720cf56700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7208071ea0 con 0x7f720806bbe0 2023-08-25T03:34:12.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.925+0000 7f720659c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f720815b130 0x7f720806db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:12.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.925+0000 7f720659c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f720815b130 0x7f720806db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59302/0 (socket says 172.21.15.2:59302) 2023-08-25T03:34:12.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.925+0000 7f720659c700 1 -- 172.21.15.2:0/1606681484 learned_addr learned my addr 172.21.15.2:0/1606681484 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:12.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.925+0000 7f7206d9d700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f72080b1ee0 0x7f720806a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:45970/0 (socket says 172.21.15.2:45970) 2023-08-25T03:34:12.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.926+0000 7f7205d9b700 1 --2- 172.21.15.2:0/1606681484 >> v2:172.21.15.31:3300/0 conn(0x7f720806bbe0 0x7f72080b19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:12.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.926+0000 7f720659c700 1 -- 172.21.15.2:0/1606681484 >> v2:172.21.15.31:3300/0 conn(0x7f720806bbe0 msgr2=0x7f72080b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:12.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.926+0000 7f720659c700 1 --2- 172.21.15.2:0/1606681484 >> v2:172.21.15.31:3300/0 conn(0x7f720806bbe0 0x7f72080b19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:12.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.926+0000 7f720659c700 1 -- 172.21.15.2:0/1606681484 >> v1:172.21.15.2:6789/0 conn(0x7f72080b1ee0 legacy=0x7f720806a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:12.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.926+0000 7f720659c700 1 -- 172.21.15.2:0/1606681484 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7208071a10 con 0x7f720815b130 2023-08-25T03:34:12.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.926+0000 7f720559a700 1 -- 172.21.15.2:0/1606681484 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3646380708 0 0) 0x7f7200001490 con 0x7f72080b1ee0 2023-08-25T03:34:12.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.926+0000 7f720659c700 1 --2- 172.21.15.2:0/1606681484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f720815b130 0x7f720806db10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f72080b2810 tx=0x7f71fc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c0989ca3c78c05 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:12.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.926+0000 7f720559a700 1 -- 172.21.15.2:0/1606681484 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc00b860 con 0x7f720815b130 2023-08-25T03:34:12.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.926+0000 7f720559a700 1 -- 172.21.15.2:0/1606681484 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71fc00b9c0 con 0x7f720815b130 2023-08-25T03:34:12.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.926+0000 7f720559a700 1 -- 172.21.15.2:0/1606681484 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc00bd50 con 0x7f720815b130 2023-08-25T03:34:12.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.926+0000 7f720cf56700 1 -- 172.21.15.2:0/1606681484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f720815b130 msgr2=0x7f720806db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:12.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.926+0000 7f720cf56700 1 --2- 172.21.15.2:0/1606681484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f720815b130 0x7f720806db10 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f72080b2810 tx=0x7f71fc009c00 comp rx=0 tx=0).stop 2023-08-25T03:34:12.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.927+0000 7f720cf56700 1 -- 172.21.15.2:0/1606681484 shutdown_connections 2023-08-25T03:34:12.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.927+0000 7f720cf56700 1 --2- 172.21.15.2:0/1606681484 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f720815b130 0x7f720806db10 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:12.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.927+0000 7f720cf56700 1 --2- 172.21.15.2:0/1606681484 >> v2:172.21.15.31:3300/0 conn(0x7f720806bbe0 0x7f72080b19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:12.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.927+0000 7f720cf56700 1 -- 172.21.15.2:0/1606681484 >> 172.21.15.2:0/1606681484 conn(0x7f72080cfea0 msgr2=0x7f72080d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:12.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.927+0000 7f720cf56700 1 -- 172.21.15.2:0/1606681484 shutdown_connections 2023-08-25T03:34:12.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.927+0000 7f720cf56700 1 -- 172.21.15.2:0/1606681484 wait complete. 2023-08-25T03:34:12.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.928+0000 7f720cf56700 1 Processor -- start 2023-08-25T03:34:12.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.928+0000 7f720cf56700 1 -- start start 2023-08-25T03:34:12.933 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.929+0000 7f720cf56700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f720806bbe0 0x7f720815d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:12.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.929+0000 7f720cf56700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72080b1ee0 0x7f72080c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:12.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.929+0000 7f720cf56700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f720815b130 0x7f720814dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:12.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.929+0000 7f720cf56700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720815cef0 con 0x7f720806bbe0 2023-08-25T03:34:12.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.929+0000 7f720cf56700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72080b3df0 con 0x7f72080b1ee0 2023-08-25T03:34:12.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.929+0000 7f720cf56700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720815c5c0 con 0x7f720815b130 2023-08-25T03:34:12.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.929+0000 7f720659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f720806bbe0 0x7f720815d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:12.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.929+0000 7f720659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f720806bbe0 0x7f720815d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59308/0 (socket says 172.21.15.2:59308) 2023-08-25T03:34:12.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.929+0000 7f720659c700 1 -- 172.21.15.2:0/290074376 learned_addr learned my addr 172.21.15.2:0/290074376 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:12.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.929+0000 7f720659c700 1 -- 172.21.15.2:0/290074376 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f720815b130 msgr2=0x7f720814dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:12.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.929+0000 7f720659c700 1 --2- 172.21.15.2:0/290074376 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f720815b130 0x7f720814dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:12.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.929+0000 7f7205d9b700 1 --2- 172.21.15.2:0/290074376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72080b1ee0 0x7f72080c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:12.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.929+0000 7f720659c700 1 -- 172.21.15.2:0/290074376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72080b1ee0 msgr2=0x7f72080c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:12.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.929+0000 7f720659c700 1 --2- 172.21.15.2:0/290074376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72080b1ee0 0x7f72080c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:12.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.929+0000 7f720659c700 1 -- 172.21.15.2:0/290074376 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc009550 con 0x7f720806bbe0 2023-08-25T03:34:12.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.929+0000 7f7205d9b700 1 --2- 172.21.15.2:0/290074376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72080b1ee0 0x7f72080c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:12.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.929+0000 7f720659c700 1 --2- 172.21.15.2:0/290074376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f720806bbe0 0x7f720815d380 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009c40 tx=0x7f71fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:12.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.929+0000 7f71f6ffd700 1 -- 172.21.15.2:0/290074376 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc0044f0 con 0x7f720806bbe0 2023-08-25T03:34:12.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.929+0000 7f71f6ffd700 1 -- 172.21.15.2:0/290074376 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71fc01f070 con 0x7f720806bbe0 2023-08-25T03:34:12.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.929+0000 7f720cf56700 1 -- 172.21.15.2:0/290074376 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f720815c8a0 con 0x7f720806bbe0 2023-08-25T03:34:12.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.929+0000 7f71f6ffd700 1 -- 172.21.15.2:0/290074376 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71fc0038a0 con 0x7f720806bbe0 2023-08-25T03:34:12.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.930+0000 7f720cf56700 1 -- 172.21.15.2:0/290074376 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72080c9880 con 0x7f720806bbe0 2023-08-25T03:34:12.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.931+0000 7f720cf56700 1 -- 172.21.15.2:0/290074376 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7208066f70 con 0x7f720806bbe0 2023-08-25T03:34:12.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.931+0000 7f71f6ffd700 1 -- 172.21.15.2:0/290074376 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f71fc021030 con 0x7f720806bbe0 2023-08-25T03:34:12.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.931+0000 7f71f6ffd700 1 --2- 172.21.15.2:0/290074376 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f71e806a0e0 0x7f71e806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:12.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.931+0000 7f71f6ffd700 1 -- 172.21.15.2:0/290074376 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f71fc089a60 con 0x7f720806bbe0 2023-08-25T03:34:12.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.931+0000 7f7205d9b700 1 --2- 172.21.15.2:0/290074376 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f71e806a0e0 0x7f71e806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:12.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.932+0000 7f7205d9b700 1 --2- 172.21.15.2:0/290074376 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f71e806a0e0 0x7f71e806c5a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f71f8006fd0 tx=0x7f71f8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:12.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:12.936+0000 7f71f6ffd700 1 -- 172.21.15.2:0/290074376 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f71fc055140 con 0x7f720806bbe0 2023-08-25T03:34:13.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.190+0000 7f720cf56700 1 -- 172.21.15.2:0/290074376 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f72080746d0 con 0x7f720806bbe0 2023-08-25T03:34:13.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.194+0000 7f71f6ffd700 1 -- 172.21.15.2:0/290074376 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+169485 (secure 0 0 0) 0x7f71fc015090 con 0x7f720806bbe0 2023-08-25T03:34:13.197 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:13.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.196+0000 7f720cf56700 1 -- 172.21.15.2:0/290074376 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f71e806a0e0 msgr2=0x7f71e806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:13.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.197+0000 7f720cf56700 1 --2- 172.21.15.2:0/290074376 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f71e806a0e0 0x7f71e806c5a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f71f8006fd0 tx=0x7f71f8008040 comp rx=0 tx=0).stop 2023-08-25T03:34:13.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.197+0000 7f720cf56700 1 -- 172.21.15.2:0/290074376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f720806bbe0 msgr2=0x7f720815d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:13.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.197+0000 7f720cf56700 1 --2- 172.21.15.2:0/290074376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f720806bbe0 0x7f720815d380 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f71fc009c40 tx=0x7f71fc004750 comp rx=0 tx=0).stop 2023-08-25T03:34:13.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.197+0000 7f720cf56700 1 -- 172.21.15.2:0/290074376 shutdown_connections 2023-08-25T03:34:13.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.197+0000 7f720cf56700 1 --2- 172.21.15.2:0/290074376 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f71e806a0e0 0x7f71e806c5a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:13.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.197+0000 7f720cf56700 1 --2- 172.21.15.2:0/290074376 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f720806bbe0 0x7f720815d380 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:13.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.197+0000 7f720cf56700 1 --2- 172.21.15.2:0/290074376 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f720815b130 0x7f720814dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:13.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.197+0000 7f720cf56700 1 --2- 172.21.15.2:0/290074376 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f72080b1ee0 0x7f72080c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:13.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.197+0000 7f720cf56700 1 -- 172.21.15.2:0/290074376 >> 172.21.15.2:0/290074376 conn(0x7f72080cfea0 msgr2=0x7f72080cc830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:13.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.197+0000 7f720cf56700 1 -- 172.21.15.2:0/290074376 shutdown_connections 2023-08-25T03:34:13.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.197+0000 7f720cf56700 1 -- 172.21.15.2:0/290074376 wait complete. 2023-08-25T03:34:13.209 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":5,"active_gid":4108,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6832","nonce":3528779779},{"type":"v1","addr":"172.21.15.2:6833","nonce":3528779779}]},"active_addr":"172.21.15.2:6833/3528779779","active_change":"2023-08-25T03:33:43.492362+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":3820084411}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":1659654718}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":2578994439}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.2:0","nonce":3157212133}]}]} 2023-08-25T03:34:13.214 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-08-25T03:34:13.215 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-08-25T03:34:13.215 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:34:13.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.339+0000 7fcd2a020700 1 Processor -- start 2023-08-25T03:34:13.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.339+0000 7fcd2a020700 1 -- start start 2023-08-25T03:34:13.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.340+0000 7fcd2a020700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fcd2415e890 0x7fcd24071430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:13.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.340+0000 7fcd2a020700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd240caea0 0x7fcd240cb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:13.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.340+0000 7fcd2a020700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd240720b0 con 0x7fcd240b19a0 2023-08-25T03:34:13.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.340+0000 7fcd2a020700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd24071dd0 con 0x7fcd2415e890 2023-08-25T03:34:13.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.340+0000 7fcd2a020700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2415ddb0 con 0x7fcd240caea0 2023-08-25T03:34:13.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.340+0000 7fcd22ffd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fcd2415e890 0x7fcd24071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:13.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.340+0000 7fcd22ffd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fcd2415e890 0x7fcd24071430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59318/0 (socket says 172.21.15.2:59318) 2023-08-25T03:34:13.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.340+0000 7fcd22ffd700 1 -- 172.21.15.2:0/587477377 learned_addr learned my addr 172.21.15.2:0/587477377 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:13.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.340+0000 7fcd23fff700 1 --2- 172.21.15.2:0/587477377 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd240caea0 0x7fcd240cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:13.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.340+0000 7fcd22ffd700 1 -- 172.21.15.2:0/587477377 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd240caea0 msgr2=0x7fcd240cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:13.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.340+0000 7fcd22ffd700 1 --2- 172.21.15.2:0/587477377 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd240caea0 0x7fcd240cb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:13.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.340+0000 7fcd22ffd700 1 -- 172.21.15.2:0/587477377 >> v1:172.21.15.31:6789/0 conn(0x7fcd240b19a0 legacy=0x7fcd240b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:13.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.340+0000 7fcd22ffd700 1 -- 172.21.15.2:0/587477377 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd2415e090 con 0x7fcd2415e890 2023-08-25T03:34:13.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.340+0000 7fcd22ffd700 1 --2- 172.21.15.2:0/587477377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd2415e890 0x7fcd24071430 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800bb90 tx=0x7fcd1800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771dc78eabbb9ec4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:13.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.340+0000 7fcd227fc700 1 -- 172.21.15.2:0/587477377 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1800dd10 con 0x7fcd2415e890 2023-08-25T03:34:13.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.341+0000 7fcd227fc700 1 -- 172.21.15.2:0/587477377 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd1800de70 con 0x7fcd2415e890 2023-08-25T03:34:13.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.341+0000 7fcd227fc700 1 -- 172.21.15.2:0/587477377 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd18004230 con 0x7fcd2415e890 2023-08-25T03:34:13.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.341+0000 7fcd2a020700 1 -- 172.21.15.2:0/587477377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd2415e890 msgr2=0x7fcd24071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:13.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.341+0000 7fcd2a020700 1 --2- 172.21.15.2:0/587477377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd2415e890 0x7fcd24071430 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800bb90 tx=0x7fcd1800bea0 comp rx=0 tx=0).stop 2023-08-25T03:34:13.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.341+0000 7fcd2a020700 1 -- 172.21.15.2:0/587477377 shutdown_connections 2023-08-25T03:34:13.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.341+0000 7fcd2a020700 1 --2- 172.21.15.2:0/587477377 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd240caea0 0x7fcd240cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:13.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.341+0000 7fcd2a020700 1 --2- 172.21.15.2:0/587477377 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd2415e890 0x7fcd24071430 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:13.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.341+0000 7fcd2a020700 1 -- 172.21.15.2:0/587477377 >> 172.21.15.2:0/587477377 conn(0x7fcd240cfea0 msgr2=0x7fcd240d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:13.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.341+0000 7fcd2a020700 1 -- 172.21.15.2:0/587477377 shutdown_connections 2023-08-25T03:34:13.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.341+0000 7fcd2a020700 1 -- 172.21.15.2:0/587477377 wait complete. 2023-08-25T03:34:13.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.343+0000 7fcd2a020700 1 Processor -- start 2023-08-25T03:34:13.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.343+0000 7fcd2a020700 1 -- start start 2023-08-25T03:34:13.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.344+0000 7fcd2a020700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd240b19a0 0x7fcd241ef400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:13.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.344+0000 7fcd2a020700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd240caea0 0x7fcd240c82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:13.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.344+0000 7fcd2a020700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd2415e890 0x7fcd24157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:13.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.344+0000 7fcd2a020700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd241ef940 con 0x7fcd240caea0 2023-08-25T03:34:13.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.344+0000 7fcd2a020700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2406b8c0 con 0x7fcd2415e890 2023-08-25T03:34:13.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.344+0000 7fcd2a020700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2406acb0 con 0x7fcd240b19a0 2023-08-25T03:34:13.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.344+0000 7fcd22ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd240caea0 0x7fcd240c82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:13.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.344+0000 7fcd22ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd240caea0 0x7fcd240c82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59324/0 (socket says 172.21.15.2:59324) 2023-08-25T03:34:13.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.344+0000 7fcd22ffd700 1 -- 172.21.15.2:0/326210271 learned_addr learned my addr 172.21.15.2:0/326210271 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:13.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.344+0000 7fcd237fe700 1 --2- 172.21.15.2:0/326210271 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd240b19a0 0x7fcd241ef400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:13.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.344+0000 7fcd22ffd700 1 -- 172.21.15.2:0/326210271 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd240b19a0 msgr2=0x7fcd241ef400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:13.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.344+0000 7fcd22ffd700 1 --2- 172.21.15.2:0/326210271 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd240b19a0 0x7fcd241ef400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:13.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.344+0000 7fcd23fff700 1 --2- 172.21.15.2:0/326210271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd2415e890 0x7fcd24157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:13.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.344+0000 7fcd22ffd700 1 -- 172.21.15.2:0/326210271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd2415e890 msgr2=0x7fcd24157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:13.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.344+0000 7fcd22ffd700 1 --2- 172.21.15.2:0/326210271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd2415e890 0x7fcd24157400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:13.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.344+0000 7fcd22ffd700 1 -- 172.21.15.2:0/326210271 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1800b7f0 con 0x7fcd240caea0 2023-08-25T03:34:13.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.344+0000 7fcd237fe700 1 --2- 172.21.15.2:0/326210271 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd240b19a0 0x7fcd241ef400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:13.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.344+0000 7fcd22ffd700 1 --2- 172.21.15.2:0/326210271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd240caea0 0x7fcd240c82f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800b340 tx=0x7fcd18004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:13.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.345+0000 7fcd0bfff700 1 -- 172.21.15.2:0/326210271 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd18020070 con 0x7fcd240caea0 2023-08-25T03:34:13.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.345+0000 7fcd0bfff700 1 -- 172.21.15.2:0/326210271 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd18003aa0 con 0x7fcd240caea0 2023-08-25T03:34:13.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.345+0000 7fcd2a020700 1 -- 172.21.15.2:0/326210271 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd2406af90 con 0x7fcd240caea0 2023-08-25T03:34:13.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.345+0000 7fcd0bfff700 1 -- 172.21.15.2:0/326210271 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd18003e50 con 0x7fcd240caea0 2023-08-25T03:34:13.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.345+0000 7fcd2a020700 1 -- 172.21.15.2:0/326210271 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd2406b550 con 0x7fcd240caea0 2023-08-25T03:34:13.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.345+0000 7fcd2a020700 1 -- 172.21.15.2:0/326210271 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd2406b6e0 con 0x7fcd240caea0 2023-08-25T03:34:13.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.346+0000 7fcd0bfff700 1 -- 172.21.15.2:0/326210271 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcd1802d3f0 con 0x7fcd240caea0 2023-08-25T03:34:13.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.346+0000 7fcd0bfff700 1 --2- 172.21.15.2:0/326210271 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fcd1006a0e0 0x7fcd1006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:13.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.347+0000 7fcd0bfff700 1 -- 172.21.15.2:0/326210271 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcd1808b7d0 con 0x7fcd240caea0 2023-08-25T03:34:13.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.347+0000 7fcd237fe700 1 --2- 172.21.15.2:0/326210271 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fcd1006a0e0 0x7fcd1006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:13.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.347+0000 7fcd237fe700 1 --2- 172.21.15.2:0/326210271 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fcd1006a0e0 0x7fcd1006c5a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcd14003f40 tx=0x7fcd14009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:13.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.351+0000 7fcd0bfff700 1 -- 172.21.15.2:0/326210271 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcd18056f50 con 0x7fcd240caea0 2023-08-25T03:34:13.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.553+0000 7fcd2a020700 1 -- 172.21.15.2:0/326210271 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd24066f70 con 0x7fcd240caea0 2023-08-25T03:34:13.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.554+0000 7fcd0bfff700 1 -- 172.21.15.2:0/326210271 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18135 (secure 0 0 0) 0x7fcd1805a570 con 0x7fcd240caea0 2023-08-25T03:34:13.555 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:13.556 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":24,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:34:04.604059+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:34:00.869118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T03:34:01.987016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:34:13.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.556+0000 7fcd2a020700 1 -- 172.21.15.2:0/326210271 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fcd1006a0e0 msgr2=0x7fcd1006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:13.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.556+0000 7fcd2a020700 1 --2- 172.21.15.2:0/326210271 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fcd1006a0e0 0x7fcd1006c5a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcd14003f40 tx=0x7fcd14009040 comp rx=0 tx=0).stop 2023-08-25T03:34:13.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.556+0000 7fcd2a020700 1 -- 172.21.15.2:0/326210271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd240caea0 msgr2=0x7fcd240c82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:13.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.556+0000 7fcd2a020700 1 --2- 172.21.15.2:0/326210271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd240caea0 0x7fcd240c82f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fcd1800b340 tx=0x7fcd18004750 comp rx=0 tx=0).stop 2023-08-25T03:34:13.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.556+0000 7fcd2a020700 1 -- 172.21.15.2:0/326210271 shutdown_connections 2023-08-25T03:34:13.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.556+0000 7fcd2a020700 1 --2- 172.21.15.2:0/326210271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd2415e890 0x7fcd24157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:13.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.556+0000 7fcd2a020700 1 --2- 172.21.15.2:0/326210271 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd240b19a0 0x7fcd241ef400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:13.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.556+0000 7fcd2a020700 1 --2- 172.21.15.2:0/326210271 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fcd1006a0e0 0x7fcd1006c5a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:13.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.556+0000 7fcd2a020700 1 --2- 172.21.15.2:0/326210271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcd240caea0 0x7fcd240c82f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:13.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.556+0000 7fcd2a020700 1 -- 172.21.15.2:0/326210271 >> 172.21.15.2:0/326210271 conn(0x7fcd240cfea0 msgr2=0x7fcd240cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:13.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.556+0000 7fcd2a020700 1 -- 172.21.15.2:0/326210271 shutdown_connections 2023-08-25T03:34:13.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.556+0000 7fcd2a020700 1 -- 172.21.15.2:0/326210271 wait complete. 2023-08-25T03:34:13.570 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-08-25T03:34:13.570 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:34:13.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.695+0000 7f48a4642700 1 Processor -- start 2023-08-25T03:34:13.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.695+0000 7f48a4642700 1 -- start start 2023-08-25T03:34:13.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.696+0000 7f48a4642700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f489c15a030 0x7f489c0cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:13.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.696+0000 7f48a4642700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f489c0cc1f0 0x7f489c0ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:13.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.696+0000 7f48a4642700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f489c071dc0 con 0x7f489c15a410 2023-08-25T03:34:13.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.696+0000 7f48a4642700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f489c0720a0 con 0x7f489c15a030 2023-08-25T03:34:13.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.696+0000 7f48a4642700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489c0cb4b0 con 0x7f489c0cc1f0 2023-08-25T03:34:13.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.696+0000 7f48a2bdf700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f489c15a410 0x7f489c0ccd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46000/0 (socket says 172.21.15.2:46000) 2023-08-25T03:34:13.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.696+0000 7f48a2bdf700 1 -- 172.21.15.2:0/1976652908 learned_addr learned my addr 172.21.15.2:0/1976652908 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:13.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.696+0000 7f48a23de700 1 --2- 172.21.15.2:0/1976652908 >> v2:172.21.15.31:3300/0 conn(0x7f489c15a030 0x7f489c0cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:13.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.696+0000 7f48a1bdd700 1 --2- 172.21.15.2:0/1976652908 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f489c0cc1f0 0x7f489c0ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:13.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.696+0000 7f48a13dc700 1 -- 172.21.15.2:0/1976652908 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2989043161 0 0) 0x7f489c071dc0 con 0x7f489c15a410 2023-08-25T03:34:13.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.696+0000 7f48a13dc700 1 -- 172.21.15.2:0/1976652908 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48880035b0 con 0x7f489c15a410 2023-08-25T03:34:13.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.697+0000 7f48a23de700 1 -- 172.21.15.2:0/1976652908 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f489c0cc1f0 msgr2=0x7f489c0ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:13.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.697+0000 7f48a23de700 1 --2- 172.21.15.2:0/1976652908 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f489c0cc1f0 0x7f489c0ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:13.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.697+0000 7f48a23de700 1 -- 172.21.15.2:0/1976652908 >> v1:172.21.15.2:6789/0 conn(0x7f489c15a410 legacy=0x7f489c0ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:13.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.697+0000 7f48a23de700 1 -- 172.21.15.2:0/1976652908 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489c0cb790 con 0x7f489c15a030 2023-08-25T03:34:13.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.697+0000 7f48a1bdd700 1 --2- 172.21.15.2:0/1976652908 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f489c0cc1f0 0x7f489c0ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:34:13.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.697+0000 7f48a23de700 1 --2- 172.21.15.2:0/1976652908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f489c15a030 0x7f489c0cbcb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4898009840 tx=0x7f4898009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54fffc138ffe35ec server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:13.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.697+0000 7f48a13dc700 1 -- 172.21.15.2:0/1976652908 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4898011830 con 0x7f489c15a030 2023-08-25T03:34:13.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.697+0000 7f48a13dc700 1 -- 172.21.15.2:0/1976652908 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4898011990 con 0x7f489c15a030 2023-08-25T03:34:13.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.697+0000 7f48a13dc700 1 -- 172.21.15.2:0/1976652908 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4898011d00 con 0x7f489c15a030 2023-08-25T03:34:13.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.698+0000 7f48a4642700 1 -- 172.21.15.2:0/1976652908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f489c15a030 msgr2=0x7f489c0cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:13.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.698+0000 7f48a4642700 1 --2- 172.21.15.2:0/1976652908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f489c15a030 0x7f489c0cbcb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4898009840 tx=0x7f4898009b50 comp rx=0 tx=0).stop 2023-08-25T03:34:13.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.698+0000 7f48a4642700 1 -- 172.21.15.2:0/1976652908 shutdown_connections 2023-08-25T03:34:13.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.698+0000 7f48a4642700 1 --2- 172.21.15.2:0/1976652908 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f489c15a030 0x7f489c0cbcb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:13.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.698+0000 7f48a4642700 1 --2- 172.21.15.2:0/1976652908 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f489c0cc1f0 0x7f489c0ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:13.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.698+0000 7f48a4642700 1 -- 172.21.15.2:0/1976652908 >> 172.21.15.2:0/1976652908 conn(0x7f489c0b1cf0 msgr2=0x7f489c0b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:13.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.698+0000 7f48a4642700 1 -- 172.21.15.2:0/1976652908 shutdown_connections 2023-08-25T03:34:13.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.698+0000 7f48a4642700 1 -- 172.21.15.2:0/1976652908 wait complete. 2023-08-25T03:34:13.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.699+0000 7f48a4642700 1 Processor -- start 2023-08-25T03:34:13.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.699+0000 7f48a4642700 1 -- start start 2023-08-25T03:34:13.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.700+0000 7f48a4642700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f489c0cc1f0 0x7f489c14dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:13.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.700+0000 7f48a4642700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f489c15a030 0x7f489c06a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:13.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.700+0000 7f48a4642700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f489c15a410 0x7f489c15a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:13.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.700+0000 7f48a4642700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489c149a50 con 0x7f489c15a030 2023-08-25T03:34:13.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.700+0000 7f48a4642700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489c15cad0 con 0x7f489c0cc1f0 2023-08-25T03:34:13.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.700+0000 7f48a4642700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489c15bee0 con 0x7f489c15a410 2023-08-25T03:34:13.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.700+0000 7f48a1bdd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f489c15a030 0x7f489c06a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:13.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.700+0000 7f48a1bdd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f489c15a030 0x7f489c06a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59338/0 (socket says 172.21.15.2:59338) 2023-08-25T03:34:13.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.700+0000 7f48a1bdd700 1 -- 172.21.15.2:0/1897210413 learned_addr learned my addr 172.21.15.2:0/1897210413 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:13.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.700+0000 7f48a2bdf700 1 --2- 172.21.15.2:0/1897210413 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f489c15a410 0x7f489c15a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:13.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.700+0000 7f48a23de700 1 --2- 172.21.15.2:0/1897210413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f489c0cc1f0 0x7f489c14dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:13.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.700+0000 7f48a1bdd700 1 -- 172.21.15.2:0/1897210413 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f489c15a410 msgr2=0x7f489c15a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:13.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.700+0000 7f48a1bdd700 1 --2- 172.21.15.2:0/1897210413 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f489c15a410 0x7f489c15a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:13.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.700+0000 7f48a1bdd700 1 -- 172.21.15.2:0/1897210413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f489c0cc1f0 msgr2=0x7f489c14dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:13.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.700+0000 7f48a1bdd700 1 --2- 172.21.15.2:0/1897210413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f489c0cc1f0 0x7f489c14dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:13.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.700+0000 7f48a1bdd700 1 -- 172.21.15.2:0/1897210413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48900094a0 con 0x7f489c15a030 2023-08-25T03:34:13.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.700+0000 7f48a2bdf700 1 --2- 172.21.15.2:0/1897210413 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f489c15a410 0x7f489c15a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:13.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.700+0000 7f48a23de700 1 --2- 172.21.15.2:0/1897210413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f489c0cc1f0 0x7f489c14dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:13.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.700+0000 7f48a1bdd700 1 --2- 172.21.15.2:0/1897210413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f489c15a030 0x7f489c06a650 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f489000ebe0 tx=0x7f4890009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:13.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.700+0000 7f488effd700 1 -- 172.21.15.2:0/1897210413 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489000c8b0 con 0x7f489c15a030 2023-08-25T03:34:13.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.700+0000 7f488effd700 1 -- 172.21.15.2:0/1897210413 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489000ca10 con 0x7f489c15a030 2023-08-25T03:34:13.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.700+0000 7f48a4642700 1 -- 172.21.15.2:0/1897210413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48980094a0 con 0x7f489c15a030 2023-08-25T03:34:13.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.700+0000 7f488effd700 1 -- 172.21.15.2:0/1897210413 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489000cda0 con 0x7f489c15a030 2023-08-25T03:34:13.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.700+0000 7f48a4642700 1 -- 172.21.15.2:0/1897210413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f489c15c480 con 0x7f489c15a030 2023-08-25T03:34:13.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.701+0000 7f48a4642700 1 -- 172.21.15.2:0/1897210413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f489c066f70 con 0x7f489c15a030 2023-08-25T03:34:13.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.702+0000 7f488effd700 1 -- 172.21.15.2:0/1897210413 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4890013470 con 0x7f489c15a030 2023-08-25T03:34:13.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.702+0000 7f488effd700 1 --2- 172.21.15.2:0/1897210413 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f488006a0e0 0x7f488006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:13.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.702+0000 7f488effd700 1 -- 172.21.15.2:0/1897210413 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4890088720 con 0x7f489c15a030 2023-08-25T03:34:13.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.702+0000 7f48a23de700 1 --2- 172.21.15.2:0/1897210413 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f488006a0e0 0x7f488006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:13.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.703+0000 7f48a23de700 1 --2- 172.21.15.2:0/1897210413 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f488006a0e0 0x7f488006c5a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4898009fd0 tx=0x7f4898008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:13.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.707+0000 7f488effd700 1 -- 172.21.15.2:0/1897210413 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4890053d70 con 0x7f489c15a030 2023-08-25T03:34:13.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.909+0000 7f48a4642700 1 -- 172.21.15.2:0/1897210413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f489c0746d0 con 0x7f489c15a030 2023-08-25T03:34:13.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.910+0000 7f488effd700 1 -- 172.21.15.2:0/1897210413 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18135 (secure 0 0 0) 0x7f4890057390 con 0x7f489c15a030 2023-08-25T03:34:13.911 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:13.911 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":24,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:34:04.604059+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:34:00.869118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T03:34:01.987016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:34:13.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.912+0000 7f48a4642700 1 -- 172.21.15.2:0/1897210413 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f488006a0e0 msgr2=0x7f488006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:13.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.912+0000 7f48a4642700 1 --2- 172.21.15.2:0/1897210413 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f488006a0e0 0x7f488006c5a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4898009fd0 tx=0x7f4898008030 comp rx=0 tx=0).stop 2023-08-25T03:34:13.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.912+0000 7f48a4642700 1 -- 172.21.15.2:0/1897210413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f489c15a030 msgr2=0x7f489c06a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:13.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.912+0000 7f48a4642700 1 --2- 172.21.15.2:0/1897210413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f489c15a030 0x7f489c06a650 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f489000ebe0 tx=0x7f4890009e20 comp rx=0 tx=0).stop 2023-08-25T03:34:13.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.912+0000 7f48a4642700 1 -- 172.21.15.2:0/1897210413 shutdown_connections 2023-08-25T03:34:13.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.912+0000 7f48a4642700 1 --2- 172.21.15.2:0/1897210413 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f489c15a410 0x7f489c15a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:13.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.912+0000 7f48a4642700 1 --2- 172.21.15.2:0/1897210413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f489c0cc1f0 0x7f489c14dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:13.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.912+0000 7f48a4642700 1 --2- 172.21.15.2:0/1897210413 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f488006a0e0 0x7f488006c5a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:13.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.912+0000 7f48a4642700 1 --2- 172.21.15.2:0/1897210413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f489c15a030 0x7f489c06a650 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:13.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.912+0000 7f48a4642700 1 -- 172.21.15.2:0/1897210413 >> 172.21.15.2:0/1897210413 conn(0x7f489c0b1cf0 msgr2=0x7f489c150000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:13.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.912+0000 7f48a4642700 1 -- 172.21.15.2:0/1897210413 shutdown_connections 2023-08-25T03:34:13.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:13.912+0000 7f48a4642700 1 -- 172.21.15.2:0/1897210413 wait complete. 2023-08-25T03:34:13.926 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-08-25T03:34:13.926 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-08-25T03:34:13.927 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-08-25T03:34:13.928 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-08-25T03:34:13.929 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-08-25T03:34:13.930 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-08-25T03:34:13.930 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-08-25T03:34:13.931 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-08-25T03:34:14.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.151+0000 7f17d2ea2700 1 Processor -- start 2023-08-25T03:34:14.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.151+0000 7f17d2ea2700 1 -- start start 2023-08-25T03:34:14.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.152+0000 7fcf61532700 1 Processor -- start 2023-08-25T03:34:14.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.152+0000 7fcf61532700 1 -- start start 2023-08-25T03:34:14.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.152+0000 7f17d2ea2700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f17cc0cad70 0x7f17cc15d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.152+0000 7f17d2ea2700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f17cc15d580 0x7f17cc0ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.152+0000 7f17d2ea2700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17cc15d9c0 con 0x7f17cc1570b0 2023-08-25T03:34:14.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.152+0000 7f17d2ea2700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17cc0717c0 con 0x7f17cc15d580 2023-08-25T03:34:14.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.152+0000 7f17d2ea2700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17cc0cca90 con 0x7f17cc0cad70 2023-08-25T03:34:14.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.153+0000 7f17d143f700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f17cc15d580 0x7f17cc0ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.153+0000 7f17d143f700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f17cc15d580 0x7f17cc0ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59348/0 (socket says 172.21.15.2:59348) 2023-08-25T03:34:14.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.153+0000 7f17d143f700 1 -- 172.21.15.2:0/2471760532 learned_addr learned my addr 172.21.15.2:0/2471760532 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:14.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.153+0000 7f17cbfff700 1 --2- 172.21.15.2:0/2471760532 >> v2:172.21.15.31:3300/0 conn(0x7f17cc0cad70 0x7f17cc15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.153+0000 7f17d143f700 1 -- 172.21.15.2:0/2471760532 >> v2:172.21.15.31:3300/0 conn(0x7f17cc0cad70 msgr2=0x7f17cc15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:14.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.153+0000 7f17d143f700 1 --2- 172.21.15.2:0/2471760532 >> v2:172.21.15.31:3300/0 conn(0x7f17cc0cad70 0x7f17cc15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.153+0000 7f17d143f700 1 -- 172.21.15.2:0/2471760532 >> v1:172.21.15.31:6789/0 conn(0x7f17cc1570b0 legacy=0x7f17cc0702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.153+0000 7f17d143f700 1 -- 172.21.15.2:0/2471760532 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17cc0709e0 con 0x7f17cc15d580 2023-08-25T03:34:14.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.153+0000 7f17d143f700 1 --2- 172.21.15.2:0/2471760532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17cc15d580 0x7f17cc0ccee0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0729f0 tx=0x7f17c400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ebd5934a762fe0a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.153+0000 7fcf61532700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcf5c070460 0x7fcf5c070840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.153+0000 7fcf61532700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fcf5c0cd010 0x7fcf5c0cd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.154+0000 7fcf61532700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf5c0ce6c0 con 0x7fcf5c160f60 2023-08-25T03:34:14.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.154+0000 7fcf61532700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf5c157ec0 con 0x7fcf5c0cd010 2023-08-25T03:34:14.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.154+0000 7fcf61532700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c070d80 con 0x7fcf5c070460 2023-08-25T03:34:14.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.154+0000 7fcf5affd700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcf5c070460 0x7fcf5c070840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.154+0000 7fcf5affd700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcf5c070460 0x7fcf5c070840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.2:59594/0 (socket says 172.21.15.2:59594) 2023-08-25T03:34:14.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.154+0000 7fcf5affd700 1 -- 172.21.15.2:0/1423810276 learned_addr learned my addr 172.21.15.2:0/1423810276 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:14.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.154+0000 7f17cb7fe700 1 -- 172.21.15.2:0/2471760532 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c4010070 con 0x7f17cc15d580 2023-08-25T03:34:14.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.154+0000 7f17cb7fe700 1 -- 172.21.15.2:0/2471760532 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c400dce0 con 0x7f17cc15d580 2023-08-25T03:34:14.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.154+0000 7f17cb7fe700 1 -- 172.21.15.2:0/2471760532 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c400b310 con 0x7f17cc15d580 2023-08-25T03:34:14.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.154+0000 7fe7bc27d700 1 Processor -- start 2023-08-25T03:34:14.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.154+0000 7fcf5affd700 1 -- 172.21.15.2:0/1423810276 >> v2:172.21.15.31:3300/0 conn(0x7fcf5c0cd010 msgr2=0x7fcf5c0cd490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-25T03:34:14.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.154+0000 7fcf5affd700 1 --2- 172.21.15.2:0/1423810276 >> v2:172.21.15.31:3300/0 conn(0x7fcf5c0cd010 0x7fcf5c0cd490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.154+0000 7fcf5affd700 1 -- 172.21.15.2:0/1423810276 >> v1:172.21.15.31:6789/0 conn(0x7fcf5c160f60 legacy=0x7fcf5c1613f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-25T03:34:14.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.154+0000 7fcf5affd700 1 -- 172.21.15.2:0/1423810276 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5c0cdab0 con 0x7fcf5c070460 2023-08-25T03:34:14.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.155+0000 7f17d2ea2700 1 -- 172.21.15.2:0/2471760532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17cc15d580 msgr2=0x7f17cc0ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.155+0000 7f17d2ea2700 1 --2- 172.21.15.2:0/2471760532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17cc15d580 0x7f17cc0ccee0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0729f0 tx=0x7f17c400d430 comp rx=0 tx=0).stop 2023-08-25T03:34:14.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.155+0000 7fcf5affd700 1 --2- 172.21.15.2:0/1423810276 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcf5c070460 0x7fcf5c070840 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcf500098f0 tx=0x7fcf50009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9de3246f5f50043a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.155+0000 7fe7bc27d700 1 -- start start 2023-08-25T03:34:14.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.155+0000 7f17d2ea2700 1 -- 172.21.15.2:0/2471760532 shutdown_connections 2023-08-25T03:34:14.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.155+0000 7f17d2ea2700 1 --2- 172.21.15.2:0/2471760532 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17cc15d580 0x7f17cc0ccee0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.155+0000 7f17d2ea2700 1 --2- 172.21.15.2:0/2471760532 >> v2:172.21.15.31:3300/0 conn(0x7f17cc0cad70 0x7f17cc15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.155+0000 7f17d2ea2700 1 -- 172.21.15.2:0/2471760532 >> 172.21.15.2:0/2471760532 conn(0x7f17cc0ceec0 msgr2=0x7f17cc0cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:14.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.155+0000 7f17d2ea2700 1 -- 172.21.15.2:0/2471760532 shutdown_connections 2023-08-25T03:34:14.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.155+0000 7f17d2ea2700 1 -- 172.21.15.2:0/2471760532 wait complete. 2023-08-25T03:34:14.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.156+0000 7fcf59ffb700 1 -- 172.21.15.2:0/1423810276 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf50004230 con 0x7fcf5c070460 2023-08-25T03:34:14.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.156+0000 7fcf59ffb700 1 -- 172.21.15.2:0/1423810276 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf50004390 con 0x7fcf5c070460 2023-08-25T03:34:14.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.156+0000 7fcf59ffb700 1 -- 172.21.15.2:0/1423810276 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf500036a0 con 0x7fcf5c070460 2023-08-25T03:34:14.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.156+0000 7fe7bc27d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe7b4161680 0x7fe7b4165d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.156+0000 7fe7bc27d700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b41658a0 con 0x7fe7b40c1e90 2023-08-25T03:34:14.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.156+0000 7fe7bc27d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b41625c0 con 0x7fe7b4162170 2023-08-25T03:34:14.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.156+0000 7fe7bc27d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7b4165320 con 0x7fe7b4161680 2023-08-25T03:34:14.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.157+0000 7fe7b9818700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fe7b40c1e90 0x7fe7b4161a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46026/0 (socket says 172.21.15.2:46026) 2023-08-25T03:34:14.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.157+0000 7fe7b9818700 1 -- 172.21.15.2:0/1615569333 learned_addr learned my addr 172.21.15.2:0/1615569333 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:14.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.157+0000 7fe7ba019700 1 --2- 172.21.15.2:0/1615569333 >> v2:172.21.15.31:3300/0 conn(0x7fe7b4161680 0x7fe7b4165d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.157+0000 7fe7b9017700 1 -- 172.21.15.2:0/1615569333 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2685371014 0 0) 0x7fe7b41658a0 con 0x7fe7b40c1e90 2023-08-25T03:34:14.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.157+0000 7fe7b9017700 1 -- 172.21.15.2:0/1615569333 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7a00035b0 con 0x7fe7b40c1e90 2023-08-25T03:34:14.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.157+0000 7fcf61532700 1 -- 172.21.15.2:0/1423810276 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcf5c070460 msgr2=0x7fcf5c070840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.157+0000 7fcf61532700 1 --2- 172.21.15.2:0/1423810276 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcf5c070460 0x7fcf5c070840 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fcf500098f0 tx=0x7fcf50009c00 comp rx=0 tx=0).stop 2023-08-25T03:34:14.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.157+0000 7fcf61532700 1 -- 172.21.15.2:0/1423810276 shutdown_connections 2023-08-25T03:34:14.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.157+0000 7fcf61532700 1 --2- 172.21.15.2:0/1423810276 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcf5c070460 0x7fcf5c070840 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.157+0000 7fcf61532700 1 --2- 172.21.15.2:0/1423810276 >> v2:172.21.15.31:3300/0 conn(0x7fcf5c0cd010 0x7fcf5c0cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.157+0000 7fcf61532700 1 -- 172.21.15.2:0/1423810276 >> 172.21.15.2:0/1423810276 conn(0x7fcf5c0ceec0 msgr2=0x7fcf5c0cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:14.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.157+0000 7fcf61532700 1 -- 172.21.15.2:0/1423810276 shutdown_connections 2023-08-25T03:34:14.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.157+0000 7fcf61532700 1 -- 172.21.15.2:0/1423810276 wait complete. 2023-08-25T03:34:14.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.158+0000 7f17d2ea2700 1 Processor -- start 2023-08-25T03:34:14.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.158+0000 7f17d2ea2700 1 -- start start 2023-08-25T03:34:14.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.158+0000 7fe7b9017700 1 -- 172.21.15.2:0/1615569333 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2641238241 0 0) 0x7fe7a00035b0 con 0x7fe7b40c1e90 2023-08-25T03:34:14.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.158+0000 7fe7b9017700 1 -- 172.21.15.2:0/1615569333 >> v2:172.21.15.31:3300/0 conn(0x7fe7b4161680 msgr2=0x7fe7b4165d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:14.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.158+0000 7fe7b9017700 1 --2- 172.21.15.2:0/1615569333 >> v2:172.21.15.31:3300/0 conn(0x7fe7b4161680 0x7fe7b4165d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.158+0000 7fe7b9017700 1 -- 172.21.15.2:0/1615569333 >> v1:172.21.15.31:6789/0 conn(0x7fe7b4162170 legacy=0x7fe7b40abd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:14.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.158+0000 7fe7b9017700 1 -- 172.21.15.2:0/1615569333 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b4166270 con 0x7fe7b40c1e90 2023-08-25T03:34:14.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.158+0000 7fe7b9017700 1 -- 172.21.15.2:0/1615569333 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7fe7b0001670 con 0x7fe7b40c1e90 2023-08-25T03:34:14.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.158+0000 7fe7b9017700 1 -- 172.21.15.2:0/1615569333 >> v1:172.21.15.2:6789/0 conn(0x7fe7b40c1e90 legacy=0x7fe7b4161a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.158+0000 7fe7b9017700 1 --2- 172.21.15.2:0/1615569333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7a0003a50 0x7fe7a0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.158+0000 7fe7b9017700 1 --2- 172.21.15.2:0/1615569333 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7a0004850 0x7fe7a0004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.158+0000 7fe7b9017700 1 --2- 172.21.15.2:0/1615569333 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7a0005260 0x7fe7a000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.158+0000 7fe7b9017700 1 -- 172.21.15.2:0/1615569333 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b4165320 con 0x7fe7a0004850 2023-08-25T03:34:14.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.158+0000 7fe7b9017700 1 -- 172.21.15.2:0/1615569333 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a000c190 con 0x7fe7a0003a50 2023-08-25T03:34:14.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.158+0000 7fe7b9017700 1 -- 172.21.15.2:0/1615569333 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a000c350 con 0x7fe7a0005260 2023-08-25T03:34:14.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.158+0000 7fe7b9818700 1 --2- 172.21.15.2:0/1615569333 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7a0004850 0x7fe7a0004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.159+0000 7f67e5b8d700 1 Processor -- start 2023-08-25T03:34:14.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.159+0000 7f64d66fd700 1 Processor -- start 2023-08-25T03:34:14.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.159+0000 7fe7b9818700 1 -- 172.21.15.2:0/1615569333 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7a0005260 msgr2=0x7fe7a000ba10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-25T03:34:14.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.159+0000 7fe7b9818700 1 --2- 172.21.15.2:0/1615569333 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7a0005260 0x7fe7a000ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.159+0000 7fe7b9818700 1 -- 172.21.15.2:0/1615569333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7a0003a50 msgr2=0x7fe7a0003e50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-25T03:34:14.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.159+0000 7fe7b9818700 1 --2- 172.21.15.2:0/1615569333 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7a0003a50 0x7fe7a0003e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.159+0000 7fe7b9818700 1 -- 172.21.15.2:0/1615569333 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a00035b0 con 0x7fe7a0004850 2023-08-25T03:34:14.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.159+0000 7fe7b9818700 1 --2- 172.21.15.2:0/1615569333 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7a0004850 0x7fe7a0004cd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000c190 tx=0x7fe7a0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be47340a70393ae7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.159+0000 7fcf61532700 1 Processor -- start 2023-08-25T03:34:14.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.160+0000 7fcf61532700 1 -- start start 2023-08-25T03:34:14.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.159+0000 7f17d2ea2700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f17cc0cad70 0x7f17cc158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.159+0000 7f17d2ea2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17cc1570b0 0x7f17cc159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.159+0000 7f17d2ea2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17cc15d580 0x7f17cc0c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.159+0000 7f17d2ea2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc159860 con 0x7f17cc15d580 2023-08-25T03:34:14.182 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.159+0000 7f17d2ea2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc09c0a0 con 0x7f17cc1570b0 2023-08-25T03:34:14.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.159+0000 7f17d2ea2700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc09c380 con 0x7f17cc0cad70 2023-08-25T03:34:14.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.160+0000 7fe7b9017700 1 -- 172.21.15.2:0/1615569333 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b000cd30 con 0x7fe7a0004850 2023-08-25T03:34:14.183 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.159+0000 7f67e5b8d700 1 -- start start 2023-08-25T03:34:14.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.160+0000 7f67e5b8d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f67e015d390 0x7f67e00b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.160+0000 7f67e5b8d700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f67e00ce4f0 0x7f67e00d1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.160+0000 7f67e5b8d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e00d1410 con 0x7f67e0160e00 2023-08-25T03:34:14.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.160+0000 7f67e5b8d700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67e015d770 con 0x7f67e00ce4f0 2023-08-25T03:34:14.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.160+0000 7f67e5b8d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67e01615c0 con 0x7f67e015d390 2023-08-25T03:34:14.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.160+0000 7f67deffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f67e0160e00 0x7f67e00cdde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.2:54060/0 (socket says 172.21.15.2:54060) 2023-08-25T03:34:14.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.160+0000 7f67deffd700 1 -- 172.21.15.2:0/4218561424 learned_addr learned my addr 172.21.15.2:0/4218561424 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:14.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.159+0000 7f64d66fd700 1 -- start start 2023-08-25T03:34:14.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.160+0000 7f64d66fd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f64d015d750 0x7f64d01643d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.160+0000 7f64d66fd700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d00709e0 con 0x7f64d01570b0 2023-08-25T03:34:14.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.160+0000 7f64d66fd700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d0157490 con 0x7f64d00cad70 2023-08-25T03:34:14.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.160+0000 7f64d66fd700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d00717c0 con 0x7f64d015d750 2023-08-25T03:34:14.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.160+0000 7f17d0c3e700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f17cc0cad70 0x7f17cc158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.160+0000 7f17cbfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17cc1570b0 0x7f17cc159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.160+0000 7f17d0c3e700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f17cc0cad70 0x7f17cc158de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.2:59608/0 (socket says 172.21.15.2:59608) 2023-08-25T03:34:14.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.160+0000 7f17d0c3e700 1 -- 172.21.15.2:0/1796900296 learned_addr learned my addr 172.21.15.2:0/1796900296 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:14.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7f64cf7fe700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f64d00cad70 0x7f64d015d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.2:54068/0 (socket says 172.21.15.2:54068) 2023-08-25T03:34:14.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7f64cf7fe700 1 -- 172.21.15.2:0/4054957463 learned_addr learned my addr 172.21.15.2:0/4054957463 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:14.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.160+0000 7f67dffff700 1 --2- 172.21.15.2:0/4218561424 >> v2:172.21.15.2:3300/0 conn(0x7f67e00ce4f0 0x7f67e00d1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7f67df7fe700 1 --2- 172.21.15.2:0/4218561424 >> v2:172.21.15.31:3300/0 conn(0x7f67e015d390 0x7f67e00b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7f17d143f700 1 --2- 172.21.15.2:0/1796900296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17cc15d580 0x7f17cc0c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.188 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7f64f4284700 1 Processor -- start 2023-08-25T03:34:14.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7fcf61532700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcf5c070460 0x7fcf5c155630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7fcf61532700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf5c0cd010 0x7fcf5c0c3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7fcf61532700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf5c160f60 0x7fcf5c0acf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7fcf61532700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c0ae990 con 0x7fcf5c0cd010 2023-08-25T03:34:14.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7fcf61532700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c155b70 con 0x7fcf5c160f60 2023-08-25T03:34:14.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7fcf61532700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c1f9860 con 0x7fcf5c070460 2023-08-25T03:34:14.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7fcf5affd700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcf5c070460 0x7fcf5c155630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.160+0000 7fe7b9017700 1 -- 172.21.15.2:0/1615569333 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b0005960 con 0x7fe7a0004850 2023-08-25T03:34:14.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.160+0000 7fe7b9017700 1 -- 172.21.15.2:0/1615569333 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b0005d10 con 0x7fe7a0004850 2023-08-25T03:34:14.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.160+0000 7fe7ba81a700 1 -- 172.21.15.2:0/1615569333 reap_dead start 2023-08-25T03:34:14.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.160+0000 7fe7bc27d700 1 -- 172.21.15.2:0/1615569333 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7a0004850 msgr2=0x7fe7a0004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.160+0000 7fe7bc27d700 1 --2- 172.21.15.2:0/1615569333 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7a0004850 0x7fe7a0004cd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe7b000c190 tx=0x7fe7a0005880 comp rx=0 tx=0).stop 2023-08-25T03:34:14.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7fe7bc27d700 1 -- 172.21.15.2:0/1615569333 shutdown_connections 2023-08-25T03:34:14.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7fe7bc27d700 1 --2- 172.21.15.2:0/1615569333 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7a0004850 0x7fe7a0004cd0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7fe7bc27d700 1 -- 172.21.15.2:0/1615569333 >> 172.21.15.2:0/1615569333 conn(0x7fe7b40ceec0 msgr2=0x7fe7b40cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:14.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7fcf5b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf5c160f60 0x7fcf5c0acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7fcf5a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf5c0cd010 0x7fcf5c0c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7fcf5b7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf5c160f60 0x7fcf5c0acf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.2:50674/0 (socket says 172.21.15.2:50674) 2023-08-25T03:34:14.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7fcf5b7fe700 1 -- 172.21.15.2:0/2211737749 learned_addr learned my addr 172.21.15.2:0/2211737749 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:14.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7fe7bc27d700 1 -- 172.21.15.2:0/1615569333 shutdown_connections 2023-08-25T03:34:14.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7f64ceffd700 1 -- 172.21.15.2:0/4054957463 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3897675602 0 0) 0x7f64d0157490 con 0x7f64d00cad70 2023-08-25T03:34:14.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7f64ceffd700 1 -- 172.21.15.2:0/4054957463 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64b80035b0 con 0x7f64d00cad70 2023-08-25T03:34:14.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7fe7bc27d700 1 -- 172.21.15.2:0/1615569333 wait complete. 2023-08-25T03:34:14.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7f67dffff700 1 -- 172.21.15.2:0/4218561424 >> v2:172.21.15.31:3300/0 conn(0x7f67e015d390 msgr2=0x7f67e00b8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:14.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7f67dffff700 1 --2- 172.21.15.2:0/4218561424 >> v2:172.21.15.31:3300/0 conn(0x7f67e015d390 0x7f67e00b8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7f67dffff700 1 -- 172.21.15.2:0/4218561424 >> v1:172.21.15.31:6789/0 conn(0x7f67e0160e00 legacy=0x7f67e00cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7f67dffff700 1 -- 172.21.15.2:0/4218561424 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e01618a0 con 0x7f67e00ce4f0 2023-08-25T03:34:14.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7f64ceffd700 1 -- 172.21.15.2:0/4054957463 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 288669503 0 0) 0x7f64d00709e0 con 0x7f64d01570b0 2023-08-25T03:34:14.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7f64ceffd700 1 -- 172.21.15.2:0/4054957463 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64d0157490 con 0x7f64d01570b0 2023-08-25T03:34:14.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7f64ceffd700 1 -- 172.21.15.2:0/4054957463 <== mon.1 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1039881043 0 0) 0x7f64b80035b0 con 0x7f64d00cad70 2023-08-25T03:34:14.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7f64ceffd700 1 -- 172.21.15.2:0/4054957463 >> v2:172.21.15.2:3300/0 conn(0x7f64d015d750 msgr2=0x7f64d01643d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:14.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7f64ceffd700 1 --2- 172.21.15.2:0/4054957463 >> v2:172.21.15.2:3300/0 conn(0x7f64d015d750 0x7f64d01643d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7f64ceffd700 1 -- 172.21.15.2:0/4054957463 >> v1:172.21.15.2:6789/0 conn(0x7f64d01570b0 legacy=0x7f64d00702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7f64ceffd700 1 -- 172.21.15.2:0/4054957463 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d0070c40 con 0x7f64d00cad70 2023-08-25T03:34:14.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7f64ceffd700 1 -- 172.21.15.2:0/4054957463 <== mon.1 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f64c8001a30 con 0x7f64d00cad70 2023-08-25T03:34:14.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7f64ceffd700 1 -- 172.21.15.2:0/4054957463 >> v1:172.21.15.31:6789/0 conn(0x7f64d00cad70 legacy=0x7f64d015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7f64ceffd700 1 --2- 172.21.15.2:0/4054957463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64b8003be0 0x7f64b8003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7f64ceffd700 1 --2- 172.21.15.2:0/4054957463 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64b8004850 0x7f64b8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7f64ceffd700 1 --2- 172.21.15.2:0/4054957463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64b8005330 0x7f64b80059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7f64ceffd700 1 -- 172.21.15.2:0/4054957463 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b8006290 con 0x7f64b8003be0 2023-08-25T03:34:14.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7f64ceffd700 1 -- 172.21.15.2:0/4054957463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64b8006400 con 0x7f64b8005330 2023-08-25T03:34:14.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7f64ceffd700 1 -- 172.21.15.2:0/4054957463 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64b80065c0 con 0x7f64b8004850 2023-08-25T03:34:14.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7f64d4c9a700 1 --2- 172.21.15.2:0/4054957463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64b8005330 0x7f64b80059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7f64cf7fe700 1 --2- 172.21.15.2:0/4054957463 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64b8004850 0x7f64b8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7fcf5affd700 1 -- 172.21.15.2:0/2211737749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf5c160f60 msgr2=0x7fcf5c0acf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7fcf5affd700 1 --2- 172.21.15.2:0/2211737749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf5c160f60 0x7fcf5c0acf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7fcf5affd700 1 -- 172.21.15.2:0/2211737749 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf5c0cd010 msgr2=0x7fcf5c0c3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7fcf5affd700 1 --2- 172.21.15.2:0/2211737749 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcf5c0cd010 0x7fcf5c0c3280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7fcf5affd700 1 -- 172.21.15.2:0/2211737749 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf50009550 con 0x7fcf5c070460 2023-08-25T03:34:14.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7f17d143f700 1 -- 172.21.15.2:0/1796900296 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f17cc0cad70 msgr2=0x7f17cc158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7f17d143f700 1 --2- 172.21.15.2:0/1796900296 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f17cc0cad70 0x7f17cc158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7f17d143f700 1 -- 172.21.15.2:0/1796900296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17cc1570b0 msgr2=0x7f17cc159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7f17d143f700 1 --2- 172.21.15.2:0/1796900296 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17cc1570b0 0x7f17cc159320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7f17d143f700 1 -- 172.21.15.2:0/1796900296 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17c400bd40 con 0x7f17cc15d580 2023-08-25T03:34:14.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7f17d143f700 1 --2- 172.21.15.2:0/1796900296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17cc15d580 0x7f17cc0c7040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f17c4017000 tx=0x7f17c4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7fcf5affd700 1 --2- 172.21.15.2:0/2211737749 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcf5c070460 0x7fcf5c155630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0c94a0 tx=0x7fcf50004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7f64d4c9a700 1 -- 172.21.15.2:0/4054957463 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64b8004850 msgr2=0x7f64b8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:14.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7f64d4c9a700 1 --2- 172.21.15.2:0/4054957463 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64b8004850 0x7f64b8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7f64d4c9a700 1 -- 172.21.15.2:0/4054957463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64b8003be0 msgr2=0x7f64b8003fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:14.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7f64d4c9a700 1 --2- 172.21.15.2:0/4054957463 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64b8003be0 0x7f64b8003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7f17c97fa700 1 -- 172.21.15.2:0/1796900296 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c400de50 con 0x7f17cc15d580 2023-08-25T03:34:14.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7f64d4c9a700 1 -- 172.21.15.2:0/4054957463 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b80035b0 con 0x7f64b8005330 2023-08-25T03:34:14.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7f64d4c9a700 1 -- 172.21.15.2:0/4054957463 reap_dead start 2023-08-25T03:34:14.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7f17d2ea2700 1 -- 172.21.15.2:0/1796900296 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17cc0c6840 con 0x7f17cc15d580 2023-08-25T03:34:14.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7fcf437fe700 1 -- 172.21.15.2:0/2211737749 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf50003e90 con 0x7fcf5c070460 2023-08-25T03:34:14.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.162+0000 7f17d2ea2700 1 -- 172.21.15.2:0/1796900296 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17cc14e900 con 0x7f17cc15d580 2023-08-25T03:34:14.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.163+0000 7fcf437fe700 1 -- 172.21.15.2:0/2211737749 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5000b660 con 0x7fcf5c070460 2023-08-25T03:34:14.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.163+0000 7fcf437fe700 1 -- 172.21.15.2:0/2211737749 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5000b9f0 con 0x7fcf5c070460 2023-08-25T03:34:14.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.163+0000 7fcf61532700 1 -- 172.21.15.2:0/2211737749 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf5c1f9b40 con 0x7fcf5c070460 2023-08-25T03:34:14.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.163+0000 7fcf61532700 1 -- 172.21.15.2:0/2211737749 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5c1fa100 con 0x7fcf5c070460 2023-08-25T03:34:14.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.163+0000 7f17c97fa700 1 -- 172.21.15.2:0/1796900296 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c4009810 con 0x7f17cc15d580 2023-08-25T03:34:14.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.163+0000 7f17c97fa700 1 -- 172.21.15.2:0/1796900296 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c4009ba0 con 0x7f17cc15d580 2023-08-25T03:34:14.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.163+0000 7f64d4c9a700 1 --2- 172.21.15.2:0/4054957463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64b8005330 0x7f64b80059e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f64c4009910 tx=0x7f64b8005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a403527d1c3e6ad server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.163+0000 7f67dffff700 1 --2- 172.21.15.2:0/4218561424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67e00ce4f0 0x7f67e00d1b80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f67e0068780 tx=0x7f67d0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8326736f16317d28 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.161+0000 7f64f4284700 1 -- start start 2023-08-25T03:34:14.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.163+0000 7f64f4284700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f64ec161680 0x7f64ec165d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.163+0000 7f64f4284700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64ec0c1e90 0x7f64ec0c22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.163+0000 7f64f4284700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ec0c2830 con 0x7f64ec161a60 2023-08-25T03:34:14.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.163+0000 7fe7bc27d700 1 Processor -- start 2023-08-25T03:34:14.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.163+0000 7f64f4284700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ec165320 con 0x7f64ec161680 2023-08-25T03:34:14.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.163+0000 7f64f4284700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec1658a0 con 0x7f64ec0c1e90 2023-08-25T03:34:14.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.163+0000 7f64f181f700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64ec0c1e90 0x7f64ec0c22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.164+0000 7f64f181f700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64ec0c1e90 0x7f64ec0c22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.2:59642/0 (socket says 172.21.15.2:59642) 2023-08-25T03:34:14.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.164+0000 7f64f181f700 1 -- 172.21.15.2:0/4096652705 learned_addr learned my addr 172.21.15.2:0/4096652705 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:14.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.164+0000 7f64f2020700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f64ec161680 0x7f64ec165d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.163+0000 7f64ceffd700 1 -- 172.21.15.2:0/4054957463 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4004220 con 0x7f64b8005330 2023-08-25T03:34:14.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.163+0000 7f64ceffd700 1 -- 172.21.15.2:0/4054957463 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c4004380 con 0x7f64b8005330 2023-08-25T03:34:14.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.163+0000 7f64ceffd700 1 -- 172.21.15.2:0/4054957463 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c40036a0 con 0x7f64b8005330 2023-08-25T03:34:14.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.164+0000 7fe7bc27d700 1 -- start start 2023-08-25T03:34:14.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.164+0000 7f64d66fd700 1 -- 172.21.15.2:0/4054957463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64b8005330 msgr2=0x7f64b80059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.164+0000 7f64d66fd700 1 --2- 172.21.15.2:0/4054957463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64b8005330 0x7f64b80059e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f64c4009910 tx=0x7f64b8005880 comp rx=0 tx=0).stop 2023-08-25T03:34:14.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.164+0000 7fcf61532700 1 -- 172.21.15.2:0/2211737749 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcf5c0ce070 con 0x7fcf5c070460 2023-08-25T03:34:14.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.164+0000 7f64d66fd700 1 -- 172.21.15.2:0/4054957463 shutdown_connections 2023-08-25T03:34:14.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.164+0000 7f64d66fd700 1 --2- 172.21.15.2:0/4054957463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64b8005330 0x7f64b80059e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.164+0000 7f64d66fd700 1 -- 172.21.15.2:0/4054957463 >> 172.21.15.2:0/4054957463 conn(0x7f64d00ceec0 msgr2=0x7f64d00cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:14.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.165+0000 7f64d66fd700 1 -- 172.21.15.2:0/4054957463 shutdown_connections 2023-08-25T03:34:14.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.165+0000 7f67de7fc700 1 -- 172.21.15.2:0/4218561424 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d0004230 con 0x7f67e00ce4f0 2023-08-25T03:34:14.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.165+0000 7f67de7fc700 1 -- 172.21.15.2:0/4218561424 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d0004390 con 0x7f67e00ce4f0 2023-08-25T03:34:14.212 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.165+0000 7f67de7fc700 1 -- 172.21.15.2:0/4218561424 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d00036a0 con 0x7f67e00ce4f0 2023-08-25T03:34:14.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.165+0000 7f64d66fd700 1 -- 172.21.15.2:0/4054957463 wait complete. 2023-08-25T03:34:14.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.165+0000 7f17c97fa700 1 -- 172.21.15.2:0/1796900296 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f17c4009d00 con 0x7f17cc15d580 2023-08-25T03:34:14.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.165+0000 7f17c97fa700 1 --2- 172.21.15.2:0/1796900296 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f17b006a0e0 0x7f17b006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.213 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.166+0000 7f17c97fa700 1 -- 172.21.15.2:0/1796900296 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f17c4089500 con 0x7f17cc15d580 2023-08-25T03:34:14.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.165+0000 7fcf437fe700 1 -- 172.21.15.2:0/2211737749 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcf50022070 con 0x7fcf5c070460 2023-08-25T03:34:14.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.165+0000 7fcf437fe700 1 --2- 172.21.15.2:0/2211737749 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fcf4806a1b0 0x7fcf4806c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.166+0000 7fcf437fe700 1 -- 172.21.15.2:0/2211737749 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcf5008a060 con 0x7fcf5c070460 2023-08-25T03:34:14.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.166+0000 7fcf5a7fc700 1 --2- 172.21.15.2:0/2211737749 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fcf4806a1b0 0x7fcf4806c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.166+0000 7fcf437fe700 1 --2- 172.21.15.2:0/2211737749 >> [v2:172.21.15.31:6800/1678889094,v1:172.21.15.31:6801/1678889094] conn(0x7fcf48072880 0x7fcf48074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.166+0000 7fcf437fe700 1 -- 172.21.15.2:0/2211737749 --> [v2:172.21.15.31:6800/1678889094,v1:172.21.15.31:6801/1678889094] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf48075350 con 0x7fcf48072880 2023-08-25T03:34:14.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.166+0000 7f67e5b8d700 1 -- 172.21.15.2:0/4218561424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67e00ce4f0 msgr2=0x7f67e00d1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.166+0000 7f67e5b8d700 1 --2- 172.21.15.2:0/4218561424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67e00ce4f0 0x7f67e00d1b80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f67e0068780 tx=0x7f67d0009b20 comp rx=0 tx=0).stop 2023-08-25T03:34:14.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.166+0000 7fcf5b7fe700 1 --2- 172.21.15.2:0/2211737749 >> [v2:172.21.15.31:6800/1678889094,v1:172.21.15.31:6801/1678889094] conn(0x7fcf48072880 0x7fcf48074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.165+0000 7fe7bc27d700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7a0004850 0x7fe7b41fa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.165+0000 7fe7bc27d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7b41f86f0 0x7fe7b406bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.165+0000 7fe7bc27d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7b41f8b70 0x7fe7b406c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.165+0000 7fe7bc27d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b40b4a40 con 0x7fe7b41f8b70 2023-08-25T03:34:14.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.165+0000 7fe7bc27d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b40af9f0 con 0x7fe7b41f86f0 2023-08-25T03:34:14.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.165+0000 7fe7bc27d700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b40afcd0 con 0x7fe7a0004850 2023-08-25T03:34:14.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.166+0000 7fcf437fe700 1 -- 172.21.15.2:0/2211737749 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf500557c0 con 0x7fcf5c070460 2023-08-25T03:34:14.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.166+0000 7fcf5a7fc700 1 --2- 172.21.15.2:0/2211737749 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fcf4806a1b0 0x7fcf4806c670 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0ae160 tx=0x7fcf4c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.164+0000 7f64f2020700 1 -- 172.21.15.2:0/4096652705 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64ec0c1e90 msgr2=0x7f64ec0c22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:14.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.164+0000 7f64f2020700 1 --2- 172.21.15.2:0/4096652705 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64ec0c1e90 0x7f64ec0c22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.164+0000 7f64f2020700 1 -- 172.21.15.2:0/4096652705 >> v1:172.21.15.31:6789/0 conn(0x7f64ec161a60 legacy=0x7f64ec161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:14.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.164+0000 7f64f2020700 1 -- 172.21.15.2:0/4096652705 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ec1625c0 con 0x7f64ec161680 2023-08-25T03:34:14.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.164+0000 7f64f2020700 1 --2- 172.21.15.2:0/4096652705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64ec161680 0x7f64ec165d30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f64e8009840 tx=0x7f64e8009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e94533b6d77b17aa server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.165+0000 7f64f101e700 1 -- 172.21.15.2:0/4096652705 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e8004230 con 0x7f64ec161680 2023-08-25T03:34:14.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.165+0000 7f64f101e700 1 -- 172.21.15.2:0/4096652705 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e8004390 con 0x7f64ec161680 2023-08-25T03:34:14.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.165+0000 7f64f101e700 1 -- 172.21.15.2:0/4096652705 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e80036a0 con 0x7f64ec161680 2023-08-25T03:34:14.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.166+0000 7f64f4284700 1 -- 172.21.15.2:0/4096652705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64ec161680 msgr2=0x7f64ec165d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.166+0000 7f64f4284700 1 --2- 172.21.15.2:0/4096652705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64ec161680 0x7f64ec165d30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f64e8009840 tx=0x7f64e8009b50 comp rx=0 tx=0).stop 2023-08-25T03:34:14.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.166+0000 7fe7ba019700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7a0004850 0x7fe7b41fa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.166+0000 7fe7ba019700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7a0004850 0x7fe7b41fa4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.2:59648/0 (socket says 172.21.15.2:59648) 2023-08-25T03:34:14.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.166+0000 7fe7ba019700 1 -- 172.21.15.2:0/4034411578 learned_addr learned my addr 172.21.15.2:0/4034411578 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:14.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.166+0000 7f64f4284700 1 -- 172.21.15.2:0/4096652705 shutdown_connections 2023-08-25T03:34:14.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.166+0000 7f64f4284700 1 --2- 172.21.15.2:0/4096652705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64ec161680 0x7f64ec165d30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.166+0000 7f64f4284700 1 --2- 172.21.15.2:0/4096652705 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64ec0c1e90 0x7f64ec0c22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.166+0000 7f64f4284700 1 -- 172.21.15.2:0/4096652705 >> 172.21.15.2:0/4096652705 conn(0x7f64ec0ceec0 msgr2=0x7f64ec0cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:14.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.166+0000 7fe7ba81a700 1 --2- 172.21.15.2:0/4034411578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7b41f8b70 0x7fe7b406c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.166+0000 7f64d66fd700 1 Processor -- start 2023-08-25T03:34:14.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.167+0000 7f64f4284700 1 -- 172.21.15.2:0/4096652705 shutdown_connections 2023-08-25T03:34:14.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.167+0000 7f67e5b8d700 1 -- 172.21.15.2:0/4218561424 shutdown_connections 2023-08-25T03:34:14.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.167+0000 7f67e5b8d700 1 --2- 172.21.15.2:0/4218561424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67e00ce4f0 0x7f67e00d1b80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.167+0000 7f67e5b8d700 1 --2- 172.21.15.2:0/4218561424 >> v2:172.21.15.31:3300/0 conn(0x7f67e015d390 0x7f67e00b8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.167+0000 7f67e5b8d700 1 -- 172.21.15.2:0/4218561424 >> 172.21.15.2:0/4218561424 conn(0x7f67e006abe0 msgr2=0x7f67e006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:14.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.167+0000 7f17d2ea2700 1 --2- 172.21.15.2:0/1796900296 >> [v2:172.21.15.31:6808/1680871561,v1:172.21.15.31:6809/1680871561] conn(0x7f17ac0015e0 0x7f17ac003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.167+0000 7f17d2ea2700 1 -- 172.21.15.2:0/1796900296 --> [v2:172.21.15.31:6808/1680871561,v1:172.21.15.31:6809/1680871561] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f17ac006bd0 con 0x7f17ac0015e0 2023-08-25T03:34:14.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.166+0000 7f3a9cc44700 1 Processor -- start 2023-08-25T03:34:14.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.167+0000 7f3a9cc44700 1 -- start start 2023-08-25T03:34:14.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.167+0000 7fcf5b7fe700 1 --2- 172.21.15.2:0/2211737749 >> [v2:172.21.15.31:6800/1678889094,v1:172.21.15.31:6801/1678889094] conn(0x7fcf48072880 0x7fcf48074ca0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.167+0000 7f64d66fd700 1 -- start start 2023-08-25T03:34:14.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.168+0000 7f3a9cc44700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3a980b5930 0x7f3a980d1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.168+0000 7f3a9cc44700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a980d2640 con 0x7f3a980d2230 2023-08-25T03:34:14.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.168+0000 7f3a9cc44700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a980b82f0 con 0x7f3a981619d0 2023-08-25T03:34:14.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.168+0000 7f3a9cc44700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a980cd220 con 0x7f3a980b5930 2023-08-25T03:34:14.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.168+0000 7f64d66fd700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64b8005330 0x7f64d0154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.168+0000 7f64d66fd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64d0157620 0x7f64d00c5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.168+0000 7f64d66fd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64d006df10 0x7f64d006e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.168+0000 7f64d66fd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d0158730 con 0x7f64d0157620 2023-08-25T03:34:14.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.168+0000 7f64d66fd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d006e8e0 con 0x7f64d006df10 2023-08-25T03:34:14.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.168+0000 7f64d66fd700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d0157ab0 con 0x7f64b8005330 2023-08-25T03:34:14.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.168+0000 7f3a95d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3a980b5930 0x7f3a980d1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.168+0000 7f3a95d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3a980b5930 0x7f3a980d1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.2:50706/0 (socket says 172.21.15.2:50706) 2023-08-25T03:34:14.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.168+0000 7f3a95d9b700 1 -- 172.21.15.2:0/937304705 learned_addr learned my addr 172.21.15.2:0/937304705 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:14.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.167+0000 7f64f4284700 1 -- 172.21.15.2:0/4096652705 wait complete. 2023-08-25T03:34:14.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.168+0000 7f64f4284700 1 Processor -- start 2023-08-25T03:34:14.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.167+0000 7fe7b9818700 1 --2- 172.21.15.2:0/4034411578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7b41f86f0 0x7fe7b406bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.167+0000 7fe7ba81a700 1 -- 172.21.15.2:0/4034411578 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7a0004850 msgr2=0x7fe7b41fa4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.167+0000 7fe7ba81a700 1 --2- 172.21.15.2:0/4034411578 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7a0004850 0x7fe7b41fa4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.167+0000 7fe7ba81a700 1 -- 172.21.15.2:0/4034411578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7b41f86f0 msgr2=0x7fe7b406bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.167+0000 7fe7ba81a700 1 --2- 172.21.15.2:0/4034411578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7b41f86f0 0x7fe7b406bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.167+0000 7fe7ba81a700 1 -- 172.21.15.2:0/4034411578 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b40affb0 con 0x7fe7b41f8b70 2023-08-25T03:34:14.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.167+0000 7fe7ba81a700 1 --2- 172.21.15.2:0/4034411578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7b41f8b70 0x7fe7b406c010 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40717c0 tx=0x7fe7ac00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.167+0000 7fe7aaffd700 1 -- 172.21.15.2:0/4034411578 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac007110 con 0x7fe7b41f8b70 2023-08-25T03:34:14.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.167+0000 7fe7aaffd700 1 -- 172.21.15.2:0/4034411578 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7ac007270 con 0x7fe7b41f8b70 2023-08-25T03:34:14.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.167+0000 7fe7aaffd700 1 -- 172.21.15.2:0/4034411578 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac007620 con 0x7fe7b41f8b70 2023-08-25T03:34:14.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.167+0000 7fe7bc27d700 1 -- 172.21.15.2:0/4034411578 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b40b0290 con 0x7fe7b41f8b70 2023-08-25T03:34:14.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.168+0000 7fe7bc27d700 1 -- 172.21.15.2:0/4034411578 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b40b0990 con 0x7fe7b41f8b70 2023-08-25T03:34:14.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.169+0000 7fe7bc27d700 1 -- 172.21.15.2:0/4034411578 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe7b4162b20 con 0x7fe7b41f8b70 2023-08-25T03:34:14.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.168+0000 7fcf437fe700 1 -- 172.21.15.2:0/2211737749 <== osd.5 v2:172.21.15.31:6800/1678889094 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fcf48075350 con 0x7fcf48072880 2023-08-25T03:34:14.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.168+0000 7f17cbfff700 1 --2- 172.21.15.2:0/1796900296 >> [v2:172.21.15.31:6808/1680871561,v1:172.21.15.31:6809/1680871561] conn(0x7f17ac0015e0 0x7f17ac003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.168+0000 7f17d0c3e700 1 --2- 172.21.15.2:0/1796900296 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f17b006a0e0 0x7f17b006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.169+0000 7f3a95d9b700 1 -- 172.21.15.2:0/937304705 >> v1:172.21.15.31:6789/0 conn(0x7f3a981619d0 legacy=0x7f3a980ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.169+0000 7f3a95d9b700 1 -- 172.21.15.2:0/937304705 >> v1:172.21.15.2:6789/0 conn(0x7f3a980d2230 legacy=0x7f3a980b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:14.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.169+0000 7f3a95d9b700 1 -- 172.21.15.2:0/937304705 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a980b6e40 con 0x7f3a980b5930 2023-08-25T03:34:14.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.169+0000 7f64cf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64d0157620 0x7f64d00c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.169+0000 7f64cf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64d0157620 0x7f64d00c5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59420/0 (socket says 172.21.15.2:59420) 2023-08-25T03:34:14.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.169+0000 7f64cf7fe700 1 -- 172.21.15.2:0/2732516070 learned_addr learned my addr 172.21.15.2:0/2732516070 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:14.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.169+0000 7f17d0c3e700 1 --2- 172.21.15.2:0/1796900296 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f17b006a0e0 0x7f17b006c5a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f17c0001920 tx=0x7f17c0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.169+0000 7f64cffff700 1 --2- 172.21.15.2:0/2732516070 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64b8005330 0x7f64d0154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.169+0000 7f17cbfff700 1 --2- 172.21.15.2:0/1796900296 >> [v2:172.21.15.31:6808/1680871561,v1:172.21.15.31:6809/1680871561] conn(0x7f17ac0015e0 0x7f17ac003aa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.169+0000 7f67e5b8d700 1 -- 172.21.15.2:0/4218561424 shutdown_connections 2023-08-25T03:34:14.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.169+0000 7f64cf7fe700 1 -- 172.21.15.2:0/2732516070 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64b8005330 msgr2=0x7f64d0154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.169+0000 7f64cf7fe700 1 --2- 172.21.15.2:0/2732516070 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64b8005330 0x7f64d0154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.169+0000 7f64cf7fe700 1 -- 172.21.15.2:0/2732516070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64d006df10 msgr2=0x7f64d006e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.169+0000 7f64cf7fe700 1 --2- 172.21.15.2:0/2732516070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64d006df10 0x7f64d006e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.169+0000 7f64cf7fe700 1 -- 172.21.15.2:0/2732516070 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d0157d90 con 0x7f64d0157620 2023-08-25T03:34:14.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.170+0000 7f64cf7fe700 1 --2- 172.21.15.2:0/2732516070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64d0157620 0x7f64d00c5250 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f64c800ff40 tx=0x7f64c800ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.169+0000 7f64f4284700 1 -- start start 2023-08-25T03:34:14.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.170+0000 7f64f4284700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64ec0c1e90 0x7f64ec0ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.170+0000 7f64f4284700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64ec161680 0x7f64ec06bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.170+0000 7f64f4284700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64ec161a60 0x7f64ec0c4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.170+0000 7f64f4284700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec0ad760 con 0x7f64ec161680 2023-08-25T03:34:14.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.170+0000 7f64f4284700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec150900 con 0x7f64ec161a60 2023-08-25T03:34:14.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.170+0000 7f64f4284700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ec0c3cb0 con 0x7f64ec0c1e90 2023-08-25T03:34:14.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.170+0000 7f67e5b8d700 1 -- 172.21.15.2:0/4218561424 wait complete. 2023-08-25T03:34:14.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.170+0000 7f64f2821700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64ec161a60 0x7f64ec0c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.170+0000 7f64f2821700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64ec161a60 0x7f64ec0c4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.2:50718/0 (socket says 172.21.15.2:50718) 2023-08-25T03:34:14.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.170+0000 7f64f2020700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64ec0c1e90 0x7f64ec0ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.170+0000 7f64f2821700 1 -- 172.21.15.2:0/1752053408 learned_addr learned my addr 172.21.15.2:0/1752053408 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:14.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.170+0000 7f3a95d9b700 1 --2- 172.21.15.2:0/937304705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a980b5930 0x7f3a980d1cf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0098f0 tx=0x7f3a8c009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7259c8ceffb7b660 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.171+0000 7f64f2821700 1 -- 172.21.15.2:0/1752053408 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64ec0c1e90 msgr2=0x7f64ec0ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.171+0000 7f64f2821700 1 --2- 172.21.15.2:0/1752053408 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64ec0c1e90 0x7f64ec0ad220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.171+0000 7f64f2821700 1 -- 172.21.15.2:0/1752053408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64ec161680 msgr2=0x7f64ec06bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.171+0000 7f64f2821700 1 --2- 172.21.15.2:0/1752053408 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64ec161680 0x7f64ec06bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.171+0000 7f64f2821700 1 -- 172.21.15.2:0/1752053408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e80094a0 con 0x7f64ec161a60 2023-08-25T03:34:14.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.171+0000 7f3a9559a700 1 -- 172.21.15.2:0/937304705 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c004230 con 0x7f3a980b5930 2023-08-25T03:34:14.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.171+0000 7f3a9559a700 1 -- 172.21.15.2:0/937304705 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8c004390 con 0x7f3a980b5930 2023-08-25T03:34:14.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.171+0000 7f3a9559a700 1 -- 172.21.15.2:0/937304705 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c0036a0 con 0x7f3a980b5930 2023-08-25T03:34:14.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.170+0000 7f64ccff9700 1 -- 172.21.15.2:0/2732516070 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c800da70 con 0x7f64d0157620 2023-08-25T03:34:14.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.171+0000 7f64ccff9700 1 -- 172.21.15.2:0/2732516070 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c8004e90 con 0x7f64d0157620 2023-08-25T03:34:14.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.171+0000 7f64ccff9700 1 -- 172.21.15.2:0/2732516070 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c800ddf0 con 0x7f64d0157620 2023-08-25T03:34:14.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.171+0000 7f64f2020700 1 --2- 172.21.15.2:0/1752053408 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64ec0c1e90 0x7f64ec0ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:34:14.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.171+0000 7f64d66fd700 1 -- 172.21.15.2:0/2732516070 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d0158070 con 0x7f64d0157620 2023-08-25T03:34:14.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.171+0000 7f64d66fd700 1 -- 172.21.15.2:0/2732516070 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d01546c0 con 0x7f64d0157620 2023-08-25T03:34:14.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.172+0000 7f64d66fd700 1 -- 172.21.15.2:0/2732516070 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f64b0000fc0 con 0x7f64d0157620 2023-08-25T03:34:14.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.172+0000 7f64f2821700 1 --2- 172.21.15.2:0/1752053408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64ec161a60 0x7f64ec0c4140 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f64e400b970 tx=0x7f64e400bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.172+0000 7f67e5b8d700 1 Processor -- start 2023-08-25T03:34:14.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.172+0000 7fe7aaffd700 1 -- 172.21.15.2:0/4034411578 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe7ac007780 con 0x7fe7b41f8b70 2023-08-25T03:34:14.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.172+0000 7fe7aaffd700 1 --2- 172.21.15.2:0/4034411578 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe79806a130 0x7fe79806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.172+0000 7fe7aaffd700 1 -- 172.21.15.2:0/4034411578 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe7ac088380 con 0x7fe7b41f8b70 2023-08-25T03:34:14.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.172+0000 7fe7ba019700 1 --2- 172.21.15.2:0/4034411578 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe79806a130 0x7fe79806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.173+0000 7fe7aaffd700 1 --2- 172.21.15.2:0/4034411578 >> [v2:172.21.15.2:6816/83694260,v1:172.21.15.2:6817/83694260] conn(0x7fe7980729f0 0x7fe798074e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.173+0000 7fe7aaffd700 1 -- 172.21.15.2:0/4034411578 --> [v2:172.21.15.2:6816/83694260,v1:172.21.15.2:6817/83694260] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe7980754c0 con 0x7fe7980729f0 2023-08-25T03:34:14.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.173+0000 7fe7aaffd700 1 -- 172.21.15.2:0/4034411578 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7ac088820 con 0x7fe7b41f8b70 2023-08-25T03:34:14.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.173+0000 7f3a9cc44700 1 -- 172.21.15.2:0/937304705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a980b5930 msgr2=0x7f3a980d1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.173+0000 7f3a9cc44700 1 --2- 172.21.15.2:0/937304705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a980b5930 0x7f3a980d1cf0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c0098f0 tx=0x7f3a8c009c00 comp rx=0 tx=0).stop 2023-08-25T03:34:14.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.173+0000 7f64e2ffd700 1 -- 172.21.15.2:0/1752053408 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e400c930 con 0x7f64ec161a60 2023-08-25T03:34:14.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.173+0000 7f64e2ffd700 1 -- 172.21.15.2:0/1752053408 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e400cab0 con 0x7f64ec161a60 2023-08-25T03:34:14.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.173+0000 7f64e2ffd700 1 -- 172.21.15.2:0/1752053408 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e400ce60 con 0x7f64ec161a60 2023-08-25T03:34:14.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.173+0000 7fe7ba019700 1 --2- 172.21.15.2:0/4034411578 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe79806a130 0x7fe79806c5f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40b1ab0 tx=0x7fe7a4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.173+0000 7f67e5b8d700 1 -- start start 2023-08-25T03:34:14.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.174+0000 7f64f4284700 1 -- 172.21.15.2:0/1752053408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ec0c5f90 con 0x7f64ec161a60 2023-08-25T03:34:14.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.174+0000 7f64f4284700 1 -- 172.21.15.2:0/1752053408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ec0c6510 con 0x7f64ec161a60 2023-08-25T03:34:14.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.174+0000 7f3a9cc44700 1 -- 172.21.15.2:0/937304705 shutdown_connections 2023-08-25T03:34:14.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.174+0000 7f3a9cc44700 1 --2- 172.21.15.2:0/937304705 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a980b5930 0x7f3a980d1cf0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.174+0000 7f3a9cc44700 1 -- 172.21.15.2:0/937304705 >> 172.21.15.2:0/937304705 conn(0x7f3a980b4c00 msgr2=0x7f3a980b5010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:14.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.174+0000 7f3a9cc44700 1 -- 172.21.15.2:0/937304705 shutdown_connections 2023-08-25T03:34:14.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.174+0000 7f3a9cc44700 1 -- 172.21.15.2:0/937304705 wait complete. 2023-08-25T03:34:14.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.174+0000 7f67e5b8d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67e00ce4f0 0x7f67e014e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.174+0000 7f67e5b8d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67e015d390 0x7f67e0155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.174+0000 7f67e5b8d700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f67e0160e00 0x7f67e0156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.174+0000 7f67e5b8d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e0156a20 con 0x7f67e015d390 2023-08-25T03:34:14.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.174+0000 7f67e5b8d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e0156740 con 0x7f67e00ce4f0 2023-08-25T03:34:14.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.174+0000 7f67e5b8d700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e01f6410 con 0x7f67e0160e00 2023-08-25T03:34:14.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.174+0000 7f67df7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67e00ce4f0 0x7f67e014e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.174+0000 7f67df7fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67e00ce4f0 0x7f67e014e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.2:50732/0 (socket says 172.21.15.2:50732) 2023-08-25T03:34:14.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.174+0000 7f67df7fe700 1 -- 172.21.15.2:0/1962358348 learned_addr learned my addr 172.21.15.2:0/1962358348 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:14.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.174+0000 7f67dffff700 1 --2- 172.21.15.2:0/1962358348 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f67e0160e00 0x7f67e0156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.175+0000 7f64f4284700 1 -- 172.21.15.2:0/1752053408 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f64ec162840 con 0x7f64ec161a60 2023-08-25T03:34:14.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.175+0000 7f67df7fe700 1 -- 172.21.15.2:0/1962358348 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f67e0160e00 msgr2=0x7f67e0156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.175+0000 7f67df7fe700 1 --2- 172.21.15.2:0/1962358348 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f67e0160e00 0x7f67e0156110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.175+0000 7f67df7fe700 1 -- 172.21.15.2:0/1962358348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67e015d390 msgr2=0x7f67e0155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.175+0000 7f67df7fe700 1 --2- 172.21.15.2:0/1962358348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f67e015d390 0x7f67e0155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.175+0000 7f67df7fe700 1 -- 172.21.15.2:0/1962358348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d00094a0 con 0x7f67e00ce4f0 2023-08-25T03:34:14.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.174+0000 7f64ccff9700 1 -- 172.21.15.2:0/2732516070 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f64c8013380 con 0x7f64d0157620 2023-08-25T03:34:14.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.175+0000 7f64ccff9700 1 --2- 172.21.15.2:0/2732516070 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f64bc06a1a0 0x7f64bc06c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.175+0000 7f64ccff9700 1 -- 172.21.15.2:0/2732516070 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f64c8088690 con 0x7f64d0157620 2023-08-25T03:34:14.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.175+0000 7f64cffff700 1 --2- 172.21.15.2:0/2732516070 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f64bc06a1a0 0x7f64bc06c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.175+0000 7f64ccff9700 1 --2- 172.21.15.2:0/2732516070 >> [v2:172.21.15.31:6824/4389955,v1:172.21.15.31:6825/4389955] conn(0x7f64bc072a60 0x7f64bc074e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.175+0000 7f64ccff9700 1 -- 172.21.15.2:0/2732516070 --> [v2:172.21.15.31:6824/4389955,v1:172.21.15.31:6825/4389955] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f64bc075530 con 0x7f64bc072a60 2023-08-25T03:34:14.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.175+0000 7f64ccff9700 1 -- 172.21.15.2:0/2732516070 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64c8088a40 con 0x7f64d0157620 2023-08-25T03:34:14.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.175+0000 7f67df7fe700 1 --2- 172.21.15.2:0/1962358348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67e00ce4f0 0x7f67e014e7d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f67d400e810 tx=0x7f67d400eb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.175+0000 7f64d4c9a700 1 --2- 172.21.15.2:0/2732516070 >> [v2:172.21.15.31:6824/4389955,v1:172.21.15.31:6825/4389955] conn(0x7f64bc072a60 0x7f64bc074e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.175+0000 7f67c7fff700 1 -- 172.21.15.2:0/1962358348 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d4013610 con 0x7f67e00ce4f0 2023-08-25T03:34:14.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.175+0000 7f67c7fff700 1 -- 172.21.15.2:0/1962358348 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d4013770 con 0x7f67e00ce4f0 2023-08-25T03:34:14.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.175+0000 7f67e5b8d700 1 -- 172.21.15.2:0/1962358348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e01f66f0 con 0x7f67e00ce4f0 2023-08-25T03:34:14.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.175+0000 7f67e5b8d700 1 -- 172.21.15.2:0/1962358348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e0155940 con 0x7f67e00ce4f0 2023-08-25T03:34:14.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.176+0000 7f64d4c9a700 1 --2- 172.21.15.2:0/2732516070 >> [v2:172.21.15.31:6824/4389955,v1:172.21.15.31:6825/4389955] conn(0x7f64bc072a60 0x7f64bc074e80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.176+0000 7f3a9cc44700 1 Processor -- start 2023-08-25T03:34:14.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.176+0000 7f3a9cc44700 1 -- start start 2023-08-25T03:34:14.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.175+0000 7f67c7fff700 1 -- 172.21.15.2:0/1962358348 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d4013b20 con 0x7f67e00ce4f0 2023-08-25T03:34:14.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.176+0000 7f67e5b8d700 1 -- 172.21.15.2:0/1962358348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f67c0000fc0 con 0x7f67e00ce4f0 2023-08-25T03:34:14.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.177+0000 7f17c97fa700 1 -- 172.21.15.2:0/1796900296 <== osd.6 v2:172.21.15.31:6808/1680871561 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f17ac006bd0 con 0x7f17ac0015e0 2023-08-25T03:34:14.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.177+0000 7f64e2ffd700 1 -- 172.21.15.2:0/1752053408 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f64e4014440 con 0x7f64ec161a60 2023-08-25T03:34:14.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.177+0000 7f64cffff700 1 --2- 172.21.15.2:0/2732516070 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f64bc06a1a0 0x7f64bc06c660 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f64c0007fd0 tx=0x7f64c0007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.177+0000 7fe7b9818700 1 --2- 172.21.15.2:0/4034411578 >> [v2:172.21.15.2:6816/83694260,v1:172.21.15.2:6817/83694260] conn(0x7fe7980729f0 0x7fe798074e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.177+0000 7f64ccff9700 1 -- 172.21.15.2:0/2732516070 <== osd.4 v2:172.21.15.31:6824/4389955 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f64bc075530 con 0x7f64bc072a60 2023-08-25T03:34:14.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.170+0000 7f3bf2afe700 1 Processor -- start 2023-08-25T03:34:14.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.170+0000 7f3bf2afe700 1 -- start start 2023-08-25T03:34:14.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.172+0000 7f3bf2afe700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3bec070bb0 0x7f3bec16e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.172+0000 7f3bf2afe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3bec0cdfc0 0x7f3bec0c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.172+0000 7f3bf2afe700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bec0ca150 con 0x7f3bec070f90 2023-08-25T03:34:14.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.172+0000 7f3bf2afe700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bec0ccb20 con 0x7f3bec0cdfc0 2023-08-25T03:34:14.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.172+0000 7f3bf2afe700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec0ca390 con 0x7f3bec070bb0 2023-08-25T03:34:14.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.172+0000 7f3bebfff700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f3bec070f90 0x7f3bec08a0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46098/0 (socket says 172.21.15.2:46098) 2023-08-25T03:34:14.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.172+0000 7f3bebfff700 1 -- 172.21.15.2:0/1536522811 learned_addr learned my addr 172.21.15.2:0/1536522811 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:14.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.172+0000 7f3bf089a700 1 --2- 172.21.15.2:0/1536522811 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3bec070bb0 0x7f3bec16e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.173+0000 7f3bf109b700 1 --2- 172.21.15.2:0/1536522811 >> v2:172.21.15.31:3300/0 conn(0x7f3bec0cdfc0 0x7f3bec0c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.172+0000 7f3beb7fe700 1 -- 172.21.15.2:0/1536522811 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1968177087 0 0) 0x7f3bdc001110 con 0x7f3bec070f90 2023-08-25T03:34:14.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.177+0000 7f64e2ffd700 1 --2- 172.21.15.2:0/1752053408 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f64d006a130 0x7f64d006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.177+0000 7f64e2ffd700 1 -- 172.21.15.2:0/1752053408 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f64e4088100 con 0x7f64ec161a60 2023-08-25T03:34:14.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.177+0000 7f3bf089a700 1 -- 172.21.15.2:0/1536522811 >> v2:172.21.15.31:3300/0 conn(0x7f3bec0cdfc0 msgr2=0x7f3bec0c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:14.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.177+0000 7f3bf089a700 1 --2- 172.21.15.2:0/1536522811 >> v2:172.21.15.31:3300/0 conn(0x7f3bec0cdfc0 0x7f3bec0c9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.177+0000 7f3bf089a700 1 -- 172.21.15.2:0/1536522811 >> v1:172.21.15.2:6789/0 conn(0x7f3bec070f90 legacy=0x7f3bec08a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.177+0000 7f3bf089a700 1 -- 172.21.15.2:0/1536522811 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bec0ca670 con 0x7f3bec070bb0 2023-08-25T03:34:14.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.177+0000 7fe7b9818700 1 --2- 172.21.15.2:0/4034411578 >> [v2:172.21.15.2:6816/83694260,v1:172.21.15.2:6817/83694260] conn(0x7fe7980729f0 0x7fe798074e10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.177+0000 7f64e2ffd700 1 --2- 172.21.15.2:0/1752053408 >> [v2:172.21.15.2:6808/2226089280,v1:172.21.15.2:6809/2226089280] conn(0x7f64d00729f0 0x7f64d0074e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.177+0000 7f64e2ffd700 1 -- 172.21.15.2:0/1752053408 --> [v2:172.21.15.2:6808/2226089280,v1:172.21.15.2:6809/2226089280] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f64d00754c0 con 0x7f64d00729f0 2023-08-25T03:34:14.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.177+0000 7f64e2ffd700 1 -- 172.21.15.2:0/1752053408 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f64e408a6e0 con 0x7f64ec161a60 2023-08-25T03:34:14.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.178+0000 7f3bf089a700 1 --2- 172.21.15.2:0/1536522811 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3bec070bb0 0x7f3bec16e710 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3be0007830 tx=0x7f3be0007b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8281d2f0287a1ac9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.178+0000 7f3a9cc44700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a980b5930 0x7f3a9814fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.178+0000 7f3a9cc44700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3a980d2230 0x7f3a98150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.178+0000 7f3a9cc44700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a981619d0 0x7f3a98071c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.178+0000 7f3a9cc44700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a98070ee0 con 0x7f3a980b5930 2023-08-25T03:34:14.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.178+0000 7f3a9cc44700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a98153310 con 0x7f3a981619d0 2023-08-25T03:34:14.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.178+0000 7f3a9cc44700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a98071160 con 0x7f3a980d2230 2023-08-25T03:34:14.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.178+0000 7f3a96d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a981619d0 0x7f3a98071c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.178+0000 7f3a96d9d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a981619d0 0x7f3a98071c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.2:50746/0 (socket says 172.21.15.2:50746) 2023-08-25T03:34:14.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.178+0000 7f3a96d9d700 1 -- 172.21.15.2:0/225990300 learned_addr learned my addr 172.21.15.2:0/225990300 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:14.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.178+0000 7f3a95d9b700 1 --2- 172.21.15.2:0/225990300 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3a980d2230 0x7f3a98150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.178+0000 7f64f181f700 1 --2- 172.21.15.2:0/1752053408 >> [v2:172.21.15.2:6808/2226089280,v1:172.21.15.2:6809/2226089280] conn(0x7f64d00729f0 0x7f64d0074e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.178+0000 7f3a95d9b700 1 -- 172.21.15.2:0/225990300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a981619d0 msgr2=0x7f3a98071c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.178+0000 7f3a95d9b700 1 --2- 172.21.15.2:0/225990300 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a981619d0 0x7f3a98071c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.178+0000 7f3a95d9b700 1 -- 172.21.15.2:0/225990300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a980b5930 msgr2=0x7f3a9814fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.178+0000 7f3a95d9b700 1 --2- 172.21.15.2:0/225990300 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3a980b5930 0x7f3a9814fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.178+0000 7f3a95d9b700 1 -- 172.21.15.2:0/225990300 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a8c009550 con 0x7f3a980d2230 2023-08-25T03:34:14.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.178+0000 7fe7aaffd700 1 -- 172.21.15.2:0/4034411578 <== osd.2 v2:172.21.15.2:6816/83694260 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fe7980754c0 con 0x7fe7980729f0 2023-08-25T03:34:14.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.178+0000 7f64f181f700 1 --2- 172.21.15.2:0/1752053408 >> [v2:172.21.15.2:6808/2226089280,v1:172.21.15.2:6809/2226089280] conn(0x7f64d00729f0 0x7f64d0074e10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.178+0000 7f3a95d9b700 1 --2- 172.21.15.2:0/225990300 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3a980d2230 0x7f3a98150550 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c015000 tx=0x7f3a8c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.179+0000 7f3a86ffd700 1 -- 172.21.15.2:0/225990300 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c00b660 con 0x7f3a980d2230 2023-08-25T03:34:14.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.178+0000 7f3beb7fe700 1 -- 172.21.15.2:0/1536522811 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be000f070 con 0x7f3bec070bb0 2023-08-25T03:34:14.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.178+0000 7f3beb7fe700 1 -- 172.21.15.2:0/1536522811 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be000c7d0 con 0x7f3bec070bb0 2023-08-25T03:34:14.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.178+0000 7f3beb7fe700 1 -- 172.21.15.2:0/1536522811 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be000cb60 con 0x7f3bec070bb0 2023-08-25T03:34:14.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.179+0000 7f3bf2afe700 1 -- 172.21.15.2:0/1536522811 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3bec070bb0 msgr2=0x7f3bec16e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.179+0000 7f3bf2afe700 1 --2- 172.21.15.2:0/1536522811 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3bec070bb0 0x7f3bec16e710 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3be0007830 tx=0x7f3be0007b40 comp rx=0 tx=0).stop 2023-08-25T03:34:14.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.179+0000 7f3a86ffd700 1 -- 172.21.15.2:0/225990300 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8c00b7c0 con 0x7f3a980d2230 2023-08-25T03:34:14.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.179+0000 7f3a9cc44700 1 -- 172.21.15.2:0/225990300 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a98071440 con 0x7f3a980d2230 2023-08-25T03:34:14.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.179+0000 7f3a9cc44700 1 -- 172.21.15.2:0/225990300 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a981501b0 con 0x7f3a980d2230 2023-08-25T03:34:14.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.179+0000 7f3a86ffd700 1 -- 172.21.15.2:0/225990300 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c00bb70 con 0x7f3a980d2230 2023-08-25T03:34:14.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.181+0000 7f3a86ffd700 1 -- 172.21.15.2:0/225990300 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3a8c021070 con 0x7f3a980d2230 2023-08-25T03:34:14.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.181+0000 7f3a86ffd700 1 --2- 172.21.15.2:0/225990300 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3a7806a0e0 0x7f3a7806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.181+0000 7f3a86ffd700 1 -- 172.21.15.2:0/225990300 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3a8c089cc0 con 0x7f3a980d2230 2023-08-25T03:34:14.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.182+0000 7f64e2ffd700 1 -- 172.21.15.2:0/1752053408 <== osd.0 v2:172.21.15.2:6808/2226089280 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f64d00754c0 con 0x7f64d00729f0 2023-08-25T03:34:14.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.182+0000 7f64f2020700 1 --2- 172.21.15.2:0/1752053408 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f64d006a130 0x7f64d006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.182+0000 7f3a9659c700 1 --2- 172.21.15.2:0/225990300 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3a7806a0e0 0x7f3a7806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.182+0000 7f3a9659c700 1 --2- 172.21.15.2:0/225990300 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3a7806a0e0 0x7f3a7806c5a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3a900012e0 tx=0x7f3a90001780 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.182+0000 7f64f2020700 1 --2- 172.21.15.2:0/1752053408 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f64d006a130 0x7f64d006c5f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f64e80090b0 tx=0x7f64e8010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.183+0000 7f3a9cc44700 1 --2- 172.21.15.2:0/225990300 >> [v2:172.21.15.2:6800/2145903773,v1:172.21.15.2:6801/2145903773] conn(0x7f3a98150a90 0x7f3a980ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.183+0000 7f3a9cc44700 1 -- 172.21.15.2:0/225990300 --> [v2:172.21.15.2:6800/2145903773,v1:172.21.15.2:6801/2145903773] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3a980b8590 con 0x7f3a98150a90 2023-08-25T03:34:14.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.183+0000 7f3a96d9d700 1 --2- 172.21.15.2:0/225990300 >> [v2:172.21.15.2:6800/2145903773,v1:172.21.15.2:6801/2145903773] conn(0x7f3a98150a90 0x7f3a980ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.183+0000 7f3a96d9d700 1 --2- 172.21.15.2:0/225990300 >> [v2:172.21.15.2:6800/2145903773,v1:172.21.15.2:6801/2145903773] conn(0x7f3a98150a90 0x7f3a980ca1a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.184+0000 7f3a86ffd700 1 -- 172.21.15.2:0/225990300 <== osd.1 v2:172.21.15.2:6800/2145903773 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f3a880015a0 con 0x7f3a98150a90 2023-08-25T03:34:14.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.189+0000 7f67c7fff700 1 -- 172.21.15.2:0/1962358348 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f67d4013c80 con 0x7f67e00ce4f0 2023-08-25T03:34:14.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.189+0000 7f67c7fff700 1 --2- 172.21.15.2:0/1962358348 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f67cc06a0e0 0x7f67cc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.190+0000 7f67c7fff700 1 -- 172.21.15.2:0/1962358348 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f67d40886b0 con 0x7f67e00ce4f0 2023-08-25T03:34:14.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.190+0000 7f67deffd700 1 --2- 172.21.15.2:0/1962358348 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f67cc06a0e0 0x7f67cc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.190+0000 7f67c7fff700 1 --2- 172.21.15.2:0/1962358348 >> [v2:172.21.15.2:6822/2180868296,v1:172.21.15.2:6824/2180868296] conn(0x7f67cc072920 0x7f67cc074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.190+0000 7f67c7fff700 1 -- 172.21.15.2:0/1962358348 --> [v2:172.21.15.2:6822/2180868296,v1:172.21.15.2:6824/2180868296] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f67cc0753f0 con 0x7f67cc072920 2023-08-25T03:34:14.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.190+0000 7f67c7fff700 1 -- 172.21.15.2:0/1962358348 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67d4088ad0 con 0x7f67e00ce4f0 2023-08-25T03:34:14.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.190+0000 7f67deffd700 1 --2- 172.21.15.2:0/1962358348 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f67cc06a0e0 0x7f67cc06c5a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f67d80012e0 tx=0x7f67d8001780 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.193+0000 7fcf61532700 1 -- 172.21.15.2:0/2211737749 --> [v2:172.21.15.31:6800/1678889094,v1:172.21.15.31:6801/1678889094] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcf5c0cdd90 con 0x7fcf48072880 2023-08-25T03:34:14.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.194+0000 7fcf437fe700 1 -- 172.21.15.2:0/2211737749 <== osd.5 v2:172.21.15.31:6800/1678889094 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcf5400f040 con 0x7fcf48072880 2023-08-25T03:34:14.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.194+0000 7fcf61532700 1 -- 172.21.15.2:0/2211737749 >> [v2:172.21.15.31:6800/1678889094,v1:172.21.15.31:6801/1678889094] conn(0x7fcf48072880 msgr2=0x7fcf48074ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.194+0000 7fcf61532700 1 --2- 172.21.15.2:0/2211737749 >> [v2:172.21.15.31:6800/1678889094,v1:172.21.15.31:6801/1678889094] conn(0x7fcf48072880 0x7fcf48074ca0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.194+0000 7fcf61532700 1 -- 172.21.15.2:0/2211737749 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fcf4806a1b0 msgr2=0x7fcf4806c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.194+0000 7fcf61532700 1 --2- 172.21.15.2:0/2211737749 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fcf4806a1b0 0x7fcf4806c670 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0ae160 tx=0x7fcf4c008040 comp rx=0 tx=0).stop 2023-08-25T03:34:14.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.194+0000 7fcf61532700 1 -- 172.21.15.2:0/2211737749 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcf5c070460 msgr2=0x7fcf5c155630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.194+0000 7fcf61532700 1 --2- 172.21.15.2:0/2211737749 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcf5c070460 0x7fcf5c155630 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcf5c0c94a0 tx=0x7fcf50004750 comp rx=0 tx=0).stop 2023-08-25T03:34:14.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.194+0000 7fcf5b7fe700 1 -- 172.21.15.2:0/2211737749 reap_dead start 2023-08-25T03:34:14.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.194+0000 7fcf61532700 1 -- 172.21.15.2:0/2211737749 shutdown_connections 2023-08-25T03:34:14.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.194+0000 7fcf61532700 1 -- 172.21.15.2:0/2211737749 >> 172.21.15.2:0/2211737749 conn(0x7fcf5c0ceec0 msgr2=0x7fcf5c0ac120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:14.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.194+0000 7fcf61532700 1 -- 172.21.15.2:0/2211737749 shutdown_connections 2023-08-25T03:34:14.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.194+0000 7fcf61532700 1 -- 172.21.15.2:0/2211737749 wait complete. 2023-08-25T03:34:14.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.198+0000 7fe7bc27d700 1 -- 172.21.15.2:0/4034411578 --> [v2:172.21.15.2:6816/83694260,v1:172.21.15.2:6817/83694260] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe7b4162840 con 0x7fe7980729f0 2023-08-25T03:34:14.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.199+0000 7fe7aaffd700 1 -- 172.21.15.2:0/4034411578 <== osd.2 v2:172.21.15.2:6816/83694260 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe7b0001530 con 0x7fe7980729f0 2023-08-25T03:34:14.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.199+0000 7fe7bc27d700 1 -- 172.21.15.2:0/4034411578 >> [v2:172.21.15.2:6816/83694260,v1:172.21.15.2:6817/83694260] conn(0x7fe7980729f0 msgr2=0x7fe798074e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.199+0000 7fe7bc27d700 1 --2- 172.21.15.2:0/4034411578 >> [v2:172.21.15.2:6816/83694260,v1:172.21.15.2:6817/83694260] conn(0x7fe7980729f0 0x7fe798074e10 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.199+0000 7fe7bc27d700 1 -- 172.21.15.2:0/4034411578 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe79806a130 msgr2=0x7fe79806c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.199+0000 7fe7bc27d700 1 --2- 172.21.15.2:0/4034411578 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe79806a130 0x7fe79806c5f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40b1ab0 tx=0x7fe7a4008040 comp rx=0 tx=0).stop 2023-08-25T03:34:14.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.199+0000 7fe7bc27d700 1 -- 172.21.15.2:0/4034411578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7b41f8b70 msgr2=0x7fe7b406c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.200+0000 7fe7bc27d700 1 --2- 172.21.15.2:0/4034411578 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7b41f8b70 0x7fe7b406c010 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40717c0 tx=0x7fe7ac00c9e0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.200+0000 7fe7ba81a700 1 -- 172.21.15.2:0/4034411578 reap_dead start 2023-08-25T03:34:14.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.200+0000 7fe7bc27d700 1 -- 172.21.15.2:0/4034411578 shutdown_connections 2023-08-25T03:34:14.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.200+0000 7fe7bc27d700 1 -- 172.21.15.2:0/4034411578 >> 172.21.15.2:0/4034411578 conn(0x7fe7b40ceec0 msgr2=0x7fe7b40ce6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:14.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.200+0000 7fe7bc27d700 1 -- 172.21.15.2:0/4034411578 shutdown_connections 2023-08-25T03:34:14.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.200+0000 7fe7bc27d700 1 -- 172.21.15.2:0/4034411578 wait complete. 2023-08-25T03:34:14.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.201+0000 7f67dffff700 1 --2- 172.21.15.2:0/1962358348 >> [v2:172.21.15.2:6822/2180868296,v1:172.21.15.2:6824/2180868296] conn(0x7f67cc072920 0x7f67cc074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.201+0000 7f3bf2afe700 1 -- 172.21.15.2:0/1536522811 shutdown_connections 2023-08-25T03:34:14.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.201+0000 7f3bf2afe700 1 --2- 172.21.15.2:0/1536522811 >> v2:172.21.15.31:3300/0 conn(0x7f3bec0cdfc0 0x7f3bec0c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.201+0000 7f3bf2afe700 1 --2- 172.21.15.2:0/1536522811 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3bec070bb0 0x7f3bec16e710 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.201+0000 7f3bf2afe700 1 -- 172.21.15.2:0/1536522811 >> 172.21.15.2:0/1536522811 conn(0x7f3bec0ceec0 msgr2=0x7f3bec0cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:14.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.201+0000 7f67dffff700 1 --2- 172.21.15.2:0/1962358348 >> [v2:172.21.15.2:6822/2180868296,v1:172.21.15.2:6824/2180868296] conn(0x7f67cc072920 0x7f67cc074d40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.201+0000 7f3bf2afe700 1 -- 172.21.15.2:0/1536522811 shutdown_connections 2023-08-25T03:34:14.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.201+0000 7f3bf2afe700 1 -- 172.21.15.2:0/1536522811 wait complete. 2023-08-25T03:34:14.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.203+0000 7f3bf2afe700 1 Processor -- start 2023-08-25T03:34:14.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.203+0000 7f3bf2afe700 1 -- start start 2023-08-25T03:34:14.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.204+0000 7f3bf2afe700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3bec070bb0 0x7f3bec2067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.204+0000 7f3bf2afe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bec070f90 0x7f3bec06df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.204+0000 7f3bf2afe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bec0cdfc0 0x7f3bec06e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.204+0000 7f3bf2afe700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec0b0a10 con 0x7f3bec0cdfc0 2023-08-25T03:34:14.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.204+0000 7f3bf2afe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec0b1870 con 0x7f3bec070f90 2023-08-25T03:34:14.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.204+0000 7f3bf2afe700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec0b0c60 con 0x7f3bec070bb0 2023-08-25T03:34:14.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.205+0000 7f3bf089a700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3bec070bb0 0x7f3bec2067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.205+0000 7f3bf109b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bec0cdfc0 0x7f3bec06e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.205+0000 7f3bf089a700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3bec070bb0 0x7f3bec2067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.2:59716/0 (socket says 172.21.15.2:59716) 2023-08-25T03:34:14.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.205+0000 7f3bf089a700 1 -- 172.21.15.2:0/2815944907 learned_addr learned my addr 172.21.15.2:0/2815944907 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:14.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.205+0000 7f3bf089a700 1 -- 172.21.15.2:0/2815944907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bec070f90 msgr2=0x7f3bec06df40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-25T03:34:14.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.205+0000 7f3bf089a700 1 --2- 172.21.15.2:0/2815944907 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3bec070f90 0x7f3bec06df40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.205+0000 7f3bf089a700 1 -- 172.21.15.2:0/2815944907 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bec0cdfc0 msgr2=0x7f3bec06e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.205+0000 7f3bf089a700 1 --2- 172.21.15.2:0/2815944907 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3bec0cdfc0 0x7f3bec06e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.205+0000 7f3bf089a700 1 -- 172.21.15.2:0/2815944907 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be0007510 con 0x7f3bec070bb0 2023-08-25T03:34:14.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.205+0000 7f3bf089a700 1 --2- 172.21.15.2:0/2815944907 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3bec070bb0 0x7f3bec2067c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3be00042f0 tx=0x7f3be0004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.206+0000 7f3be97fa700 1 -- 172.21.15.2:0/2815944907 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0003ab0 con 0x7f3bec070bb0 2023-08-25T03:34:14.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.206+0000 7f3be97fa700 1 -- 172.21.15.2:0/2815944907 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be0003c10 con 0x7f3bec070bb0 2023-08-25T03:34:14.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.206+0000 7f3be97fa700 1 -- 172.21.15.2:0/2815944907 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0007390 con 0x7f3bec070bb0 2023-08-25T03:34:14.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.206+0000 7f3bf2afe700 1 -- 172.21.15.2:0/2815944907 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bec0b0f40 con 0x7f3bec070bb0 2023-08-25T03:34:14.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.206+0000 7f3bf2afe700 1 -- 172.21.15.2:0/2815944907 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bec0b1500 con 0x7f3bec070bb0 2023-08-25T03:34:14.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.208+0000 7f3be97fa700 1 -- 172.21.15.2:0/2815944907 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3be001d440 con 0x7f3bec070bb0 2023-08-25T03:34:14.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.208+0000 7f3be97fa700 1 --2- 172.21.15.2:0/2815944907 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3bd406a1a0 0x7f3bd406c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.209+0000 7f3be97fa700 1 -- 172.21.15.2:0/2815944907 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3be0089bd0 con 0x7f3bec070bb0 2023-08-25T03:34:14.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.209+0000 7f17d2ea2700 1 -- 172.21.15.2:0/1796900296 --> [v2:172.21.15.31:6808/1680871561,v1:172.21.15.31:6809/1680871561] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f17ac005c80 con 0x7f17ac0015e0 2023-08-25T03:34:14.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.209+0000 7f17c97fa700 1 -- 172.21.15.2:0/1796900296 <== osd.6 v2:172.21.15.31:6808/1680871561 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f17ac005c80 con 0x7f17ac0015e0 2023-08-25T03:34:14.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.210+0000 7f3bf2afe700 1 --2- 172.21.15.2:0/2815944907 >> [v2:172.21.15.31:6816/3294787191,v1:172.21.15.31:6817/3294787191] conn(0x7f3bc80015e0 0x7f3bc8003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.210+0000 7f3bf2afe700 1 -- 172.21.15.2:0/2815944907 --> [v2:172.21.15.31:6816/3294787191,v1:172.21.15.31:6817/3294787191] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3bc8006bd0 con 0x7f3bc80015e0 2023-08-25T03:34:14.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.210+0000 7f67c7fff700 1 -- 172.21.15.2:0/1962358348 <== osd.3 v2:172.21.15.2:6822/2180868296 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f67cc0753f0 con 0x7f67cc072920 2023-08-25T03:34:14.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.210+0000 7f3bf109b700 1 --2- 172.21.15.2:0/2815944907 >> [v2:172.21.15.31:6816/3294787191,v1:172.21.15.31:6817/3294787191] conn(0x7f3bc80015e0 0x7f3bc8003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.210+0000 7f17d2ea2700 1 -- 172.21.15.2:0/1796900296 >> [v2:172.21.15.31:6808/1680871561,v1:172.21.15.31:6809/1680871561] conn(0x7f17ac0015e0 msgr2=0x7f17ac003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.210+0000 7f17d2ea2700 1 --2- 172.21.15.2:0/1796900296 >> [v2:172.21.15.31:6808/1680871561,v1:172.21.15.31:6809/1680871561] conn(0x7f17ac0015e0 0x7f17ac003aa0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.210+0000 7f64d66fd700 1 -- 172.21.15.2:0/2732516070 --> [v2:172.21.15.31:6824/4389955,v1:172.21.15.31:6825/4389955] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f64b0002d40 con 0x7f64bc072a60 2023-08-25T03:34:14.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.210+0000 7f17d2ea2700 1 -- 172.21.15.2:0/1796900296 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f17b006a0e0 msgr2=0x7f17b006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.210+0000 7f17d2ea2700 1 --2- 172.21.15.2:0/1796900296 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f17b006a0e0 0x7f17b006c5a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f17c0001920 tx=0x7f17c0009040 comp rx=0 tx=0).stop 2023-08-25T03:34:14.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.210+0000 7f17d2ea2700 1 -- 172.21.15.2:0/1796900296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17cc15d580 msgr2=0x7f17cc0c7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.210+0000 7f17d2ea2700 1 --2- 172.21.15.2:0/1796900296 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17cc15d580 0x7f17cc0c7040 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f17c4017000 tx=0x7f17c4004ad0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.210+0000 7f17d143f700 1 -- 172.21.15.2:0/1796900296 reap_dead start 2023-08-25T03:34:14.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.210+0000 7f17d2ea2700 1 -- 172.21.15.2:0/1796900296 shutdown_connections 2023-08-25T03:34:14.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.210+0000 7f17d2ea2700 1 -- 172.21.15.2:0/1796900296 >> 172.21.15.2:0/1796900296 conn(0x7f17cc0ceec0 msgr2=0x7f17cc15c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:14.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.210+0000 7f17d2ea2700 1 -- 172.21.15.2:0/1796900296 shutdown_connections 2023-08-25T03:34:14.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.211+0000 7f17d2ea2700 1 -- 172.21.15.2:0/1796900296 wait complete. 2023-08-25T03:34:14.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.211+0000 7f3bf109b700 1 --2- 172.21.15.2:0/2815944907 >> [v2:172.21.15.31:6816/3294787191,v1:172.21.15.31:6817/3294787191] conn(0x7f3bc80015e0 0x7f3bc8003aa0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.211+0000 7f3bebfff700 1 --2- 172.21.15.2:0/2815944907 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3bd406a1a0 0x7f3bd406c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.291 INFO:teuthology.orchestra.run.smithi002.stdout:60129542152 2023-08-25T03:34:14.292 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-08-25T03:34:14.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.212+0000 7f64ccff9700 1 -- 172.21.15.2:0/2732516070 <== osd.4 v2:172.21.15.31:6824/4389955 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f64b0002d40 con 0x7f64bc072a60 2023-08-25T03:34:14.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.212+0000 7f64d66fd700 1 -- 172.21.15.2:0/2732516070 >> [v2:172.21.15.31:6824/4389955,v1:172.21.15.31:6825/4389955] conn(0x7f64bc072a60 msgr2=0x7f64bc074e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.212+0000 7f64d66fd700 1 --2- 172.21.15.2:0/2732516070 >> [v2:172.21.15.31:6824/4389955,v1:172.21.15.31:6825/4389955] conn(0x7f64bc072a60 0x7f64bc074e80 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.211+0000 7f3bebfff700 1 --2- 172.21.15.2:0/2815944907 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3bd406a1a0 0x7f3bd406c660 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc001370 tx=0x7f3bdc0012a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.213+0000 7f64d66fd700 1 -- 172.21.15.2:0/2732516070 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f64bc06a1a0 msgr2=0x7f64bc06c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.213+0000 7f64d66fd700 1 --2- 172.21.15.2:0/2732516070 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f64bc06a1a0 0x7f64bc06c660 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f64c0007fd0 tx=0x7f64c0007de0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.213+0000 7f64d66fd700 1 -- 172.21.15.2:0/2732516070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64d0157620 msgr2=0x7f64d00c5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.213+0000 7f64d66fd700 1 --2- 172.21.15.2:0/2732516070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f64d0157620 0x7f64d00c5250 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f64c800ff40 tx=0x7f64c800ff70 comp rx=0 tx=0).stop 2023-08-25T03:34:14.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.213+0000 7f64d4c9a700 1 -- 172.21.15.2:0/2732516070 reap_dead start 2023-08-25T03:34:14.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.213+0000 7f64d66fd700 1 -- 172.21.15.2:0/2732516070 shutdown_connections 2023-08-25T03:34:14.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.213+0000 7f64d66fd700 1 -- 172.21.15.2:0/2732516070 >> 172.21.15.2:0/2732516070 conn(0x7f64d00ceec0 msgr2=0x7f64d00ae0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:14.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.213+0000 7f3be97fa700 1 -- 172.21.15.2:0/2815944907 <== osd.7 v2:172.21.15.31:6816/3294787191 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f3bc8006bd0 con 0x7f3bc80015e0 2023-08-25T03:34:14.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.213+0000 7f64d66fd700 1 -- 172.21.15.2:0/2732516070 shutdown_connections 2023-08-25T03:34:14.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.213+0000 7f64d66fd700 1 -- 172.21.15.2:0/2732516070 wait complete. 2023-08-25T03:34:14.296 INFO:teuthology.orchestra.run.smithi002.stdout:55834574856 2023-08-25T03:34:14.297 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-08-25T03:34:14.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.216+0000 7f64f4284700 1 -- 172.21.15.2:0/1752053408 --> [v2:172.21.15.2:6808/2226089280,v1:172.21.15.2:6809/2226089280] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f64ec166270 con 0x7f64d00729f0 2023-08-25T03:34:14.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.217+0000 7f64e2ffd700 1 -- 172.21.15.2:0/1752053408 <== osd.0 v2:172.21.15.2:6808/2226089280 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f64dc009750 con 0x7f64d00729f0 2023-08-25T03:34:14.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.217+0000 7f3a9cc44700 1 -- 172.21.15.2:0/225990300 --> [v2:172.21.15.2:6800/2145903773,v1:172.21.15.2:6801/2145903773] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3a980b8b50 con 0x7f3a98150a90 2023-08-25T03:34:14.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.218+0000 7f3a86ffd700 1 -- 172.21.15.2:0/225990300 <== osd.1 v2:172.21.15.2:6800/2145903773 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3a880019e0 con 0x7f3a98150a90 2023-08-25T03:34:14.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.218+0000 7f3a9cc44700 1 -- 172.21.15.2:0/225990300 >> [v2:172.21.15.2:6800/2145903773,v1:172.21.15.2:6801/2145903773] conn(0x7f3a98150a90 msgr2=0x7f3a980ca1a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.218+0000 7f3a9cc44700 1 --2- 172.21.15.2:0/225990300 >> [v2:172.21.15.2:6800/2145903773,v1:172.21.15.2:6801/2145903773] conn(0x7f3a98150a90 0x7f3a980ca1a0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.220+0000 7f3a9cc44700 1 -- 172.21.15.2:0/225990300 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3a7806a0e0 msgr2=0x7f3a7806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.220+0000 7f3a9cc44700 1 --2- 172.21.15.2:0/225990300 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3a7806a0e0 0x7f3a7806c5a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3a900012e0 tx=0x7f3a90001780 comp rx=0 tx=0).stop 2023-08-25T03:34:14.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.220+0000 7f3a9cc44700 1 -- 172.21.15.2:0/225990300 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3a980d2230 msgr2=0x7f3a98150550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.220+0000 7f3a9cc44700 1 --2- 172.21.15.2:0/225990300 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3a980d2230 0x7f3a98150550 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c015000 tx=0x7f3a8c004750 comp rx=0 tx=0).stop 2023-08-25T03:34:14.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.220+0000 7f3a96d9d700 1 -- 172.21.15.2:0/225990300 reap_dead start 2023-08-25T03:34:14.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.220+0000 7f3a9cc44700 1 -- 172.21.15.2:0/225990300 shutdown_connections 2023-08-25T03:34:14.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.220+0000 7f3a9cc44700 1 -- 172.21.15.2:0/225990300 >> 172.21.15.2:0/225990300 conn(0x7f3a980b4c00 msgr2=0x7f3a980cc590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:14.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.220+0000 7f3a9cc44700 1 -- 172.21.15.2:0/225990300 shutdown_connections 2023-08-25T03:34:14.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.220+0000 7f64f4284700 1 -- 172.21.15.2:0/1752053408 >> [v2:172.21.15.2:6808/2226089280,v1:172.21.15.2:6809/2226089280] conn(0x7f64d00729f0 msgr2=0x7f64d0074e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.220+0000 7f64f4284700 1 --2- 172.21.15.2:0/1752053408 >> [v2:172.21.15.2:6808/2226089280,v1:172.21.15.2:6809/2226089280] conn(0x7f64d00729f0 0x7f64d0074e10 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.220+0000 7f64f4284700 1 -- 172.21.15.2:0/1752053408 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f64d006a130 msgr2=0x7f64d006c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.220+0000 7f64f4284700 1 --2- 172.21.15.2:0/1752053408 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f64d006a130 0x7f64d006c5f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f64e80090b0 tx=0x7f64e8010000 comp rx=0 tx=0).stop 2023-08-25T03:34:14.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.220+0000 7f64f4284700 1 -- 172.21.15.2:0/1752053408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64ec161a60 msgr2=0x7f64ec0c4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.220+0000 7f64f4284700 1 --2- 172.21.15.2:0/1752053408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64ec161a60 0x7f64ec0c4140 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f64e400b970 tx=0x7f64e400bc80 comp rx=0 tx=0).stop 2023-08-25T03:34:14.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.220+0000 7f3a9cc44700 1 -- 172.21.15.2:0/225990300 wait complete. 2023-08-25T03:34:14.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.220+0000 7f64f2821700 1 -- 172.21.15.2:0/1752053408 reap_dead start 2023-08-25T03:34:14.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.222+0000 7f64f4284700 1 -- 172.21.15.2:0/1752053408 shutdown_connections 2023-08-25T03:34:14.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.222+0000 7f64f4284700 1 -- 172.21.15.2:0/1752053408 >> 172.21.15.2:0/1752053408 conn(0x7f64ec0ceec0 msgr2=0x7f64ec164530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:14.304 INFO:teuthology.orchestra.run.smithi002.stdout:60129542153 2023-08-25T03:34:14.304 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-08-25T03:34:14.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.232+0000 7f64f4284700 1 -- 172.21.15.2:0/1752053408 shutdown_connections 2023-08-25T03:34:14.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.233+0000 7f64f4284700 1 -- 172.21.15.2:0/1752053408 wait complete. 2023-08-25T03:34:14.306 INFO:teuthology.orchestra.run.smithi002.stdout:60129542153 2023-08-25T03:34:14.306 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-08-25T03:34:14.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.238+0000 7f3bf2afe700 1 -- 172.21.15.2:0/2815944907 --> [v2:172.21.15.31:6816/3294787191,v1:172.21.15.31:6817/3294787191] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3bc8005cb0 con 0x7f3bc80015e0 2023-08-25T03:34:14.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.239+0000 7f3be97fa700 1 -- 172.21.15.2:0/2815944907 <== osd.7 v2:172.21.15.31:6816/3294787191 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3bc8005cb0 con 0x7f3bc80015e0 2023-08-25T03:34:14.308 INFO:teuthology.orchestra.run.smithi002.stdout:55834574857 2023-08-25T03:34:14.308 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-08-25T03:34:14.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.240+0000 7f3bf2afe700 1 -- 172.21.15.2:0/2815944907 >> [v2:172.21.15.31:6816/3294787191,v1:172.21.15.31:6817/3294787191] conn(0x7f3bc80015e0 msgr2=0x7f3bc8003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.240+0000 7f3bf2afe700 1 --2- 172.21.15.2:0/2815944907 >> [v2:172.21.15.31:6816/3294787191,v1:172.21.15.31:6817/3294787191] conn(0x7f3bc80015e0 0x7f3bc8003aa0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.240+0000 7f3bf2afe700 1 -- 172.21.15.2:0/2815944907 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3bd406a1a0 msgr2=0x7f3bd406c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.240+0000 7f3bf2afe700 1 --2- 172.21.15.2:0/2815944907 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3bd406a1a0 0x7f3bd406c660 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc001370 tx=0x7f3bdc0012a0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.240+0000 7f3bf2afe700 1 -- 172.21.15.2:0/2815944907 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3bec070bb0 msgr2=0x7f3bec2067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.240+0000 7f3bf2afe700 1 --2- 172.21.15.2:0/2815944907 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3bec070bb0 0x7f3bec2067c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3be00042f0 tx=0x7f3be0004750 comp rx=0 tx=0).stop 2023-08-25T03:34:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.240+0000 7f3bf109b700 1 -- 172.21.15.2:0/2815944907 reap_dead start 2023-08-25T03:34:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.240+0000 7f3bf2afe700 1 -- 172.21.15.2:0/2815944907 shutdown_connections 2023-08-25T03:34:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.240+0000 7f3bf2afe700 1 -- 172.21.15.2:0/2815944907 >> 172.21.15.2:0/2815944907 conn(0x7f3bec0ceec0 msgr2=0x7f3bec0c4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.240+0000 7f3bf2afe700 1 -- 172.21.15.2:0/2815944907 shutdown_connections 2023-08-25T03:34:14.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.240+0000 7f3bf2afe700 1 -- 172.21.15.2:0/2815944907 wait complete. 2023-08-25T03:34:14.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.241+0000 7f67e5b8d700 1 -- 172.21.15.2:0/1962358348 --> [v2:172.21.15.2:6822/2180868296,v1:172.21.15.2:6824/2180868296] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f67c0002d40 con 0x7f67cc072920 2023-08-25T03:34:14.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.241+0000 7f67c7fff700 1 -- 172.21.15.2:0/1962358348 <== osd.3 v2:172.21.15.2:6822/2180868296 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f67c0002d40 con 0x7f67cc072920 2023-08-25T03:34:14.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.241+0000 7f67e5b8d700 1 -- 172.21.15.2:0/1962358348 >> [v2:172.21.15.2:6822/2180868296,v1:172.21.15.2:6824/2180868296] conn(0x7f67cc072920 msgr2=0x7f67cc074d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.241+0000 7f67e5b8d700 1 --2- 172.21.15.2:0/1962358348 >> [v2:172.21.15.2:6822/2180868296,v1:172.21.15.2:6824/2180868296] conn(0x7f67cc072920 0x7f67cc074d40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.242+0000 7f67e5b8d700 1 -- 172.21.15.2:0/1962358348 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f67cc06a0e0 msgr2=0x7f67cc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.242+0000 7f67e5b8d700 1 --2- 172.21.15.2:0/1962358348 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f67cc06a0e0 0x7f67cc06c5a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f67d80012e0 tx=0x7f67d8001780 comp rx=0 tx=0).stop 2023-08-25T03:34:14.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.242+0000 7f67e5b8d700 1 -- 172.21.15.2:0/1962358348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67e00ce4f0 msgr2=0x7f67e014e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.242+0000 7f67e5b8d700 1 --2- 172.21.15.2:0/1962358348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67e00ce4f0 0x7f67e014e7d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f67d400e810 tx=0x7f67d400eb20 comp rx=0 tx=0).stop 2023-08-25T03:34:14.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.242+0000 7f67dffff700 1 -- 172.21.15.2:0/1962358348 reap_dead start 2023-08-25T03:34:14.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.242+0000 7f67e5b8d700 1 -- 172.21.15.2:0/1962358348 shutdown_connections 2023-08-25T03:34:14.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.242+0000 7f67e5b8d700 1 -- 172.21.15.2:0/1962358348 >> 172.21.15.2:0/1962358348 conn(0x7f67e006abe0 msgr2=0x7f67e00b61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:14.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.242+0000 7f67e5b8d700 1 -- 172.21.15.2:0/1962358348 shutdown_connections 2023-08-25T03:34:14.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.242+0000 7f67e5b8d700 1 -- 172.21.15.2:0/1962358348 wait complete. 2023-08-25T03:34:14.314 INFO:teuthology.orchestra.run.smithi002.stdout:55834574856 2023-08-25T03:34:14.314 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-08-25T03:34:14.315 INFO:teuthology.orchestra.run.smithi002.stdout:60129542153 2023-08-25T03:34:14.316 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-08-25T03:34:14.316 INFO:teuthology.orchestra.run.smithi002.stdout:55834574857 2023-08-25T03:34:14.317 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-08-25T03:34:14.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.544+0000 7f9ba37c6700 1 Processor -- start 2023-08-25T03:34:14.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.545+0000 7f9ba37c6700 1 -- start start 2023-08-25T03:34:14.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.546+0000 7f9ba37c6700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9b9c161680 0x7f9b9c165d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.546+0000 7f9ba37c6700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9b9c0c1e90 0x7f9b9c0c22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.546+0000 7f9ba37c6700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b9c0c2830 con 0x7f9b9c161a60 2023-08-25T03:34:14.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.546+0000 7f9ba37c6700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b9c165320 con 0x7f9b9c161680 2023-08-25T03:34:14.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.546+0000 7f9ba37c6700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c1658a0 con 0x7f9b9c0c1e90 2023-08-25T03:34:14.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.546+0000 7f9ba1d63700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f9b9c161a60 0x7f9b9c161eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46150/0 (socket says 172.21.15.2:46150) 2023-08-25T03:34:14.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.546+0000 7f9ba1d63700 1 -- 172.21.15.2:0/294111790 learned_addr learned my addr 172.21.15.2:0/294111790 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:14.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.546+0000 7f9ba1562700 1 --2- 172.21.15.2:0/294111790 >> v2:172.21.15.31:3300/0 conn(0x7f9b9c161680 0x7f9b9c165d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.546+0000 7f9ba1562700 1 -- 172.21.15.2:0/294111790 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9b9c0c1e90 msgr2=0x7f9b9c0c22f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-25T03:34:14.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.546+0000 7f9ba1562700 1 --2- 172.21.15.2:0/294111790 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9b9c0c1e90 0x7f9b9c0c22f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.547+0000 7f9ba1562700 1 -- 172.21.15.2:0/294111790 >> v1:172.21.15.2:6789/0 conn(0x7f9b9c161a60 legacy=0x7f9b9c161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.547+0000 7f9ba1562700 1 -- 172.21.15.2:0/294111790 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c1625c0 con 0x7f9b9c161680 2023-08-25T03:34:14.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.547+0000 7f9ba1562700 1 --2- 172.21.15.2:0/294111790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b9c161680 0x7f9b9c165d30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9b90009840 tx=0x7f9b90009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e66b579188da830c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.547+0000 7f9b8ffff700 1 -- 172.21.15.2:0/294111790 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b90011830 con 0x7f9b9c161680 2023-08-25T03:34:14.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.548+0000 7f9b8ffff700 1 -- 172.21.15.2:0/294111790 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b90011990 con 0x7f9b9c161680 2023-08-25T03:34:14.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.548+0000 7f9b8ffff700 1 -- 172.21.15.2:0/294111790 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b90011d20 con 0x7f9b9c161680 2023-08-25T03:34:14.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.549+0000 7f9ba37c6700 1 -- 172.21.15.2:0/294111790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b9c161680 msgr2=0x7f9b9c165d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.549+0000 7f9ba37c6700 1 --2- 172.21.15.2:0/294111790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b9c161680 0x7f9b9c165d30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9b90009840 tx=0x7f9b90009b50 comp rx=0 tx=0).stop 2023-08-25T03:34:14.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.550+0000 7f9ba37c6700 1 -- 172.21.15.2:0/294111790 shutdown_connections 2023-08-25T03:34:14.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.550+0000 7f9ba37c6700 1 --2- 172.21.15.2:0/294111790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b9c161680 0x7f9b9c165d30 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9b90009840 tx=0x7f9b90009b50 comp rx=0 tx=0).stop 2023-08-25T03:34:14.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.550+0000 7f9ba37c6700 1 --2- 172.21.15.2:0/294111790 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9b9c0c1e90 0x7f9b9c0c22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.550+0000 7f9ba37c6700 1 -- 172.21.15.2:0/294111790 >> 172.21.15.2:0/294111790 conn(0x7f9b9c0ceec0 msgr2=0x7f9b9c0cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:14.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.554+0000 7f9ba37c6700 1 -- 172.21.15.2:0/294111790 shutdown_connections 2023-08-25T03:34:14.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.554+0000 7f9ba37c6700 1 -- 172.21.15.2:0/294111790 wait complete. 2023-08-25T03:34:14.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.556+0000 7f9ba37c6700 1 Processor -- start 2023-08-25T03:34:14.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.556+0000 7f9ba37c6700 1 -- start start 2023-08-25T03:34:14.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.557+0000 7f9ba37c6700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9b9c0c1e90 0x7f9b9c0b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.557+0000 7f9ba37c6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b9c161a60 0x7f9b9c15dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.557+0000 7f9ba37c6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b9c06bdc0 0x7f9b9c06de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.557+0000 7f9ba37c6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c0c7ec0 con 0x7f9b9c161a60 2023-08-25T03:34:14.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.557+0000 7f9ba37c6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c15e550 con 0x7f9b9c06bdc0 2023-08-25T03:34:14.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.557+0000 7f9ba37c6700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c0c7b90 con 0x7f9b9c0c1e90 2023-08-25T03:34:14.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.558+0000 7f9ba1562700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9b9c0c1e90 0x7f9b9c0b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.558+0000 7f9ba1562700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9b9c0c1e90 0x7f9b9c0b2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.2:59738/0 (socket says 172.21.15.2:59738) 2023-08-25T03:34:14.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.558+0000 7f9ba1562700 1 -- 172.21.15.2:0/190487447 learned_addr learned my addr 172.21.15.2:0/190487447 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:14.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.558+0000 7f9ba1d63700 1 --2- 172.21.15.2:0/190487447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b9c06bdc0 0x7f9b9c06de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.558+0000 7f9ba0d61700 1 --2- 172.21.15.2:0/190487447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b9c161a60 0x7f9b9c15dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.558+0000 7f9ba1562700 1 -- 172.21.15.2:0/190487447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b9c06bdc0 msgr2=0x7f9b9c06de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.558+0000 7f9ba1562700 1 --2- 172.21.15.2:0/190487447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b9c06bdc0 0x7f9b9c06de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.558+0000 7f9ba1562700 1 -- 172.21.15.2:0/190487447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b9c161a60 msgr2=0x7f9b9c15dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.558+0000 7f9ba1562700 1 --2- 172.21.15.2:0/190487447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b9c161a60 0x7f9b9c15dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.558+0000 7f9ba1562700 1 -- 172.21.15.2:0/190487447 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b900094a0 con 0x7f9b9c0c1e90 2023-08-25T03:34:14.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.558+0000 7f9ba1562700 1 --2- 172.21.15.2:0/190487447 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9b9c0c1e90 0x7f9b9c0b2b30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9b90013000 tx=0x7f9b90004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.559+0000 7f9b8dffb700 1 -- 172.21.15.2:0/190487447 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b900044d0 con 0x7f9b9c0c1e90 2023-08-25T03:34:14.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.559+0000 7f9b8dffb700 1 -- 172.21.15.2:0/190487447 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b900036a0 con 0x7f9b9c0c1e90 2023-08-25T03:34:14.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.559+0000 7f9b8dffb700 1 -- 172.21.15.2:0/190487447 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b90003a10 con 0x7f9b9c0c1e90 2023-08-25T03:34:14.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.559+0000 7f9ba37c6700 1 -- 172.21.15.2:0/190487447 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b9c0c66f0 con 0x7f9b9c0c1e90 2023-08-25T03:34:14.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.560+0000 7f9ba37c6700 1 -- 172.21.15.2:0/190487447 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9c0c6410 con 0x7f9b9c0c1e90 2023-08-25T03:34:14.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.554+0000 7f327f3f7700 1 Processor -- start 2023-08-25T03:34:14.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.554+0000 7f327f3f7700 1 -- start start 2023-08-25T03:34:14.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.556+0000 7f327f3f7700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3278070f90 0x7f327816e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.556+0000 7f327f3f7700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32780ccb20 con 0x7f3278070bb0 2023-08-25T03:34:14.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.556+0000 7f327f3f7700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32780ca390 con 0x7f32780cdfc0 2023-08-25T03:34:14.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.556+0000 7f327f3f7700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32780ca670 con 0x7f3278070f90 2023-08-25T03:34:14.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.556+0000 7f327d994700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3278070f90 0x7f327816e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.556+0000 7f327d994700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3278070f90 0x7f327816e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.2:50782/0 (socket says 172.21.15.2:50782) 2023-08-25T03:34:14.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.556+0000 7f327d994700 1 -- 172.21.15.2:0/811325945 learned_addr learned my addr 172.21.15.2:0/811325945 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:14.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.556+0000 7f327c992700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f32780cdfc0 0x7f327808a7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.2:54180/0 (socket says 172.21.15.2:54180) 2023-08-25T03:34:14.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.556+0000 7f326bfff700 1 -- 172.21.15.2:0/811325945 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2169485092 0 0) 0x7f32780ca390 con 0x7f32780cdfc0 2023-08-25T03:34:14.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.560+0000 7f326bfff700 1 -- 172.21.15.2:0/811325945 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3264003590 con 0x7f32780cdfc0 2023-08-25T03:34:14.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.560+0000 7f326bfff700 1 -- 172.21.15.2:0/811325945 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2091030475 0 0) 0x7f32780ccb20 con 0x7f3278070bb0 2023-08-25T03:34:14.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.560+0000 7f326bfff700 1 -- 172.21.15.2:0/811325945 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32780ca390 con 0x7f3278070bb0 2023-08-25T03:34:14.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.560+0000 7f327d994700 1 -- 172.21.15.2:0/811325945 >> v1:172.21.15.31:6789/0 conn(0x7f32780cdfc0 legacy=0x7f327808a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.560+0000 7f327d994700 1 -- 172.21.15.2:0/811325945 >> v1:172.21.15.2:6789/0 conn(0x7f3278070bb0 legacy=0x7f327808a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.560+0000 7f327d994700 1 -- 172.21.15.2:0/811325945 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32780ca950 con 0x7f3278070f90 2023-08-25T03:34:14.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.561+0000 7fe1e9b07700 1 Processor -- start 2023-08-25T03:34:14.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.561+0000 7f9ba37c6700 1 -- 172.21.15.2:0/190487447 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b9c0b4e40 con 0x7f9b9c0c1e90 2023-08-25T03:34:14.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.561+0000 7fe1e9b07700 1 -- start start 2023-08-25T03:34:14.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.562+0000 7fe1e9b07700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe1e41570b0 0x7fe1e40702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.562+0000 7fe1e9b07700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe1e415d750 0x7fe1e40ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.562+0000 7fe1e9b07700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1e4157490 con 0x7fe1e40cad70 2023-08-25T03:34:14.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.562+0000 7fe1e9b07700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1e40717c0 con 0x7fe1e415d750 2023-08-25T03:34:14.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.562+0000 7fe1e9b07700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e40cca90 con 0x7fe1e41570b0 2023-08-25T03:34:14.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.562+0000 7f9b8dffb700 1 -- 172.21.15.2:0/190487447 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9b90003b70 con 0x7f9b9c0c1e90 2023-08-25T03:34:14.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.562+0000 7f9b8dffb700 1 --2- 172.21.15.2:0/190487447 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9b8006a0b0 0x7f9b8006c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.562+0000 7f9b8dffb700 1 -- 172.21.15.2:0/190487447 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9b90089ab0 con 0x7f9b9c0c1e90 2023-08-25T03:34:14.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.563+0000 7fe1e3fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe1e415d750 0x7fe1e40ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.563+0000 7fe1e2ffd700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fe1e40cad70 0x7fe1e415d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46168/0 (socket says 172.21.15.2:46168) 2023-08-25T03:34:14.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.563+0000 7fe1e2ffd700 1 -- 172.21.15.2:0/559360844 learned_addr learned my addr 172.21.15.2:0/559360844 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:14.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.563+0000 7fe1e3fff700 1 -- 172.21.15.2:0/559360844 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe1e41570b0 msgr2=0x7fe1e40702d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-25T03:34:14.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.563+0000 7fe1e3fff700 1 --2- 172.21.15.2:0/559360844 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe1e41570b0 0x7fe1e40702d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.563+0000 7fe1e27fc700 1 -- 172.21.15.2:0/559360844 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3844325302 0 0) 0x7fe1e4157490 con 0x7fe1e40cad70 2023-08-25T03:34:14.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.563+0000 7fe1e3fff700 1 -- 172.21.15.2:0/559360844 >> v1:172.21.15.2:6789/0 conn(0x7fe1e40cad70 legacy=0x7fe1e415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.563+0000 7fe1e3fff700 1 -- 172.21.15.2:0/559360844 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e4070c40 con 0x7fe1e415d750 2023-08-25T03:34:14.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.563+0000 7fe1e3fff700 1 --2- 172.21.15.2:0/559360844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1e415d750 0x7fe1e40ccee0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009d90 tx=0x7fe1dc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd333397e4368133 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.563+0000 7fe1e27fc700 1 -- 172.21.15.2:0/559360844 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc00e070 con 0x7fe1e415d750 2023-08-25T03:34:14.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.563+0000 7fe1e27fc700 1 -- 172.21.15.2:0/559360844 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1dc00bbd0 con 0x7fe1e415d750 2023-08-25T03:34:14.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.563+0000 7fe1e27fc700 1 -- 172.21.15.2:0/559360844 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc012430 con 0x7fe1e415d750 2023-08-25T03:34:14.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.564+0000 7f327d994700 1 --2- 172.21.15.2:0/811325945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3278070f90 0x7f327816e710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f32700083f0 tx=0x7f327000a870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d0a6ef23bca890a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.564+0000 7f9ba0d61700 1 --2- 172.21.15.2:0/190487447 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9b8006a0b0 0x7f9b8006c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.564+0000 7f326bfff700 1 -- 172.21.15.2:0/811325945 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c0015f0 con 0x7f3278070f90 2023-08-25T03:34:14.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.564+0000 7f326bfff700 1 -- 172.21.15.2:0/811325945 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f327000a2e0 con 0x7f3278070f90 2023-08-25T03:34:14.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.564+0000 7f9ba0d61700 1 --2- 172.21.15.2:0/190487447 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9b8006a0b0 0x7f9b8006c570 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0b93a0 tx=0x7f9b98008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.564+0000 7fe1e9b07700 1 -- 172.21.15.2:0/559360844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1e415d750 msgr2=0x7fe1e40ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.564+0000 7fe1e9b07700 1 --2- 172.21.15.2:0/559360844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1e415d750 0x7fe1e40ccee0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc009d90 tx=0x7fe1dc00b330 comp rx=0 tx=0).stop 2023-08-25T03:34:14.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.564+0000 7fe1e9b07700 1 -- 172.21.15.2:0/559360844 shutdown_connections 2023-08-25T03:34:14.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.564+0000 7fe1e9b07700 1 --2- 172.21.15.2:0/559360844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1e415d750 0x7fe1e40ccee0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.564+0000 7fe1e9b07700 1 --2- 172.21.15.2:0/559360844 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe1e41570b0 0x7fe1e40702d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.564+0000 7fe1e9b07700 1 -- 172.21.15.2:0/559360844 >> 172.21.15.2:0/559360844 conn(0x7fe1e40ceec0 msgr2=0x7fe1e40cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:14.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.565+0000 7fe1e9b07700 1 -- 172.21.15.2:0/559360844 shutdown_connections 2023-08-25T03:34:14.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.564+0000 7f326bfff700 1 -- 172.21.15.2:0/811325945 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3274001670 con 0x7f3278070f90 2023-08-25T03:34:14.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.565+0000 7f327f3f7700 1 -- 172.21.15.2:0/811325945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3278070f90 msgr2=0x7f327816e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.565+0000 7f327f3f7700 1 --2- 172.21.15.2:0/811325945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3278070f90 0x7f327816e710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f32700083f0 tx=0x7f327000a870 comp rx=0 tx=0).stop 2023-08-25T03:34:14.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.565+0000 7f327f3f7700 1 -- 172.21.15.2:0/811325945 shutdown_connections 2023-08-25T03:34:14.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.565+0000 7f327f3f7700 1 --2- 172.21.15.2:0/811325945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3278070f90 0x7f327816e710 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.565+0000 7f327f3f7700 1 -- 172.21.15.2:0/811325945 >> 172.21.15.2:0/811325945 conn(0x7f32780ceec0 msgr2=0x7f32780cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:14.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.565+0000 7f327f3f7700 1 -- 172.21.15.2:0/811325945 shutdown_connections 2023-08-25T03:34:14.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.565+0000 7f327f3f7700 1 -- 172.21.15.2:0/811325945 wait complete. 2023-08-25T03:34:14.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.566+0000 7f327f3f7700 1 Processor -- start 2023-08-25T03:34:14.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.567+0000 7ff8c4510700 1 Processor -- start 2023-08-25T03:34:14.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.567+0000 7f259dc09700 1 Processor -- start 2023-08-25T03:34:14.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.565+0000 7fe1e9b07700 1 -- 172.21.15.2:0/559360844 wait complete. 2023-08-25T03:34:14.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.566+0000 7fe1e9b07700 1 Processor -- start 2023-08-25T03:34:14.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.566+0000 7fe1e9b07700 1 -- start start 2023-08-25T03:34:14.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.567+0000 7fe1e9b07700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe1e40cad70 0x7fe1e4151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.567+0000 7fe1e9b07700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1e41570b0 0x7fe1e41f5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.567+0000 7fe1e9b07700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1e415d750 0x7fe1e409b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.567+0000 7fe1e9b07700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e414f060 con 0x7fe1e415d750 2023-08-25T03:34:14.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.567+0000 7fe1e9b07700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e4151540 con 0x7fe1e41570b0 2023-08-25T03:34:14.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.567+0000 7fe1e9b07700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e414e120 con 0x7fe1e40cad70 2023-08-25T03:34:14.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.568+0000 7fe1e37fe700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe1e40cad70 0x7fe1e4151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.568+0000 7fe1e37fe700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe1e40cad70 0x7fe1e4151000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.2:59752/0 (socket says 172.21.15.2:59752) 2023-08-25T03:34:14.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.568+0000 7fe1e37fe700 1 -- 172.21.15.2:0/2668667352 learned_addr learned my addr 172.21.15.2:0/2668667352 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:14.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.568+0000 7ff8c4510700 1 -- start start 2023-08-25T03:34:14.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.568+0000 7fe1e2ffd700 1 --2- 172.21.15.2:0/2668667352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1e41570b0 0x7fe1e41f5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.568+0000 7f7805452700 1 Processor -- start 2023-08-25T03:34:14.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.566+0000 7f327f3f7700 1 -- start start 2023-08-25T03:34:14.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.568+0000 7f327f3f7700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3278070bb0 0x7f32780c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.568+0000 7f327f3f7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3278070f90 0x7f32780b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.568+0000 7f327f3f7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32780cdfc0 0x7f327820ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.568+0000 7f327f3f7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32780b1ae0 con 0x7f32780cdfc0 2023-08-25T03:34:14.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.568+0000 7f327f3f7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32780b4a60 con 0x7f3278070f90 2023-08-25T03:34:14.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.568+0000 7f327f3f7700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32780c6fb0 con 0x7f3278070bb0 2023-08-25T03:34:14.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7f327d193700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3278070bb0 0x7f32780c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7f327d193700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3278070bb0 0x7f32780c6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.2:59766/0 (socket says 172.21.15.2:59766) 2023-08-25T03:34:14.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7f327d193700 1 -- 172.21.15.2:0/545558814 learned_addr learned my addr 172.21.15.2:0/545558814 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:14.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.567+0000 7f259dc09700 1 -- start start 2023-08-25T03:34:14.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.568+0000 7f259dc09700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f25980b20b0 0x7f2598165e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.568+0000 7f259dc09700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2598161d50 0x7f25981621a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.568+0000 7f259dc09700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25981626e0 con 0x7f259815bd80 2023-08-25T03:34:14.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.568+0000 7f259dc09700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25980ce380 con 0x7f2598161d50 2023-08-25T03:34:14.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.568+0000 7f259dc09700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25980cfba0 con 0x7f25980b20b0 2023-08-25T03:34:14.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.568+0000 7f2596ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f25980b20b0 0x7f2598165e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.568+0000 7f2596ffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f25980b20b0 0x7f2598165e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.2:50808/0 (socket says 172.21.15.2:50808) 2023-08-25T03:34:14.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.568+0000 7f2596ffd700 1 -- 172.21.15.2:0/1368098779 learned_addr learned my addr 172.21.15.2:0/1368098779 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:14.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7f2597fff700 1 --2- 172.21.15.2:0/1368098779 >> v2:172.21.15.2:3300/0 conn(0x7f2598161d50 0x7f25981621a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7f2596ffd700 1 -- 172.21.15.2:0/1368098779 >> v2:172.21.15.2:3300/0 conn(0x7f2598161d50 msgr2=0x7f25981621a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:14.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7f2596ffd700 1 --2- 172.21.15.2:0/1368098779 >> v2:172.21.15.2:3300/0 conn(0x7f2598161d50 0x7f25981621a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7f2596ffd700 1 -- 172.21.15.2:0/1368098779 >> v1:172.21.15.31:6789/0 conn(0x7f259815bd80 legacy=0x7f25980b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:14.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7f2596ffd700 1 -- 172.21.15.2:0/1368098779 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2598162930 con 0x7f25980b20b0 2023-08-25T03:34:14.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.568+0000 7f8b7c4d1700 1 Processor -- start 2023-08-25T03:34:14.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.568+0000 7f8b7c4d1700 1 -- start start 2023-08-25T03:34:14.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7fe1e3fff700 1 --2- 172.21.15.2:0/2668667352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1e415d750 0x7fe1e409b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7fe1e37fe700 1 -- 172.21.15.2:0/2668667352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1e41570b0 msgr2=0x7fe1e41f5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7fe1e37fe700 1 --2- 172.21.15.2:0/2668667352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1e41570b0 0x7fe1e41f5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7fe1e37fe700 1 -- 172.21.15.2:0/2668667352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1e415d750 msgr2=0x7fe1e409b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7fe1e37fe700 1 --2- 172.21.15.2:0/2668667352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1e415d750 0x7fe1e409b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7fe1e37fe700 1 -- 172.21.15.2:0/2668667352 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc009a70 con 0x7fe1e40cad70 2023-08-25T03:34:14.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7fe1e37fe700 1 --2- 172.21.15.2:0/2668667352 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe1e40cad70 0x7fe1e4151000 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800b580 tx=0x7fe1d800b890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7f327d994700 1 --2- 172.21.15.2:0/545558814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32780cdfc0 0x7f327820ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7f327d193700 1 -- 172.21.15.2:0/545558814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3278070f90 msgr2=0x7f32780b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7f327d193700 1 --2- 172.21.15.2:0/545558814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3278070f90 0x7f32780b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7f327d193700 1 -- 172.21.15.2:0/545558814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32780cdfc0 msgr2=0x7f327820ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7f327d193700 1 --2- 172.21.15.2:0/545558814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32780cdfc0 0x7f327820ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7f327d193700 1 -- 172.21.15.2:0/545558814 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327000ab50 con 0x7f3278070bb0 2023-08-25T03:34:14.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7f2597fff700 1 --2- 172.21.15.2:0/1368098779 >> v2:172.21.15.2:3300/0 conn(0x7f2598161d50 0x7f25981621a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:14.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7f2596ffd700 1 --2- 172.21.15.2:0/1368098779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25980b20b0 0x7f2598165e20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f25980c6e70 tx=0x7f259000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3aac5fbaaad11c8a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7f9b8dffb700 1 -- 172.21.15.2:0/190487447 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9b90055230 con 0x7f9b9c0c1e90 2023-08-25T03:34:14.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.570+0000 7f25967fc700 1 -- 172.21.15.2:0/1368098779 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259000e070 con 0x7f25980b20b0 2023-08-25T03:34:14.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.570+0000 7f25967fc700 1 -- 172.21.15.2:0/1368098779 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2590003a20 con 0x7f25980b20b0 2023-08-25T03:34:14.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.570+0000 7f25967fc700 1 -- 172.21.15.2:0/1368098779 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590003db0 con 0x7f25980b20b0 2023-08-25T03:34:14.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7ff8c4510700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff8bc0cdc50 0x7ff8bc0b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7ff8c4510700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8bc0ce030 0x7ff8bc160f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.568+0000 7f7805452700 1 -- start start 2023-08-25T03:34:14.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.570+0000 7f7805452700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f78000b43c0 0x7f780006b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7ff8c4510700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff8bc161450 0x7ff8bc06b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7ff8c4510700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8bc06d430 con 0x7ff8bc0cdc50 2023-08-25T03:34:14.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7ff8c4510700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8bc06d710 con 0x7ff8bc161450 2023-08-25T03:34:14.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.570+0000 7f7805452700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f780006b730 0x7f78000b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.570+0000 7f7805452700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7800165360 con 0x7f78001616d0 2023-08-25T03:34:14.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.570+0000 7f7805452700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f780006bbc0 con 0x7f78000b43c0 2023-08-25T03:34:14.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.570+0000 7f7805452700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7800165600 con 0x7f780006b730 2023-08-25T03:34:14.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7ff8c4510700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc06d9f0 con 0x7ff8bc0ce030 2023-08-25T03:34:14.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.570+0000 7ff8c1aab700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8bc0ce030 0x7ff8bc160f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.570+0000 7ff8c1aab700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8bc0ce030 0x7ff8bc160f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.2:59770/0 (socket says 172.21.15.2:59770) 2023-08-25T03:34:14.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.570+0000 7ff8c1aab700 1 -- 172.21.15.2:0/3475825375 learned_addr learned my addr 172.21.15.2:0/3475825375 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:14.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.570+0000 7f8b7c4d1700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8b74071030 0x7f8b74070140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.570+0000 7f8b7c4d1700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b7408ad80 con 0x7f8b7408a8f0 2023-08-25T03:34:14.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.570+0000 7f8b7c4d1700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b74070680 con 0x7f8b74070940 2023-08-25T03:34:14.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.570+0000 7f8b7c4d1700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b740cd010 con 0x7f8b74071030 2023-08-25T03:34:14.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.570+0000 7f77feffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f78001616d0 0x7f7800162750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.2:54222/0 (socket says 172.21.15.2:54222) 2023-08-25T03:34:14.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.570+0000 7f77feffd700 1 -- 172.21.15.2:0/2706598129 learned_addr learned my addr 172.21.15.2:0/2706598129 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:14.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.570+0000 7f77fe7fc700 1 --2- 172.21.15.2:0/2706598129 >> v2:172.21.15.31:3300/0 conn(0x7f78000b43c0 0x7f780006b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.570+0000 7f77ff7fe700 1 --2- 172.21.15.2:0/2706598129 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f780006b730 0x7f78000b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7fe1cbfff700 1 -- 172.21.15.2:0/2668667352 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8011830 con 0x7fe1e40cad70 2023-08-25T03:34:14.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7fe1cbfff700 1 -- 172.21.15.2:0/2668667352 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d80119b0 con 0x7fe1e40cad70 2023-08-25T03:34:14.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.569+0000 7fe1cbfff700 1 -- 172.21.15.2:0/2668667352 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8011d60 con 0x7fe1e40cad70 2023-08-25T03:34:14.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.570+0000 7fe1e9b07700 1 -- 172.21.15.2:0/2668667352 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e414e3e0 con 0x7fe1e40cad70 2023-08-25T03:34:14.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.570+0000 7fe1e9b07700 1 -- 172.21.15.2:0/2668667352 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e414e9a0 con 0x7fe1e40cad70 2023-08-25T03:34:14.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.571+0000 7f259dc09700 1 -- 172.21.15.2:0/1368098779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25980b20b0 msgr2=0x7f2598165e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.571+0000 7f259dc09700 1 --2- 172.21.15.2:0/1368098779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25980b20b0 0x7f2598165e20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f25980c6e70 tx=0x7f259000b330 comp rx=0 tx=0).stop 2023-08-25T03:34:14.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.571+0000 7ff8c2aad700 1 --2- 172.21.15.2:0/3475825375 >> v2:172.21.15.31:3300/0 conn(0x7ff8bc161450 0x7ff8bc06b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.571+0000 7f77fe7fc700 1 -- 172.21.15.2:0/2706598129 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f780006b730 msgr2=0x7f78000b24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:14.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.571+0000 7f77fe7fc700 1 --2- 172.21.15.2:0/2706598129 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f780006b730 0x7f78000b24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.571+0000 7f77fe7fc700 1 -- 172.21.15.2:0/2706598129 >> v1:172.21.15.31:6789/0 conn(0x7f78001616d0 legacy=0x7f7800162750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.571+0000 7f77fe7fc700 1 -- 172.21.15.2:0/2706598129 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78001658e0 con 0x7f78000b43c0 2023-08-25T03:34:14.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.571+0000 7f327d193700 1 --2- 172.21.15.2:0/545558814 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3278070bb0 0x7f32780c6920 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f326c00c780 tx=0x7f326c00cb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.571+0000 7ff8c1aab700 1 -- 172.21.15.2:0/3475825375 >> v2:172.21.15.31:3300/0 conn(0x7ff8bc161450 msgr2=0x7ff8bc06b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:14.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.571+0000 7ff8c1aab700 1 --2- 172.21.15.2:0/3475825375 >> v2:172.21.15.31:3300/0 conn(0x7ff8bc161450 0x7ff8bc06b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.571+0000 7ff8c1aab700 1 -- 172.21.15.2:0/3475825375 >> v2:172.21.15.2:3300/0 conn(0x7ff8bc0cdc50 msgr2=0x7ff8bc0b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:14.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.571+0000 7ff8c1aab700 1 --2- 172.21.15.2:0/3475825375 >> v2:172.21.15.2:3300/0 conn(0x7ff8bc0cdc50 0x7ff8bc0b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.571+0000 7ff8c1aab700 1 -- 172.21.15.2:0/3475825375 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8bc06dcd0 con 0x7ff8bc0ce030 2023-08-25T03:34:14.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.571+0000 7f8b7a26d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8b74071030 0x7f8b74070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.571+0000 7f8b7a26d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8b74071030 0x7f8b74070140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.2:50820/0 (socket says 172.21.15.2:50820) 2023-08-25T03:34:14.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.571+0000 7f8b7a26d700 1 -- 172.21.15.2:0/1636942348 learned_addr learned my addr 172.21.15.2:0/1636942348 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:14.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.571+0000 7f3269ffb700 1 -- 172.21.15.2:0/545558814 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c005500 con 0x7f3278070bb0 2023-08-25T03:34:14.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.572+0000 7f3269ffb700 1 -- 172.21.15.2:0/545558814 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326c003680 con 0x7f3278070bb0 2023-08-25T03:34:14.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.572+0000 7f3269ffb700 1 -- 172.21.15.2:0/545558814 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c003a10 con 0x7f3278070bb0 2023-08-25T03:34:14.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.572+0000 7f77fe7fc700 1 --2- 172.21.15.2:0/2706598129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78000b43c0 0x7f780006b1f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f77f4009840 tx=0x7f77f4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f684d6ad3c47363 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.572+0000 7ff8c1aab700 1 --2- 172.21.15.2:0/3475825375 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8bc0ce030 0x7ff8bc160f10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc0cfd60 tx=0x7ff8b8009bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc5f66c8852fc1fe server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.572+0000 7ff8c12aa700 1 -- 172.21.15.2:0/3475825375 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8011830 con 0x7ff8bc0ce030 2023-08-25T03:34:14.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.572+0000 7f8b7a26d700 1 -- 172.21.15.2:0/1636942348 >> v1:172.21.15.31:6789/0 conn(0x7f8b74070940 legacy=0x7f8b7408a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:14.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.572+0000 7f8b7a26d700 1 -- 172.21.15.2:0/1636942348 >> v1:172.21.15.2:6789/0 conn(0x7f8b7408a8f0 legacy=0x7f8b740cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.572+0000 7f8b7a26d700 1 -- 172.21.15.2:0/1636942348 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b74172020 con 0x7f8b74071030 2023-08-25T03:34:14.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.572+0000 7f8b7926b700 1 -- 172.21.15.2:0/1636942348 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1182620361 0 0) 0x7f8b7408ad80 con 0x7f8b7408a8f0 2023-08-25T03:34:14.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.572+0000 7ff8c12aa700 1 -- 172.21.15.2:0/3475825375 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b8011990 con 0x7ff8bc0ce030 2023-08-25T03:34:14.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.572+0000 7ff8c12aa700 1 -- 172.21.15.2:0/3475825375 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b8011d20 con 0x7ff8bc0ce030 2023-08-25T03:34:14.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.572+0000 7f77fdffb700 1 -- 172.21.15.2:0/2706598129 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f4011830 con 0x7f78000b43c0 2023-08-25T03:34:14.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.572+0000 7f77fdffb700 1 -- 172.21.15.2:0/2706598129 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f4011990 con 0x7f78000b43c0 2023-08-25T03:34:14.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.572+0000 7f77fdffb700 1 -- 172.21.15.2:0/2706598129 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f4011d20 con 0x7f78000b43c0 2023-08-25T03:34:14.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.572+0000 7f259dc09700 1 -- 172.21.15.2:0/1368098779 shutdown_connections 2023-08-25T03:34:14.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.572+0000 7f259dc09700 1 --2- 172.21.15.2:0/1368098779 >> v2:172.21.15.2:3300/0 conn(0x7f2598161d50 0x7f25981621a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.572+0000 7f259dc09700 1 --2- 172.21.15.2:0/1368098779 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25980b20b0 0x7f2598165e20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.572+0000 7f259dc09700 1 -- 172.21.15.2:0/1368098779 >> 172.21.15.2:0/1368098779 conn(0x7f25980cfea0 msgr2=0x7f25980d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:14.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.572+0000 7f8b7a26d700 1 --2- 172.21.15.2:0/1636942348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b74071030 0x7f8b74070140 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8b700098f0 tx=0x7f8b70009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51c3cbbd53dc37bf server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.572+0000 7fe1cbfff700 1 -- 172.21.15.2:0/2668667352 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe1d801a400 con 0x7fe1e40cad70 2023-08-25T03:34:14.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.573+0000 7fe1cbfff700 1 --2- 172.21.15.2:0/2668667352 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe1d006a1b0 0x7fe1d006c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.573+0000 7fe1cbfff700 1 -- 172.21.15.2:0/2668667352 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe1d8088470 con 0x7fe1e40cad70 2023-08-25T03:34:14.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.573+0000 7fe1e9b07700 1 -- 172.21.15.2:0/2668667352 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c40052f0 con 0x7fe1e40cad70 2023-08-25T03:34:14.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.573+0000 7f327f3f7700 1 -- 172.21.15.2:0/545558814 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f327820b390 con 0x7f3278070bb0 2023-08-25T03:34:14.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.573+0000 7f327f3f7700 1 -- 172.21.15.2:0/545558814 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32780c6410 con 0x7f3278070bb0 2023-08-25T03:34:14.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.573+0000 7ff8c4510700 1 -- 172.21.15.2:0/3475825375 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8bc0ce030 msgr2=0x7ff8bc160f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.573+0000 7ff8c4510700 1 --2- 172.21.15.2:0/3475825375 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8bc0ce030 0x7ff8bc160f10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc0cfd60 tx=0x7ff8b8009bd0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.573+0000 7ff8c4510700 1 -- 172.21.15.2:0/3475825375 shutdown_connections 2023-08-25T03:34:14.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.573+0000 7ff8c4510700 1 --2- 172.21.15.2:0/3475825375 >> v2:172.21.15.31:3300/0 conn(0x7ff8bc161450 0x7ff8bc06b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.573+0000 7ff8c4510700 1 --2- 172.21.15.2:0/3475825375 >> v2:172.21.15.2:3300/0 conn(0x7ff8bc0cdc50 0x7ff8bc0b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.573+0000 7ff8c4510700 1 --2- 172.21.15.2:0/3475825375 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8bc0ce030 0x7ff8bc160f10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.573+0000 7ff8c4510700 1 -- 172.21.15.2:0/3475825375 >> 172.21.15.2:0/3475825375 conn(0x7ff8bc0cfea0 msgr2=0x7ff8bc0d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:14.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.573+0000 7fe1e2ffd700 1 --2- 172.21.15.2:0/2668667352 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe1d006a1b0 0x7fe1d006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.573+0000 7f8b7926b700 1 -- 172.21.15.2:0/1636942348 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b70004230 con 0x7f8b74071030 2023-08-25T03:34:14.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.573+0000 7ff8c4510700 1 -- 172.21.15.2:0/3475825375 shutdown_connections 2023-08-25T03:34:14.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.573+0000 7f8b7926b700 1 -- 172.21.15.2:0/1636942348 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b70004390 con 0x7f8b74071030 2023-08-25T03:34:14.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.573+0000 7f8b7926b700 1 -- 172.21.15.2:0/1636942348 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b700036a0 con 0x7f8b74071030 2023-08-25T03:34:14.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.573+0000 7ff8c4510700 1 -- 172.21.15.2:0/3475825375 wait complete. 2023-08-25T03:34:14.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.574+0000 7f7805452700 1 -- 172.21.15.2:0/2706598129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78000b43c0 msgr2=0x7f780006b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.574+0000 7f7805452700 1 --2- 172.21.15.2:0/2706598129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78000b43c0 0x7f780006b1f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f77f4009840 tx=0x7f77f4009b50 comp rx=0 tx=0).stop 2023-08-25T03:34:14.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.574+0000 7f7805452700 1 -- 172.21.15.2:0/2706598129 shutdown_connections 2023-08-25T03:34:14.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.574+0000 7f7805452700 1 --2- 172.21.15.2:0/2706598129 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f780006b730 0x7f78000b24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.574+0000 7f7805452700 1 --2- 172.21.15.2:0/2706598129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78000b43c0 0x7f780006b1f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.622 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.574+0000 7f7805452700 1 -- 172.21.15.2:0/2706598129 >> 172.21.15.2:0/2706598129 conn(0x7f78000cfd10 msgr2=0x7f780006ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:14.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.574+0000 7f7805452700 1 -- 172.21.15.2:0/2706598129 shutdown_connections 2023-08-25T03:34:14.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.574+0000 7f7805452700 1 -- 172.21.15.2:0/2706598129 wait complete. 2023-08-25T03:34:14.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.576+0000 7f7805452700 1 Processor -- start 2023-08-25T03:34:14.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.575+0000 7f3269ffb700 1 -- 172.21.15.2:0/545558814 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f326c003b70 con 0x7f3278070bb0 2023-08-25T03:34:14.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.575+0000 7f3269ffb700 1 --2- 172.21.15.2:0/545558814 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f325c06a1b0 0x7f325c06c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.576+0000 7f3269ffb700 1 -- 172.21.15.2:0/545558814 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f326c087b60 con 0x7f3278070bb0 2023-08-25T03:34:14.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.576+0000 7f327c992700 1 --2- 172.21.15.2:0/545558814 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f325c06a1b0 0x7f325c06c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.575+0000 7ff8c4510700 1 Processor -- start 2023-08-25T03:34:14.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.575+0000 7ff8c4510700 1 -- start start 2023-08-25T03:34:14.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.574+0000 7f8b7c4d1700 1 -- 172.21.15.2:0/1636942348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b74071030 msgr2=0x7f8b74070140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.574+0000 7f8b7c4d1700 1 --2- 172.21.15.2:0/1636942348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b74071030 0x7f8b74070140 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8b700098f0 tx=0x7f8b70009c00 comp rx=0 tx=0).stop 2023-08-25T03:34:14.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.575+0000 7f8b7c4d1700 1 -- 172.21.15.2:0/1636942348 shutdown_connections 2023-08-25T03:34:14.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.575+0000 7f8b7c4d1700 1 --2- 172.21.15.2:0/1636942348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b74071030 0x7f8b74070140 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.575+0000 7f8b7c4d1700 1 -- 172.21.15.2:0/1636942348 >> 172.21.15.2:0/1636942348 conn(0x7f8b740ceec0 msgr2=0x7f8b740cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:14.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.575+0000 7f8b7c4d1700 1 -- 172.21.15.2:0/1636942348 shutdown_connections 2023-08-25T03:34:14.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.575+0000 7f8b7c4d1700 1 -- 172.21.15.2:0/1636942348 wait complete. 2023-08-25T03:34:14.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.576+0000 7fe1e2ffd700 1 --2- 172.21.15.2:0/2668667352 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe1d006a1b0 0x7fe1d006c670 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe1e406eb80 tx=0x7fe1d400a230 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.576+0000 7f327c992700 1 --2- 172.21.15.2:0/545558814 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f325c06a1b0 0x7f325c06c670 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f32780c54f0 tx=0x7f3274001450 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.576+0000 7f7805452700 1 -- start start 2023-08-25T03:34:14.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.577+0000 7f8b7c4d1700 1 Processor -- start 2023-08-25T03:34:14.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.577+0000 7f8b7c4d1700 1 -- start start 2023-08-25T03:34:14.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.577+0000 7f7805452700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f780006b730 0x7f7800141e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.577+0000 7f7805452700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78000b43c0 0x7f780006a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.577+0000 7f7805452700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78001616d0 0x7f7800162750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.577+0000 7f7805452700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78001413a0 con 0x7f78000b43c0 2023-08-25T03:34:14.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.577+0000 7f7805452700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7800142370 con 0x7f780006b730 2023-08-25T03:34:14.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.577+0000 7f7805452700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78001407b0 con 0x7f78001616d0 2023-08-25T03:34:14.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.577+0000 7ff8c4510700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8bc0cdc50 0x7ff8bc148aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.577+0000 7ff8c4510700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8bc0ce030 0x7ff8bc0cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.577+0000 7ff8c4510700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8bc161450 0x7ff8bc15eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.577+0000 7ff8c4510700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc0cc3b0 con 0x7ff8bc161450 2023-08-25T03:34:14.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.577+0000 7ff8c4510700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc0cb730 con 0x7ff8bc0cdc50 2023-08-25T03:34:14.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.577+0000 7ff8c4510700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8bc0cba10 con 0x7ff8bc0ce030 2023-08-25T03:34:14.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.577+0000 7ff8c2aad700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8bc161450 0x7ff8bc15eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.577+0000 7ff8c2aad700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8bc161450 0x7ff8bc15eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59552/0 (socket says 172.21.15.2:59552) 2023-08-25T03:34:14.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.577+0000 7ff8c2aad700 1 -- 172.21.15.2:0/2798779635 learned_addr learned my addr 172.21.15.2:0/2798779635 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:14.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.577+0000 7ff8c1aab700 1 --2- 172.21.15.2:0/2798779635 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8bc0ce030 0x7ff8bc0cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.577+0000 7ff8c22ac700 1 --2- 172.21.15.2:0/2798779635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8bc0cdc50 0x7ff8bc148aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.577+0000 7f77feffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f780006b730 0x7f7800141e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.577+0000 7f77ff7fe700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78001616d0 0x7f7800162750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.577+0000 7f77feffd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f780006b730 0x7f7800141e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.2:50848/0 (socket says 172.21.15.2:50848) 2023-08-25T03:34:14.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.577+0000 7f77ff7fe700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78001616d0 0x7f7800162750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.2:59800/0 (socket says 172.21.15.2:59800) 2023-08-25T03:34:14.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.577+0000 7f77feffd700 1 -- 172.21.15.2:0/1679089923 learned_addr learned my addr 172.21.15.2:0/1679089923 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:14.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.577+0000 7f77fe7fc700 1 --2- 172.21.15.2:0/1679089923 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78000b43c0 0x7f780006a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.578+0000 7f77feffd700 1 -- 172.21.15.2:0/1679089923 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78001616d0 msgr2=0x7f7800162750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.578+0000 7f77feffd700 1 --2- 172.21.15.2:0/1679089923 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78001616d0 0x7f7800162750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.578+0000 7f77feffd700 1 -- 172.21.15.2:0/1679089923 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78000b43c0 msgr2=0x7f780006a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.578+0000 7f77feffd700 1 --2- 172.21.15.2:0/1679089923 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78000b43c0 0x7f780006a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.578+0000 7f77feffd700 1 -- 172.21.15.2:0/1679089923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f40094a0 con 0x7f780006b730 2023-08-25T03:34:14.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.578+0000 7f259dc09700 1 -- 172.21.15.2:0/1368098779 shutdown_connections 2023-08-25T03:34:14.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.578+0000 7f259dc09700 1 -- 172.21.15.2:0/1368098779 wait complete. 2023-08-25T03:34:14.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.578+0000 7f327f3f7700 1 -- 172.21.15.2:0/545558814 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32780b4dc0 con 0x7f3278070bb0 2023-08-25T03:34:14.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.578+0000 7f8b7c4d1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b74070940 0x7f8b74169280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.578+0000 7ff8c22ac700 1 -- 172.21.15.2:0/2798779635 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8bc0ce030 msgr2=0x7ff8bc0cae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.578+0000 7ff8c22ac700 1 --2- 172.21.15.2:0/2798779635 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8bc0ce030 0x7ff8bc0cae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.578+0000 7ff8c22ac700 1 -- 172.21.15.2:0/2798779635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8bc161450 msgr2=0x7ff8bc15eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.578+0000 7ff8c22ac700 1 --2- 172.21.15.2:0/2798779635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8bc161450 0x7ff8bc15eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.578+0000 7ff8c22ac700 1 -- 172.21.15.2:0/2798779635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8b8009550 con 0x7ff8bc0cdc50 2023-08-25T03:34:14.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.578+0000 7f8b7c4d1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b74071030 0x7f8b7409ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.578+0000 7f8b7c4d1700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8b7408a8f0 0x7f8b7409bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.578+0000 7f8b7c4d1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7409c9f0 con 0x7f8b74070940 2023-08-25T03:34:14.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.578+0000 7f8b7c4d1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7409b610 con 0x7f8b74071030 2023-08-25T03:34:14.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.578+0000 7f8b7c4d1700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7409aa90 con 0x7f8b7408a8f0 2023-08-25T03:34:14.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.578+0000 7f77feffd700 1 --2- 172.21.15.2:0/1679089923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f780006b730 0x7f7800141e30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f77f000c480 tx=0x7f77f000c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.578+0000 7ff8c22ac700 1 --2- 172.21.15.2:0/2798779635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8bc0cdc50 0x7ff8bc148aa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac00b410 tx=0x7ff8ac00b720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.578+0000 7f8b7a26d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b74070940 0x7f8b74169280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.578+0000 7f8b7a26d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b74070940 0x7f8b74169280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59560/0 (socket says 172.21.15.2:59560) 2023-08-25T03:34:14.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.578+0000 7f8b7a26d700 1 -- 172.21.15.2:0/1640728115 learned_addr learned my addr 172.21.15.2:0/1640728115 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:14.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.578+0000 7f8b7aa6e700 1 --2- 172.21.15.2:0/1640728115 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8b7408a8f0 0x7f8b7409bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.578+0000 7f8b79a6c700 1 --2- 172.21.15.2:0/1640728115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b74071030 0x7f8b7409ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.579+0000 7f8b7a26d700 1 -- 172.21.15.2:0/1640728115 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8b7408a8f0 msgr2=0x7f8b7409bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.579+0000 7f8b7a26d700 1 --2- 172.21.15.2:0/1640728115 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8b7408a8f0 0x7f8b7409bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.579+0000 7f8b7a26d700 1 -- 172.21.15.2:0/1640728115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b74071030 msgr2=0x7f8b7409ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.579+0000 7f8b7a26d700 1 --2- 172.21.15.2:0/1640728115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b74071030 0x7f8b7409ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.579+0000 7f8b7a26d700 1 -- 172.21.15.2:0/1640728115 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b70009550 con 0x7f8b74070940 2023-08-25T03:34:14.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.579+0000 7f8b79a6c700 1 --2- 172.21.15.2:0/1640728115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b74071030 0x7f8b7409ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:34:14.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.579+0000 7f8b7a26d700 1 --2- 172.21.15.2:0/1640728115 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b74070940 0x7f8b74169280 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8b740cfc30 tx=0x7f8b70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.579+0000 7ff8b2ffd700 1 -- 172.21.15.2:0/2798779635 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac011620 con 0x7ff8bc0cdc50 2023-08-25T03:34:14.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.579+0000 7ff8c4510700 1 -- 172.21.15.2:0/2798779635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8bc0cbcf0 con 0x7ff8bc0cdc50 2023-08-25T03:34:14.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.579+0000 7ff8c4510700 1 -- 172.21.15.2:0/2798779635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8bc0cb370 con 0x7ff8bc0cdc50 2023-08-25T03:34:14.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.579+0000 7f8b6affd700 1 -- 172.21.15.2:0/1640728115 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7000b660 con 0x7f8b74070940 2023-08-25T03:34:14.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.579+0000 7f8b6affd700 1 -- 172.21.15.2:0/1640728115 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7000b7c0 con 0x7f8b74070940 2023-08-25T03:34:14.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.579+0000 7f8b7c4d1700 1 -- 172.21.15.2:0/1640728115 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b7409ad70 con 0x7f8b74070940 2023-08-25T03:34:14.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.579+0000 7f8b6affd700 1 -- 172.21.15.2:0/1640728115 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7000bb70 con 0x7f8b74070940 2023-08-25T03:34:14.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.579+0000 7f8b7c4d1700 1 -- 172.21.15.2:0/1640728115 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b7409b330 con 0x7f8b74070940 2023-08-25T03:34:14.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.580+0000 7f259dc09700 1 Processor -- start 2023-08-25T03:34:14.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.580+0000 7f259dc09700 1 -- start start 2023-08-25T03:34:14.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.579+0000 7ff8b2ffd700 1 -- 172.21.15.2:0/2798779635 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ac011780 con 0x7ff8bc0cdc50 2023-08-25T03:34:14.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.579+0000 7ff8b2ffd700 1 -- 172.21.15.2:0/2798779635 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ac011b30 con 0x7ff8bc0cdc50 2023-08-25T03:34:14.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.580+0000 7ff8c4510700 1 -- 172.21.15.2:0/2798779635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a40052f0 con 0x7ff8bc0cdc50 2023-08-25T03:34:14.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.581+0000 7fe1cbfff700 1 -- 172.21.15.2:0/2668667352 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe1d8015080 con 0x7fe1e40cad70 2023-08-25T03:34:14.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.581+0000 7f259dc09700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25980b20b0 0x7f25980b0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.581+0000 7f259dc09700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f259815bd80 0x7f2598149a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.581+0000 7f259dc09700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2598161d50 0x7f2598162e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.581+0000 7f259dc09700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25980b0ed0 con 0x7f2598161d50 2023-08-25T03:34:14.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.581+0000 7f259dc09700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25980b1150 con 0x7f25980b20b0 2023-08-25T03:34:14.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.581+0000 7f259dc09700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25980b1430 con 0x7f259815bd80 2023-08-25T03:34:14.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.581+0000 7f8b6affd700 1 -- 172.21.15.2:0/1640728115 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8b70022070 con 0x7f8b74070940 2023-08-25T03:34:14.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.582+0000 7f25977fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25980b20b0 0x7f25980b0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.582+0000 7f25977fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25980b20b0 0x7f25980b0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.2:50856/0 (socket says 172.21.15.2:50856) 2023-08-25T03:34:14.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.582+0000 7f25977fe700 1 -- 172.21.15.2:0/2458596778 learned_addr learned my addr 172.21.15.2:0/2458596778 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:14.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.582+0000 7f2596ffd700 1 --2- 172.21.15.2:0/2458596778 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f259815bd80 0x7f2598149a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.582+0000 7f2597fff700 1 --2- 172.21.15.2:0/2458596778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2598161d50 0x7f2598162e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.582+0000 7f25977fe700 1 -- 172.21.15.2:0/2458596778 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f259815bd80 msgr2=0x7f2598149a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.582+0000 7f25977fe700 1 --2- 172.21.15.2:0/2458596778 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f259815bd80 0x7f2598149a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.582+0000 7f25977fe700 1 -- 172.21.15.2:0/2458596778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2598161d50 msgr2=0x7f2598162e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.582+0000 7f25977fe700 1 --2- 172.21.15.2:0/2458596778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2598161d50 0x7f2598162e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.582+0000 7ff8b2ffd700 1 -- 172.21.15.2:0/2798779635 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff8ac011c90 con 0x7ff8bc0cdc50 2023-08-25T03:34:14.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.582+0000 7f25977fe700 1 -- 172.21.15.2:0/2458596778 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2590009b30 con 0x7f25980b20b0 2023-08-25T03:34:14.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.582+0000 7f25977fe700 1 --2- 172.21.15.2:0/2458596778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25980b20b0 0x7f25980b0990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f258800c4d0 tx=0x7f258800c7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.582+0000 7f8b6affd700 1 --2- 172.21.15.2:0/1640728115 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8b5806a0e0 0x7f8b5806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.582+0000 7f8b6affd700 1 -- 172.21.15.2:0/1640728115 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8b7008a240 con 0x7f8b74070940 2023-08-25T03:34:14.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.583+0000 7ff8b2ffd700 1 --2- 172.21.15.2:0/2798779635 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff8a006a0e0 0x7ff8a006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.583+0000 7f77e77fe700 1 -- 172.21.15.2:0/1679089923 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f0005d70 con 0x7f780006b730 2023-08-25T03:34:14.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.583+0000 7f77e77fe700 1 -- 172.21.15.2:0/1679089923 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f000ce90 con 0x7f780006b730 2023-08-25T03:34:14.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.583+0000 7f77e77fe700 1 -- 172.21.15.2:0/1679089923 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f00072d0 con 0x7f780006b730 2023-08-25T03:34:14.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.583+0000 7ff8b2ffd700 1 -- 172.21.15.2:0/2798779635 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff8ac088390 con 0x7ff8bc0cdc50 2023-08-25T03:34:14.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.583+0000 7ff8c1aab700 1 --2- 172.21.15.2:0/2798779635 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff8a006a0e0 0x7ff8a006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.583+0000 7ff8c1aab700 1 --2- 172.21.15.2:0/2798779635 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff8a006a0e0 0x7ff8a006c5a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009c10 tx=0x7ff8b801d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.583+0000 7f3269ffb700 1 -- 172.21.15.2:0/545558814 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f326c017080 con 0x7f3278070bb0 2023-08-25T03:34:14.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.584+0000 7f7805452700 1 -- 172.21.15.2:0/1679089923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7800140a70 con 0x7f780006b730 2023-08-25T03:34:14.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.584+0000 7f7805452700 1 -- 172.21.15.2:0/1679089923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7800141030 con 0x7f780006b730 2023-08-25T03:34:14.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.585+0000 7f8b79a6c700 1 --2- 172.21.15.2:0/1640728115 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8b5806a0e0 0x7f8b5806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.585+0000 7f8b7c4d1700 1 -- 172.21.15.2:0/1640728115 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b5c0052f0 con 0x7f8b74070940 2023-08-25T03:34:14.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.585+0000 7f8b79a6c700 1 --2- 172.21.15.2:0/1640728115 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8b5806a0e0 0x7f8b5806c5a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8b740ad130 tx=0x7f8b64007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.587+0000 7f2587fff700 1 -- 172.21.15.2:0/2458596778 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2588011790 con 0x7f25980b20b0 2023-08-25T03:34:14.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.588+0000 7f259dc09700 1 -- 172.21.15.2:0/2458596778 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25980b1710 con 0x7f25980b20b0 2023-08-25T03:34:14.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.588+0000 7f7805452700 1 -- 172.21.15.2:0/1679089923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78000746d0 con 0x7f780006b730 2023-08-25T03:34:14.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.588+0000 7f259dc09700 1 -- 172.21.15.2:0/2458596778 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25980c6ee0 con 0x7f25980b20b0 2023-08-25T03:34:14.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.588+0000 7f2587fff700 1 -- 172.21.15.2:0/2458596778 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25880039b0 con 0x7f25980b20b0 2023-08-25T03:34:14.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.588+0000 7f2587fff700 1 -- 172.21.15.2:0/2458596778 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2588011af0 con 0x7f25980b20b0 2023-08-25T03:34:14.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.588+0000 7ff8b2ffd700 1 -- 172.21.15.2:0/2798779635 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff8ac053a80 con 0x7ff8bc0cdc50 2023-08-25T03:34:14.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.589+0000 7f77e77fe700 1 -- 172.21.15.2:0/1679089923 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f77f0007430 con 0x7f780006b730 2023-08-25T03:34:14.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.590+0000 7f77e77fe700 1 --2- 172.21.15.2:0/1679089923 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f77ec06a160 0x7f77ec06c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.590+0000 7f77e77fe700 1 -- 172.21.15.2:0/1679089923 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f77f0087c20 con 0x7f780006b730 2023-08-25T03:34:14.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.594+0000 7f77e77fe700 1 -- 172.21.15.2:0/1679089923 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f77f0053420 con 0x7f780006b730 2023-08-25T03:34:14.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.596+0000 7f8b6affd700 1 -- 172.21.15.2:0/1640728115 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8b700558a0 con 0x7f8b74070940 2023-08-25T03:34:14.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.596+0000 7f2587fff700 1 -- 172.21.15.2:0/2458596778 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2588011c50 con 0x7f25980b20b0 2023-08-25T03:34:14.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.596+0000 7f2587fff700 1 --2- 172.21.15.2:0/2458596778 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f257c06a0b0 0x7f257c06c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.596+0000 7f2587fff700 1 -- 172.21.15.2:0/2458596778 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2588088250 con 0x7f25980b20b0 2023-08-25T03:34:14.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.597+0000 7f259dc09700 1 -- 172.21.15.2:0/2458596778 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25740052f0 con 0x7f25980b20b0 2023-08-25T03:34:14.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.597+0000 7f77fe7fc700 1 --2- 172.21.15.2:0/1679089923 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f77ec06a160 0x7f77ec06c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.597+0000 7f2596ffd700 1 --2- 172.21.15.2:0/2458596778 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f257c06a0b0 0x7f257c06c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.598+0000 7f77fe7fc700 1 --2- 172.21.15.2:0/1679089923 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f77ec06a160 0x7f77ec06c620 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f77f40090b0 tx=0x7f77f4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.598+0000 7f2596ffd700 1 --2- 172.21.15.2:0/2458596778 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f257c06a0b0 0x7f257c06c570 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2598070ef0 tx=0x7f2590009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.602+0000 7f9f461bb700 1 Processor -- start 2023-08-25T03:34:14.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.602+0000 7f9f461bb700 1 -- start start 2023-08-25T03:34:14.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.603+0000 7f9f461bb700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9f401570b0 0x7f9f400702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.603+0000 7f9f461bb700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f40157490 con 0x7f9f4015d750 2023-08-25T03:34:14.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.603+0000 7f9f461bb700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f40070c40 con 0x7f9f400cad70 2023-08-25T03:34:14.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.603+0000 7f9f461bb700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f400717c0 con 0x7f9f401570b0 2023-08-25T03:34:14.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.604+0000 7f9f3f7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9f401570b0 0x7f9f400702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.606+0000 7f2587fff700 1 -- 172.21.15.2:0/2458596778 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2588053830 con 0x7f25980b20b0 2023-08-25T03:34:14.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.606+0000 7f9f3effd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f9f400cad70 0x7f9f4015d040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.2:54272/0 (socket says 172.21.15.2:54272) 2023-08-25T03:34:14.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.606+0000 7f9f3effd700 1 -- 172.21.15.2:0/228792027 learned_addr learned my addr 172.21.15.2:0/228792027 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:14.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.606+0000 7f9f3e7fc700 1 -- 172.21.15.2:0/228792027 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4224875278 0 0) 0x7f9f40070c40 con 0x7f9f400cad70 2023-08-25T03:34:14.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.606+0000 7f9f3e7fc700 1 -- 172.21.15.2:0/228792027 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f280035b0 con 0x7f9f400cad70 2023-08-25T03:34:14.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.606+0000 7f9f3e7fc700 1 -- 172.21.15.2:0/228792027 <== mon.1 v1:172.21.15.31:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1026213372 0 0) 0x7f9f280035b0 con 0x7f9f400cad70 2023-08-25T03:34:14.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.606+0000 7f9f3e7fc700 1 -- 172.21.15.2:0/228792027 >> v2:172.21.15.31:3300/0 conn(0x7f9f401570b0 msgr2=0x7f9f400702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:14.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.606+0000 7f9f3e7fc700 1 --2- 172.21.15.2:0/228792027 >> v2:172.21.15.31:3300/0 conn(0x7f9f401570b0 0x7f9f400702d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.606+0000 7f9f3e7fc700 1 -- 172.21.15.2:0/228792027 >> v1:172.21.15.2:6789/0 conn(0x7f9f4015d750 legacy=0x7f9f400ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:14.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.607+0000 7f9f3e7fc700 1 -- 172.21.15.2:0/228792027 --> v1:172.21.15.31:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f40070f20 con 0x7f9f400cad70 2023-08-25T03:34:14.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.607+0000 7f9f3e7fc700 1 -- 172.21.15.2:0/228792027 <== mon.1 v1:172.21.15.31:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f9f300016b0 con 0x7f9f400cad70 2023-08-25T03:34:14.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.607+0000 7f9f3e7fc700 1 -- 172.21.15.2:0/228792027 >> v1:172.21.15.31:6789/0 conn(0x7f9f400cad70 legacy=0x7f9f4015d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.607+0000 7f9f3e7fc700 1 --2- 172.21.15.2:0/228792027 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9f28003a00 0x7f9f28003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.607+0000 7f9f3e7fc700 1 --2- 172.21.15.2:0/228792027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f28004850 0x7f9f28004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.607+0000 7f9f3e7fc700 1 --2- 172.21.15.2:0/228792027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f28005330 0x7f9f280059e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.607+0000 7f9f3e7fc700 1 -- 172.21.15.2:0/228792027 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f28006290 con 0x7f9f28004850 2023-08-25T03:34:14.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.607+0000 7f9f3e7fc700 1 -- 172.21.15.2:0/228792027 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f28006400 con 0x7f9f28005330 2023-08-25T03:34:14.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.607+0000 7f9f3e7fc700 1 -- 172.21.15.2:0/228792027 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f280065c0 con 0x7f9f28003a00 2023-08-25T03:34:14.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.607+0000 7f9f3effd700 1 --2- 172.21.15.2:0/228792027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f28004850 0x7f9f28004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.607+0000 7f9f3ffff700 1 --2- 172.21.15.2:0/228792027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f28005330 0x7f9f280059e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.607+0000 7f9f3effd700 1 -- 172.21.15.2:0/228792027 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9f28003a00 msgr2=0x7f9f28003e00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-08-25T03:34:14.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.607+0000 7f9f3effd700 1 --2- 172.21.15.2:0/228792027 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9f28003a00 0x7f9f28003e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.607+0000 7f9f3effd700 1 -- 172.21.15.2:0/228792027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f28005330 msgr2=0x7f9f280059e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:14.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.607+0000 7f9f3effd700 1 --2- 172.21.15.2:0/228792027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f28005330 0x7f9f280059e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.607+0000 7f9f3effd700 1 -- 172.21.15.2:0/228792027 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f280035b0 con 0x7f9f28004850 2023-08-25T03:34:14.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.607+0000 7f9f3ffff700 1 -- 172.21.15.2:0/228792027 reap_dead start 2023-08-25T03:34:14.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.608+0000 7f9f3effd700 1 --2- 172.21.15.2:0/228792027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f28004850 0x7f9f28004cd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000c320 tx=0x7f9f28005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60a903466db0289 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.608+0000 7f9f3e7fc700 1 -- 172.21.15.2:0/228792027 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3000ce60 con 0x7f9f28004850 2023-08-25T03:34:14.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.608+0000 7f9f3e7fc700 1 -- 172.21.15.2:0/228792027 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f30005960 con 0x7f9f28004850 2023-08-25T03:34:14.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.608+0000 7f9f3e7fc700 1 -- 172.21.15.2:0/228792027 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f30005cf0 con 0x7f9f28004850 2023-08-25T03:34:14.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.609+0000 7f9f461bb700 1 -- 172.21.15.2:0/228792027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f28004850 msgr2=0x7f9f28004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.609+0000 7f9f461bb700 1 --2- 172.21.15.2:0/228792027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f28004850 0x7f9f28004cd0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000c320 tx=0x7f9f28005880 comp rx=0 tx=0).stop 2023-08-25T03:34:14.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.609+0000 7f9f461bb700 1 -- 172.21.15.2:0/228792027 shutdown_connections 2023-08-25T03:34:14.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.609+0000 7f9f461bb700 1 --2- 172.21.15.2:0/228792027 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f28004850 0x7f9f28004cd0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.609+0000 7f9f461bb700 1 -- 172.21.15.2:0/228792027 >> 172.21.15.2:0/228792027 conn(0x7f9f400ceec0 msgr2=0x7f9f400cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:14.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.614+0000 7f9f461bb700 1 -- 172.21.15.2:0/228792027 shutdown_connections 2023-08-25T03:34:14.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.614+0000 7f9f461bb700 1 -- 172.21.15.2:0/228792027 wait complete. 2023-08-25T03:34:14.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.616+0000 7f9f461bb700 1 Processor -- start 2023-08-25T03:34:14.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.617+0000 7f9f461bb700 1 -- start start 2023-08-25T03:34:14.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.619+0000 7f9f461bb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f28004850 0x7f9f401f54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.619+0000 7f9f461bb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f401f5b70 0x7f9f4006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.619+0000 7f9f461bb700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9f400b1ae0 0x7f9f400b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.619+0000 7f9f461bb700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f400b24a0 con 0x7f9f401f5b70 2023-08-25T03:34:14.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.619+0000 7f9f461bb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f400af490 con 0x7f9f28004850 2023-08-25T03:34:14.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.619+0000 7f9f461bb700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4006e440 con 0x7f9f400b1ae0 2023-08-25T03:34:14.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.619+0000 7f9f3effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f401f5b70 0x7f9f4006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.619+0000 7f9f3effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f401f5b70 0x7f9f4006d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59590/0 (socket says 172.21.15.2:59590) 2023-08-25T03:34:14.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.619+0000 7f9f3effd700 1 -- 172.21.15.2:0/1520373568 learned_addr learned my addr 172.21.15.2:0/1520373568 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:14.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.619+0000 7f9f3effd700 1 -- 172.21.15.2:0/1520373568 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9f400b1ae0 msgr2=0x7f9f400b1f60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-25T03:34:14.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.619+0000 7f9f3effd700 1 --2- 172.21.15.2:0/1520373568 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9f400b1ae0 0x7f9f400b1f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.619+0000 7f9f3effd700 1 -- 172.21.15.2:0/1520373568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f28004850 msgr2=0x7f9f401f54b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-25T03:34:14.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.619+0000 7f9f3effd700 1 --2- 172.21.15.2:0/1520373568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f28004850 0x7f9f401f54b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.619+0000 7f9f3effd700 1 -- 172.21.15.2:0/1520373568 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f4006de80 con 0x7f9f401f5b70 2023-08-25T03:34:14.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.620+0000 7f9f3effd700 1 --2- 172.21.15.2:0/1520373568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f401f5b70 0x7f9f4006d530 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000c910 tx=0x7f9f300019f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.620+0000 7f9f27fff700 1 -- 172.21.15.2:0/1520373568 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f30001c10 con 0x7f9f401f5b70 2023-08-25T03:34:14.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.620+0000 7f9f27fff700 1 -- 172.21.15.2:0/1520373568 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f30004a80 con 0x7f9f401f5b70 2023-08-25T03:34:14.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.620+0000 7f9f27fff700 1 -- 172.21.15.2:0/1520373568 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f30004e10 con 0x7f9f401f5b70 2023-08-25T03:34:14.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.622+0000 7f9f461bb700 1 -- 172.21.15.2:0/1520373568 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f4006e160 con 0x7f9f401f5b70 2023-08-25T03:34:14.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.622+0000 7f9f461bb700 1 -- 172.21.15.2:0/1520373568 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f4006da70 con 0x7f9f401f5b70 2023-08-25T03:34:14.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.627+0000 7f9f461bb700 1 -- 172.21.15.2:0/1520373568 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f400c9c80 con 0x7f9f401f5b70 2023-08-25T03:34:14.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.639+0000 7f9f27fff700 1 -- 172.21.15.2:0/1520373568 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9f30007070 con 0x7f9f401f5b70 2023-08-25T03:34:14.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.640+0000 7f9f27fff700 1 --2- 172.21.15.2:0/1520373568 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9f2c06a1e0 0x7f9f2c06c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:14.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.640+0000 7f9f27fff700 1 -- 172.21.15.2:0/1520373568 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9f3008a0c0 con 0x7f9f401f5b70 2023-08-25T03:34:14.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.640+0000 7f9f27fff700 1 -- 172.21.15.2:0/1520373568 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9f3008a540 con 0x7f9f401f5b70 2023-08-25T03:34:14.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.643+0000 7f9f3f7fe700 1 --2- 172.21.15.2:0/1520373568 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9f2c06a1e0 0x7f9f2c06c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:14.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.644+0000 7f9f3f7fe700 1 --2- 172.21.15.2:0/1520373568 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9f2c06a1e0 0x7f9f2c06c6a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9f4004ffc0 tx=0x7f9f38007310 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:14.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.980+0000 7f9ba37c6700 1 -- 172.21.15.2:0/190487447 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9b9c0ce850 con 0x7f9b9c0c1e90 2023-08-25T03:34:14.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.980+0000 7f9b8dffb700 1 -- 172.21.15.2:0/190487447 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9b9002b020 con 0x7f9b9c0c1e90 2023-08-25T03:34:14.983 INFO:teuthology.orchestra.run.smithi002.stdout:55834574855 2023-08-25T03:34:14.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.982+0000 7f327f3f7700 1 -- 172.21.15.2:0/545558814 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f32780c9820 con 0x7f3278070bb0 2023-08-25T03:34:14.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.982+0000 7f3269ffb700 1 -- 172.21.15.2:0/545558814 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f326c056d50 con 0x7f3278070bb0 2023-08-25T03:34:14.984 INFO:teuthology.orchestra.run.smithi002.stdout:60129542152 2023-08-25T03:34:14.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.984+0000 7f9ba37c6700 1 -- 172.21.15.2:0/190487447 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9b8006a0b0 msgr2=0x7f9b8006c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.984+0000 7f9ba37c6700 1 --2- 172.21.15.2:0/190487447 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9b8006a0b0 0x7f9b8006c570 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0b93a0 tx=0x7f9b98008040 comp rx=0 tx=0).stop 2023-08-25T03:34:14.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.984+0000 7f9ba37c6700 1 -- 172.21.15.2:0/190487447 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9b9c0c1e90 msgr2=0x7f9b9c0b2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.984+0000 7f9ba37c6700 1 --2- 172.21.15.2:0/190487447 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9b9c0c1e90 0x7f9b9c0b2b30 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9b90013000 tx=0x7f9b90004750 comp rx=0 tx=0).stop 2023-08-25T03:34:14.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.984+0000 7f9ba37c6700 1 -- 172.21.15.2:0/190487447 shutdown_connections 2023-08-25T03:34:14.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.984+0000 7f9ba37c6700 1 --2- 172.21.15.2:0/190487447 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9b9c06bdc0 0x7f9b9c06de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.984+0000 7f9ba37c6700 1 --2- 172.21.15.2:0/190487447 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9b9c0c1e90 0x7f9b9c0b2b30 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.984+0000 7f9ba37c6700 1 --2- 172.21.15.2:0/190487447 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9b8006a0b0 0x7f9b8006c570 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.984+0000 7f9ba37c6700 1 --2- 172.21.15.2:0/190487447 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9b9c161a60 0x7f9b9c15dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.985+0000 7f9ba37c6700 1 -- 172.21.15.2:0/190487447 >> 172.21.15.2:0/190487447 conn(0x7f9b9c0ceec0 msgr2=0x7f9b9c0ccee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:14.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.985+0000 7f9ba37c6700 1 -- 172.21.15.2:0/190487447 shutdown_connections 2023-08-25T03:34:14.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.985+0000 7f9ba37c6700 1 -- 172.21.15.2:0/190487447 wait complete. 2023-08-25T03:34:14.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.986+0000 7f327f3f7700 1 -- 172.21.15.2:0/545558814 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f325c06a1b0 msgr2=0x7f325c06c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.986+0000 7f327f3f7700 1 --2- 172.21.15.2:0/545558814 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f325c06a1b0 0x7f325c06c670 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f32780c54f0 tx=0x7f3274001450 comp rx=0 tx=0).stop 2023-08-25T03:34:14.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.986+0000 7f327f3f7700 1 -- 172.21.15.2:0/545558814 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3278070bb0 msgr2=0x7f32780c6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:14.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.986+0000 7f327f3f7700 1 --2- 172.21.15.2:0/545558814 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3278070bb0 0x7f32780c6920 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f326c00c780 tx=0x7f326c00cb40 comp rx=0 tx=0).stop 2023-08-25T03:34:14.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.987+0000 7f327f3f7700 1 -- 172.21.15.2:0/545558814 shutdown_connections 2023-08-25T03:34:14.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.987+0000 7f327f3f7700 1 --2- 172.21.15.2:0/545558814 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f325c06a1b0 0x7f325c06c670 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f32780c54f0 tx=0x7f3274001450 comp rx=0 tx=0).stop 2023-08-25T03:34:14.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.987+0000 7f327f3f7700 1 --2- 172.21.15.2:0/545558814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f32780cdfc0 0x7f327820ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.987+0000 7f327f3f7700 1 --2- 172.21.15.2:0/545558814 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3278070bb0 0x7f32780c6920 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.987+0000 7f327f3f7700 1 --2- 172.21.15.2:0/545558814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3278070f90 0x7f32780b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:14.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.987+0000 7f327f3f7700 1 -- 172.21.15.2:0/545558814 >> 172.21.15.2:0/545558814 conn(0x7f32780ceec0 msgr2=0x7f327816acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:14.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.988+0000 7f327f3f7700 1 -- 172.21.15.2:0/545558814 shutdown_connections 2023-08-25T03:34:14.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:14.988+0000 7f327f3f7700 1 -- 172.21.15.2:0/545558814 wait complete. 2023-08-25T03:34:15.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.002+0000 7fe1e9b07700 1 -- 172.21.15.2:0/2668667352 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe1c40055a0 con 0x7fe1e40cad70 2023-08-25T03:34:15.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.003+0000 7fe1cbfff700 1 -- 172.21.15.2:0/2668667352 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe1d8057660 con 0x7fe1e40cad70 2023-08-25T03:34:15.006 INFO:teuthology.orchestra.run.smithi002.stdout:55834574855 2023-08-25T03:34:15.007 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.1 2023-08-25T03:34:15.009 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542152 for osd.6 2023-08-25T03:34:15.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.008+0000 7fe1e9b07700 1 -- 172.21.15.2:0/2668667352 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe1d006a1b0 msgr2=0x7fe1d006c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:15.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.008+0000 7fe1e9b07700 1 --2- 172.21.15.2:0/2668667352 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe1d006a1b0 0x7fe1d006c670 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe1e406eb80 tx=0x7fe1d400a230 comp rx=0 tx=0).stop 2023-08-25T03:34:15.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.008+0000 7fe1e9b07700 1 -- 172.21.15.2:0/2668667352 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe1e40cad70 msgr2=0x7fe1e4151000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:15.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.008+0000 7fe1e9b07700 1 --2- 172.21.15.2:0/2668667352 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe1e40cad70 0x7fe1e4151000 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe1d800b580 tx=0x7fe1d800b890 comp rx=0 tx=0).stop 2023-08-25T03:34:15.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.008+0000 7fe1e9b07700 1 -- 172.21.15.2:0/2668667352 shutdown_connections 2023-08-25T03:34:15.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.008+0000 7fe1e9b07700 1 --2- 172.21.15.2:0/2668667352 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe1d006a1b0 0x7fe1d006c670 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.008+0000 7fe1e9b07700 1 --2- 172.21.15.2:0/2668667352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe1e415d750 0x7fe1e409b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.008+0000 7fe1e9b07700 1 --2- 172.21.15.2:0/2668667352 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe1e40cad70 0x7fe1e4151000 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.008+0000 7fe1e9b07700 1 --2- 172.21.15.2:0/2668667352 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1e41570b0 0x7fe1e41f5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.008+0000 7fe1e9b07700 1 -- 172.21.15.2:0/2668667352 >> 172.21.15.2:0/2668667352 conn(0x7fe1e40ceec0 msgr2=0x7fe1e40acfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:15.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.008+0000 7fe1e9b07700 1 -- 172.21.15.2:0/2668667352 shutdown_connections 2023-08-25T03:34:15.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.008+0000 7fe1e9b07700 1 -- 172.21.15.2:0/2668667352 wait complete. 2023-08-25T03:34:15.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.008+0000 7ff8c4510700 1 -- 172.21.15.2:0/2798779635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7ff8a4005870 con 0x7ff8bc0cdc50 2023-08-25T03:34:15.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.009+0000 7ff8b2ffd700 1 -- 172.21.15.2:0/2798779635 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff8ac0570a0 con 0x7ff8bc0cdc50 2023-08-25T03:34:15.014 INFO:teuthology.orchestra.run.smithi002.stdout:55834574854 2023-08-25T03:34:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.015+0000 7ff8c4510700 1 -- 172.21.15.2:0/2798779635 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff8a006a0e0 msgr2=0x7ff8a006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.015+0000 7ff8c4510700 1 --2- 172.21.15.2:0/2798779635 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff8a006a0e0 0x7ff8a006c5a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff8b8009c10 tx=0x7ff8b801d040 comp rx=0 tx=0).stop 2023-08-25T03:34:15.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.015+0000 7ff8c4510700 1 -- 172.21.15.2:0/2798779635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8bc0cdc50 msgr2=0x7ff8bc148aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:15.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.015+0000 7ff8c4510700 1 --2- 172.21.15.2:0/2798779635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8bc0cdc50 0x7ff8bc148aa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac00b410 tx=0x7ff8ac00b720 comp rx=0 tx=0).stop 2023-08-25T03:34:15.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.014+0000 7f8b7c4d1700 1 -- 172.21.15.2:0/1640728115 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f8b5c0055a0 con 0x7f8b74070940 2023-08-25T03:34:15.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.015+0000 7ff8c4510700 1 -- 172.21.15.2:0/2798779635 shutdown_connections 2023-08-25T03:34:15.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.015+0000 7ff8c4510700 1 --2- 172.21.15.2:0/2798779635 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff8a006a0e0 0x7ff8a006c5a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.015+0000 7ff8c4510700 1 --2- 172.21.15.2:0/2798779635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff8bc161450 0x7ff8bc15eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.015+0000 7ff8c4510700 1 --2- 172.21.15.2:0/2798779635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8bc0cdc50 0x7ff8bc148aa0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.015+0000 7ff8c4510700 1 --2- 172.21.15.2:0/2798779635 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8bc0ce030 0x7ff8bc0cae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.015+0000 7ff8c4510700 1 -- 172.21.15.2:0/2798779635 >> 172.21.15.2:0/2798779635 conn(0x7ff8bc0cfea0 msgr2=0x7ff8bc157330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:15.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.015+0000 7ff8c4510700 1 -- 172.21.15.2:0/2798779635 shutdown_connections 2023-08-25T03:34:15.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.015+0000 7ff8c4510700 1 -- 172.21.15.2:0/2798779635 wait complete. 2023-08-25T03:34:15.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.015+0000 7f8b6affd700 1 -- 172.21.15.2:0/1640728115 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8b70058ec0 con 0x7f8b74070940 2023-08-25T03:34:15.018 INFO:teuthology.orchestra.run.smithi002.stdout:60129542152 2023-08-25T03:34:15.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.017+0000 7f7805452700 1 -- 172.21.15.2:0/1679089923 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f780006e8e0 con 0x7f780006b730 2023-08-25T03:34:15.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.018+0000 7f77e77fe700 1 -- 172.21.15.2:0/1679089923 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f77f0021020 con 0x7f780006b730 2023-08-25T03:34:15.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.018+0000 7f8b7c4d1700 1 -- 172.21.15.2:0/1640728115 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8b5806a0e0 msgr2=0x7f8b5806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:15.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.018+0000 7f8b7c4d1700 1 --2- 172.21.15.2:0/1640728115 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8b5806a0e0 0x7f8b5806c5a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8b740ad130 tx=0x7f8b64007ec0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.018+0000 7f8b7c4d1700 1 -- 172.21.15.2:0/1640728115 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b74070940 msgr2=0x7f8b74169280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:15.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.018+0000 7f8b7c4d1700 1 --2- 172.21.15.2:0/1640728115 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b74070940 0x7f8b74169280 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8b740cfc30 tx=0x7f8b70004750 comp rx=0 tx=0).stop 2023-08-25T03:34:15.020 INFO:teuthology.orchestra.run.smithi002.stdout:60129542151 2023-08-25T03:34:15.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.018+0000 7f8b7c4d1700 1 -- 172.21.15.2:0/1640728115 shutdown_connections 2023-08-25T03:34:15.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.018+0000 7f8b7c4d1700 1 --2- 172.21.15.2:0/1640728115 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8b5806a0e0 0x7f8b5806c5a0 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8b740ad130 tx=0x7f8b64007ec0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.018+0000 7f8b7c4d1700 1 --2- 172.21.15.2:0/1640728115 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8b74070940 0x7f8b74169280 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.018+0000 7f8b7c4d1700 1 --2- 172.21.15.2:0/1640728115 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8b7408a8f0 0x7f8b7409bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.018+0000 7f8b7c4d1700 1 --2- 172.21.15.2:0/1640728115 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8b74071030 0x7f8b7409ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.018+0000 7f8b7c4d1700 1 -- 172.21.15.2:0/1640728115 >> 172.21.15.2:0/1640728115 conn(0x7f8b740ceec0 msgr2=0x7f8b7416d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:15.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.018+0000 7f259dc09700 1 -- 172.21.15.2:0/2458596778 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f25740055a0 con 0x7f25980b20b0 2023-08-25T03:34:15.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.019+0000 7f8b7c4d1700 1 -- 172.21.15.2:0/1640728115 shutdown_connections 2023-08-25T03:34:15.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.019+0000 7f8b7c4d1700 1 -- 172.21.15.2:0/1640728115 wait complete. 2023-08-25T03:34:15.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.019+0000 7f2587fff700 1 -- 172.21.15.2:0/2458596778 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2588056e50 con 0x7f25980b20b0 2023-08-25T03:34:15.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.021+0000 7f7805452700 1 -- 172.21.15.2:0/1679089923 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f77ec06a160 msgr2=0x7f77ec06c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:15.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.021+0000 7f7805452700 1 --2- 172.21.15.2:0/1679089923 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f77ec06a160 0x7f77ec06c620 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f77f40090b0 tx=0x7f77f4008030 comp rx=0 tx=0).stop 2023-08-25T03:34:15.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.021+0000 7f7805452700 1 -- 172.21.15.2:0/1679089923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f780006b730 msgr2=0x7f7800141e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:15.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.021+0000 7f7805452700 1 --2- 172.21.15.2:0/1679089923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f780006b730 0x7f7800141e30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f77f000c480 tx=0x7f77f000c790 comp rx=0 tx=0).stop 2023-08-25T03:34:15.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.022+0000 7f7805452700 1 -- 172.21.15.2:0/1679089923 shutdown_connections 2023-08-25T03:34:15.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.022+0000 7f7805452700 1 --2- 172.21.15.2:0/1679089923 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78001616d0 0x7f7800162750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.022+0000 7f7805452700 1 --2- 172.21.15.2:0/1679089923 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f780006b730 0x7f7800141e30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.022+0000 7f7805452700 1 --2- 172.21.15.2:0/1679089923 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f77ec06a160 0x7f77ec06c620 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.022+0000 7f7805452700 1 --2- 172.21.15.2:0/1679089923 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78000b43c0 0x7f780006a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.022+0000 7f7805452700 1 -- 172.21.15.2:0/1679089923 >> 172.21.15.2:0/1679089923 conn(0x7f78000cfd10 msgr2=0x7f7800070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:15.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.022+0000 7f7805452700 1 -- 172.21.15.2:0/1679089923 shutdown_connections 2023-08-25T03:34:15.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.022+0000 7f7805452700 1 -- 172.21.15.2:0/1679089923 wait complete. 2023-08-25T03:34:15.027 INFO:teuthology.orchestra.run.smithi002.stdout:60129542152 2023-08-25T03:34:15.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.024+0000 7f259dc09700 1 -- 172.21.15.2:0/2458596778 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f257c06a0b0 msgr2=0x7f257c06c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:15.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.024+0000 7f259dc09700 1 --2- 172.21.15.2:0/2458596778 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f257c06a0b0 0x7f257c06c570 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2598070ef0 tx=0x7f2590009000 comp rx=0 tx=0).stop 2023-08-25T03:34:15.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.024+0000 7f259dc09700 1 -- 172.21.15.2:0/2458596778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25980b20b0 msgr2=0x7f25980b0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:15.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.024+0000 7f259dc09700 1 --2- 172.21.15.2:0/2458596778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25980b20b0 0x7f25980b0990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f258800c4d0 tx=0x7f258800c7e0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.027+0000 7f259dc09700 1 -- 172.21.15.2:0/2458596778 shutdown_connections 2023-08-25T03:34:15.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.027+0000 7f259dc09700 1 --2- 172.21.15.2:0/2458596778 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f257c06a0b0 0x7f257c06c570 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.027+0000 7f259dc09700 1 --2- 172.21.15.2:0/2458596778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2598161d50 0x7f2598162e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.027+0000 7f259dc09700 1 --2- 172.21.15.2:0/2458596778 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25980b20b0 0x7f25980b0990 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.027+0000 7f259dc09700 1 --2- 172.21.15.2:0/2458596778 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f259815bd80 0x7f2598149a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.027+0000 7f259dc09700 1 -- 172.21.15.2:0/2458596778 >> 172.21.15.2:0/2458596778 conn(0x7f25980cfea0 msgr2=0x7f25980d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:15.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.027+0000 7f259dc09700 1 -- 172.21.15.2:0/2458596778 shutdown_connections 2023-08-25T03:34:15.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.027+0000 7f259dc09700 1 -- 172.21.15.2:0/2458596778 wait complete. 2023-08-25T03:34:15.033 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.2 2023-08-25T03:34:15.038 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542152 for osd.7 2023-08-25T03:34:15.041 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.0 2023-08-25T03:34:15.041 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542151 for osd.5 2023-08-25T03:34:15.044 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542152 for osd.4 2023-08-25T03:34:15.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.047+0000 7f9f461bb700 1 -- 172.21.15.2:0/1520373568 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9f400acba0 con 0x7f9f401f5b70 2023-08-25T03:34:15.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.048+0000 7f9f27fff700 1 -- 172.21.15.2:0/1520373568 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9f30055820 con 0x7f9f401f5b70 2023-08-25T03:34:15.051 INFO:teuthology.orchestra.run.smithi002.stdout:55834574855 2023-08-25T03:34:15.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.050+0000 7f9f461bb700 1 -- 172.21.15.2:0/1520373568 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9f2c06a1e0 msgr2=0x7f9f2c06c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:15.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.050+0000 7f9f461bb700 1 --2- 172.21.15.2:0/1520373568 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9f2c06a1e0 0x7f9f2c06c6a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f9f4004ffc0 tx=0x7f9f38007310 comp rx=0 tx=0).stop 2023-08-25T03:34:15.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.050+0000 7f9f461bb700 1 -- 172.21.15.2:0/1520373568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f401f5b70 msgr2=0x7f9f4006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:15.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.050+0000 7f9f461bb700 1 --2- 172.21.15.2:0/1520373568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f401f5b70 0x7f9f4006d530 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000c910 tx=0x7f9f300019f0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.050+0000 7f9f461bb700 1 -- 172.21.15.2:0/1520373568 shutdown_connections 2023-08-25T03:34:15.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.050+0000 7f9f461bb700 1 --2- 172.21.15.2:0/1520373568 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9f400b1ae0 0x7f9f400b1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.050+0000 7f9f461bb700 1 --2- 172.21.15.2:0/1520373568 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9f28004850 0x7f9f401f54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.050+0000 7f9f461bb700 1 --2- 172.21.15.2:0/1520373568 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9f2c06a1e0 0x7f9f2c06c6a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.050+0000 7f9f461bb700 1 --2- 172.21.15.2:0/1520373568 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9f401f5b70 0x7f9f4006d530 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:15.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.050+0000 7f9f461bb700 1 -- 172.21.15.2:0/1520373568 >> 172.21.15.2:0/1520373568 conn(0x7f9f400ceec0 msgr2=0x7f9f400ace80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:15.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.050+0000 7f9f461bb700 1 -- 172.21.15.2:0/1520373568 shutdown_connections 2023-08-25T03:34:15.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:15.050+0000 7f9f461bb700 1 -- 172.21.15.2:0/1520373568 wait complete. 2023-08-25T03:34:15.063 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.3 2023-08-25T03:34:16.009 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-08-25T03:34:16.010 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-08-25T03:34:16.034 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-08-25T03:34:16.039 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-08-25T03:34:16.042 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-08-25T03:34:16.042 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-08-25T03:34:16.045 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-08-25T03:34:16.064 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-08-25T03:34:16.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.218+0000 7facf301c700 1 Processor -- start 2023-08-25T03:34:16.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.218+0000 7facf301c700 1 -- start start 2023-08-25T03:34:16.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.221+0000 7facf301c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7facec071030 0x7facec070140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.221+0000 7facf301c700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7facec08a8f0 0x7facec16a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.221+0000 7facf301c700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facec08ad80 con 0x7facec070940 2023-08-25T03:34:16.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.221+0000 7facf301c700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facec0cd010 con 0x7facec071030 2023-08-25T03:34:16.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.221+0000 7facf301c700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facec16b7b0 con 0x7facec08a8f0 2023-08-25T03:34:16.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.221+0000 7f090919f700 1 Processor -- start 2023-08-25T03:34:16.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.222+0000 7f090919f700 1 -- start start 2023-08-25T03:34:16.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.223+0000 7f090919f700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f09041570b0 0x7f09040702d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.223+0000 7f090919f700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f090415d750 0x7f09040ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.223+0000 7f090919f700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0904157490 con 0x7f09040cad70 2023-08-25T03:34:16.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.223+0000 7f090919f700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09040717c0 con 0x7f09041570b0 2023-08-25T03:34:16.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.223+0000 7f090919f700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09040cca90 con 0x7f090415d750 2023-08-25T03:34:16.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.223+0000 7facf0db8700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7facec071030 0x7facec070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.223+0000 7facf0db8700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7facec071030 0x7facec070140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59598/0 (socket says 172.21.15.2:59598) 2023-08-25T03:34:16.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.223+0000 7facf0db8700 1 -- 172.21.15.2:0/2870450480 learned_addr learned my addr 172.21.15.2:0/2870450480 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:16.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.224+0000 7f090359e700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f090415d750 0x7f09040ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.224+0000 7f090359e700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f090415d750 0x7f09040ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.2:59852/0 (socket says 172.21.15.2:59852) 2023-08-25T03:34:16.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.224+0000 7f090359e700 1 -- 172.21.15.2:0/1730732332 learned_addr learned my addr 172.21.15.2:0/1730732332 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:16.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.224+0000 7f0902d9d700 1 --2- 172.21.15.2:0/1730732332 >> v2:172.21.15.2:3300/0 conn(0x7f09041570b0 0x7f09040702d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.224+0000 7f0902d9d700 1 -- 172.21.15.2:0/1730732332 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f090415d750 msgr2=0x7f09040ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:16.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.224+0000 7f0902d9d700 1 --2- 172.21.15.2:0/1730732332 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f090415d750 0x7f09040ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.224+0000 7f0902d9d700 1 -- 172.21.15.2:0/1730732332 >> v1:172.21.15.31:6789/0 conn(0x7f09040cad70 legacy=0x7f090415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.224+0000 7f0902d9d700 1 -- 172.21.15.2:0/1730732332 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0904070c40 con 0x7f09041570b0 2023-08-25T03:34:16.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.224+0000 7facf0db8700 1 -- 172.21.15.2:0/2870450480 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7facec08a8f0 msgr2=0x7facec16a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:16.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.224+0000 7facf0db8700 1 --2- 172.21.15.2:0/2870450480 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7facec08a8f0 0x7facec16a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.224+0000 7facf0db8700 1 -- 172.21.15.2:0/2870450480 >> v1:172.21.15.31:6789/0 conn(0x7facec070940 legacy=0x7facec08a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:16.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.224+0000 7facf0db8700 1 -- 172.21.15.2:0/2870450480 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec070680 con 0x7facec071030 2023-08-25T03:34:16.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.224+0000 7f0902d9d700 1 --2- 172.21.15.2:0/1730732332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09041570b0 0x7f09040702d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f08f4008840 tx=0x7f08f400d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d48dc761fe27893 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.225+0000 7facf0db8700 1 --2- 172.21.15.2:0/2870450480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facec071030 0x7facec070140 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7face4009e50 tx=0x7face400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c0bb4deb9017f8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.226+0000 7f0901d9b700 1 -- 172.21.15.2:0/1730732332 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f4010070 con 0x7f09041570b0 2023-08-25T03:34:16.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.226+0000 7f0901d9b700 1 -- 172.21.15.2:0/1730732332 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f400d9c0 con 0x7f09041570b0 2023-08-25T03:34:16.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.226+0000 7f0901d9b700 1 -- 172.21.15.2:0/1730732332 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f400dd50 con 0x7f09041570b0 2023-08-25T03:34:16.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.227+0000 7f090919f700 1 -- 172.21.15.2:0/1730732332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09041570b0 msgr2=0x7f09040702d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.227+0000 7f090919f700 1 --2- 172.21.15.2:0/1730732332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09041570b0 0x7f09040702d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f08f4008840 tx=0x7f08f400d330 comp rx=0 tx=0).stop 2023-08-25T03:34:16.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.225+0000 7faceb7fe700 1 -- 172.21.15.2:0/2870450480 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face400e070 con 0x7facec071030 2023-08-25T03:34:16.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.225+0000 7faceb7fe700 1 -- 172.21.15.2:0/2870450480 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face4003a20 con 0x7facec071030 2023-08-25T03:34:16.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.225+0000 7faceb7fe700 1 -- 172.21.15.2:0/2870450480 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4003dd0 con 0x7facec071030 2023-08-25T03:34:16.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.226+0000 7facf301c700 1 -- 172.21.15.2:0/2870450480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facec071030 msgr2=0x7facec070140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.226+0000 7facf301c700 1 --2- 172.21.15.2:0/2870450480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facec071030 0x7facec070140 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7face4009e50 tx=0x7face400b330 comp rx=0 tx=0).stop 2023-08-25T03:34:16.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.227+0000 7facf301c700 1 -- 172.21.15.2:0/2870450480 shutdown_connections 2023-08-25T03:34:16.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.227+0000 7facf301c700 1 --2- 172.21.15.2:0/2870450480 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7facec08a8f0 0x7facec16a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.227+0000 7facf301c700 1 --2- 172.21.15.2:0/2870450480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facec071030 0x7facec070140 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.227+0000 7facf301c700 1 -- 172.21.15.2:0/2870450480 >> 172.21.15.2:0/2870450480 conn(0x7facec0ceec0 msgr2=0x7facec0cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:16.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.227+0000 7facf301c700 1 -- 172.21.15.2:0/2870450480 shutdown_connections 2023-08-25T03:34:16.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.227+0000 7facf301c700 1 -- 172.21.15.2:0/2870450480 wait complete. 2023-08-25T03:34:16.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.229+0000 7facf301c700 1 Processor -- start 2023-08-25T03:34:16.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.229+0000 7facf301c700 1 -- start start 2023-08-25T03:34:16.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.231+0000 7facf301c700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7facec070940 0x7facec099a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.231+0000 7facf301c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facec071030 0x7facec210770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.231+0000 7facf301c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facec08a8f0 0x7facec0aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.231+0000 7facf301c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec210cb0 con 0x7facec08a8f0 2023-08-25T03:34:16.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.231+0000 7facf301c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec0af490 con 0x7facec071030 2023-08-25T03:34:16.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.231+0000 7facf301c700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facec09a0f0 con 0x7facec070940 2023-08-25T03:34:16.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.231+0000 7facf15b9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facec08a8f0 0x7facec0aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.231+0000 7facf15b9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facec08a8f0 0x7facec0aef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59610/0 (socket says 172.21.15.2:59610) 2023-08-25T03:34:16.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.231+0000 7facf15b9700 1 -- 172.21.15.2:0/1065260405 learned_addr learned my addr 172.21.15.2:0/1065260405 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:16.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.231+0000 7f090919f700 1 -- 172.21.15.2:0/1730732332 shutdown_connections 2023-08-25T03:34:16.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.231+0000 7f090919f700 1 --2- 172.21.15.2:0/1730732332 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f090415d750 0x7f09040ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.231+0000 7f090919f700 1 --2- 172.21.15.2:0/1730732332 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09041570b0 0x7f09040702d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.231+0000 7f090919f700 1 -- 172.21.15.2:0/1730732332 >> 172.21.15.2:0/1730732332 conn(0x7f09040ceec0 msgr2=0x7f09040cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:16.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.232+0000 7facebfff700 1 --2- 172.21.15.2:0/1065260405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facec071030 0x7facec210770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.232+0000 7f090919f700 1 -- 172.21.15.2:0/1730732332 shutdown_connections 2023-08-25T03:34:16.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.232+0000 7f090919f700 1 -- 172.21.15.2:0/1730732332 wait complete. 2023-08-25T03:34:16.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.235+0000 7f090919f700 1 Processor -- start 2023-08-25T03:34:16.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.235+0000 7f090919f700 1 -- start start 2023-08-25T03:34:16.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.236+0000 7f090919f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09040cad70 0x7f09040c5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.236+0000 7f090919f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09041570b0 0x7f090406cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.236+0000 7f090919f700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f090415d750 0x7f09041fef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.236+0000 7f090919f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09040c4240 con 0x7f09041570b0 2023-08-25T03:34:16.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.236+0000 7f090919f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090406d330 con 0x7f09040cad70 2023-08-25T03:34:16.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.236+0000 7f090919f700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090406d740 con 0x7f090415d750 2023-08-25T03:34:16.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.236+0000 7f090259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09041570b0 0x7f090406cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.236+0000 7f090259c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09041570b0 0x7f090406cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59616/0 (socket says 172.21.15.2:59616) 2023-08-25T03:34:16.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.236+0000 7f090259c700 1 -- 172.21.15.2:0/2287035650 learned_addr learned my addr 172.21.15.2:0/2287035650 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:16.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.232+0000 7facf15b9700 1 -- 172.21.15.2:0/1065260405 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7facec070940 msgr2=0x7facec099a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.232+0000 7facf15b9700 1 --2- 172.21.15.2:0/1065260405 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7facec070940 0x7facec099a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.232+0000 7facf15b9700 1 -- 172.21.15.2:0/1065260405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facec071030 msgr2=0x7facec210770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.232+0000 7facf15b9700 1 --2- 172.21.15.2:0/1065260405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facec071030 0x7facec210770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.232+0000 7facf15b9700 1 -- 172.21.15.2:0/1065260405 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face4009b30 con 0x7facec08a8f0 2023-08-25T03:34:16.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.232+0000 7facebfff700 1 --2- 172.21.15.2:0/1065260405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facec071030 0x7facec210770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:16.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.232+0000 7facf15b9700 1 --2- 172.21.15.2:0/1065260405 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facec08a8f0 0x7facec0aef50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7face000b550 tx=0x7face000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.235+0000 7face97fa700 1 -- 172.21.15.2:0/1065260405 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0011830 con 0x7facec08a8f0 2023-08-25T03:34:16.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.236+0000 7f0902d9d700 1 --2- 172.21.15.2:0/2287035650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09040cad70 0x7f09040c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.237+0000 7f090259c700 1 -- 172.21.15.2:0/2287035650 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f090415d750 msgr2=0x7f09041fef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.237+0000 7f090259c700 1 --2- 172.21.15.2:0/2287035650 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f090415d750 0x7f09041fef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.237+0000 7f090259c700 1 -- 172.21.15.2:0/2287035650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09040cad70 msgr2=0x7f09040c5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.237+0000 7f090259c700 1 --2- 172.21.15.2:0/2287035650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09040cad70 0x7f09040c5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.237+0000 7f090259c700 1 -- 172.21.15.2:0/2287035650 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f4008520 con 0x7f09041570b0 2023-08-25T03:34:16.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.237+0000 7f090259c700 1 --2- 172.21.15.2:0/2287035650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09041570b0 0x7f090406cdf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f08f800c4e0 tx=0x7f08f800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.237+0000 7face97fa700 1 -- 172.21.15.2:0/1065260405 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face0011990 con 0x7facec08a8f0 2023-08-25T03:34:16.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.237+0000 7face97fa700 1 -- 172.21.15.2:0/1065260405 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face0011d40 con 0x7facec08a8f0 2023-08-25T03:34:16.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.238+0000 7facf301c700 1 -- 172.21.15.2:0/1065260405 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facec09a3d0 con 0x7facec08a8f0 2023-08-25T03:34:16.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.238+0000 7facf301c700 1 -- 172.21.15.2:0/1065260405 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facec09a990 con 0x7facec08a8f0 2023-08-25T03:34:16.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.239+0000 7facf301c700 1 -- 172.21.15.2:0/1065260405 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facec0c9820 con 0x7facec08a8f0 2023-08-25T03:34:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.239+0000 7f08eb7fe700 1 -- 172.21.15.2:0/2287035650 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f80052d0 con 0x7f09041570b0 2023-08-25T03:34:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.239+0000 7f08eb7fe700 1 -- 172.21.15.2:0/2287035650 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f8005430 con 0x7f09041570b0 2023-08-25T03:34:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.239+0000 7f08eb7fe700 1 -- 172.21.15.2:0/2287035650 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f80038c0 con 0x7f09041570b0 2023-08-25T03:34:16.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.240+0000 7f090919f700 1 -- 172.21.15.2:0/2287035650 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f090406da00 con 0x7f09041570b0 2023-08-25T03:34:16.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.240+0000 7f090919f700 1 -- 172.21.15.2:0/2287035650 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f090406dfc0 con 0x7f09041570b0 2023-08-25T03:34:16.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.240+0000 7face97fa700 1 -- 172.21.15.2:0/1065260405 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7face001a400 con 0x7facec08a8f0 2023-08-25T03:34:16.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.241+0000 7face97fa700 1 --2- 172.21.15.2:0/1065260405 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7facd806a0e0 0x7facd806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.241+0000 7f090919f700 1 -- 172.21.15.2:0/2287035650 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08e40052f0 con 0x7f09041570b0 2023-08-25T03:34:16.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.243+0000 7f08eb7fe700 1 -- 172.21.15.2:0/2287035650 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f08f8003a20 con 0x7f09041570b0 2023-08-25T03:34:16.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.244+0000 7f08eb7fe700 1 --2- 172.21.15.2:0/2287035650 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f08f006a1b0 0x7f08f006c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.244+0000 7f08eb7fe700 1 -- 172.21.15.2:0/2287035650 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f08f8087ee0 con 0x7f09041570b0 2023-08-25T03:34:16.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.245+0000 7f0902d9d700 1 --2- 172.21.15.2:0/2287035650 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f08f006a1b0 0x7f08f006c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.246+0000 7f0902d9d700 1 --2- 172.21.15.2:0/2287035650 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f08f006a1b0 0x7f08f006c670 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f08f400d370 tx=0x7f08f4012000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.251+0000 7f08eb7fe700 1 -- 172.21.15.2:0/2287035650 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f08f80535b0 con 0x7f09041570b0 2023-08-25T03:34:16.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.249+0000 7facf0db8700 1 --2- 172.21.15.2:0/1065260405 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7facd806a0e0 0x7facd806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.250+0000 7facf0db8700 1 --2- 172.21.15.2:0/1065260405 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7facd806a0e0 0x7facd806c5a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7face400b370 tx=0x7face4010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.252+0000 7face97fa700 1 -- 172.21.15.2:0/1065260405 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7face0057330 con 0x7facec08a8f0 2023-08-25T03:34:16.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.259+0000 7face97fa700 1 -- 172.21.15.2:0/1065260405 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7face0052fe0 con 0x7facec08a8f0 2023-08-25T03:34:16.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.274+0000 7f369368f700 1 Processor -- start 2023-08-25T03:34:16.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.274+0000 7f369368f700 1 -- start start 2023-08-25T03:34:16.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.275+0000 7f369368f700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f368c070bb0 0x7f368c16e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.275+0000 7f369368f700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f368c0ccb20 con 0x7f368c0cdfc0 2023-08-25T03:34:16.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.275+0000 7f369368f700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f368c0ca390 con 0x7f368c070f90 2023-08-25T03:34:16.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.275+0000 7f369368f700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368c0ca670 con 0x7f368c070bb0 2023-08-25T03:34:16.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.276+0000 7f369142b700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f368c070bb0 0x7f368c16e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.276+0000 7f369142b700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f368c070bb0 0x7f368c16e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.2:59870/0 (socket says 172.21.15.2:59870) 2023-08-25T03:34:16.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.276+0000 7f369142b700 1 -- 172.21.15.2:0/1804208042 learned_addr learned my addr 172.21.15.2:0/1804208042 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:16.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.278+0000 7f3683fff700 1 -- 172.21.15.2:0/1804208042 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 310552969 0 0) 0x7f368c0ccb20 con 0x7f368c0cdfc0 2023-08-25T03:34:16.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.278+0000 7f3683fff700 1 -- 172.21.15.2:0/1804208042 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3678003590 con 0x7f368c0cdfc0 2023-08-25T03:34:16.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.278+0000 7f3683fff700 1 -- 172.21.15.2:0/1804208042 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3548193400 0 0) 0x7f368c0ca390 con 0x7f368c070f90 2023-08-25T03:34:16.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.278+0000 7f3683fff700 1 -- 172.21.15.2:0/1804208042 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f368c0ccb20 con 0x7f368c070f90 2023-08-25T03:34:16.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.279+0000 7f3683fff700 1 -- 172.21.15.2:0/1804208042 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2534968881 0 0) 0x7f3678003590 con 0x7f368c0cdfc0 2023-08-25T03:34:16.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.279+0000 7f3683fff700 1 -- 172.21.15.2:0/1804208042 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f368c070bb0 msgr2=0x7f368c16e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:16.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.279+0000 7f3683fff700 1 --2- 172.21.15.2:0/1804208042 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f368c070bb0 0x7f368c16e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.279+0000 7f3683fff700 1 -- 172.21.15.2:0/1804208042 >> v1:172.21.15.31:6789/0 conn(0x7f368c070f90 legacy=0x7f368c08a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.279+0000 7f3683fff700 1 -- 172.21.15.2:0/1804208042 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368c0ca950 con 0x7f368c0cdfc0 2023-08-25T03:34:16.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.279+0000 7f3683fff700 1 -- 172.21.15.2:0/1804208042 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f36840019f0 con 0x7f368c0cdfc0 2023-08-25T03:34:16.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.279+0000 7f3683fff700 1 -- 172.21.15.2:0/1804208042 >> v1:172.21.15.2:6789/0 conn(0x7f368c0cdfc0 legacy=0x7f368c08a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.279+0000 7f3683fff700 1 --2- 172.21.15.2:0/1804208042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3678003d70 0x7f36780061a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.279+0000 7f3683fff700 1 --2- 172.21.15.2:0/1804208042 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3678006770 0x7f3678006bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.279+0000 7f3683fff700 1 --2- 172.21.15.2:0/1804208042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3678007360 0x7f36780077e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.279+0000 7f3683fff700 1 -- 172.21.15.2:0/1804208042 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c0ca670 con 0x7f3678003d70 2023-08-25T03:34:16.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.279+0000 7f3683fff700 1 -- 172.21.15.2:0/1804208042 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36780080c0 con 0x7f3678007360 2023-08-25T03:34:16.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.279+0000 7f3683fff700 1 -- 172.21.15.2:0/1804208042 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3678008280 con 0x7f3678006770 2023-08-25T03:34:16.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.279+0000 7f3690c2a700 1 --2- 172.21.15.2:0/1804208042 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3678006770 0x7f3678006bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.279+0000 7f369142b700 1 --2- 172.21.15.2:0/1804208042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3678003d70 0x7f36780061a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.280+0000 7f3691c2c700 1 --2- 172.21.15.2:0/1804208042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3678007360 0x7f36780077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.280+0000 7f369142b700 1 -- 172.21.15.2:0/1804208042 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3678006770 msgr2=0x7f3678006bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:16.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.280+0000 7f369142b700 1 --2- 172.21.15.2:0/1804208042 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3678006770 0x7f3678006bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.280+0000 7f369142b700 1 -- 172.21.15.2:0/1804208042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3678007360 msgr2=0x7f36780077e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:16.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.280+0000 7f369142b700 1 --2- 172.21.15.2:0/1804208042 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3678007360 0x7f36780077e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.280+0000 7f369142b700 1 -- 172.21.15.2:0/1804208042 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368c0ca950 con 0x7f3678003d70 2023-08-25T03:34:16.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.280+0000 7f369142b700 1 --2- 172.21.15.2:0/1804208042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3678003d70 0x7f36780061a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f368c0729f0 tx=0x7f3678003840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec53aa5520812be server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.280+0000 7f3683fff700 1 -- 172.21.15.2:0/1804208042 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368800c070 con 0x7f3678003d70 2023-08-25T03:34:16.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.281+0000 7f3683fff700 1 -- 172.21.15.2:0/1804208042 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3688013630 con 0x7f3678003d70 2023-08-25T03:34:16.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.281+0000 7f3683fff700 1 -- 172.21.15.2:0/1804208042 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36880139e0 con 0x7f3678003d70 2023-08-25T03:34:16.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.281+0000 7f369368f700 1 -- 172.21.15.2:0/1804208042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3678003d70 msgr2=0x7f36780061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.281+0000 7f369368f700 1 --2- 172.21.15.2:0/1804208042 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3678003d70 0x7f36780061a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f368c0729f0 tx=0x7f3678003840 comp rx=0 tx=0).stop 2023-08-25T03:34:16.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.281+0000 7f3691c2c700 1 -- 172.21.15.2:0/1804208042 reap_dead start 2023-08-25T03:34:16.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.282+0000 7f369368f700 1 -- 172.21.15.2:0/1804208042 shutdown_connections 2023-08-25T03:34:16.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.282+0000 7f369368f700 1 -- 172.21.15.2:0/1804208042 >> 172.21.15.2:0/1804208042 conn(0x7f368c0ceec0 msgr2=0x7f368c0cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:16.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.282+0000 7f369368f700 1 -- 172.21.15.2:0/1804208042 shutdown_connections 2023-08-25T03:34:16.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.282+0000 7f369368f700 1 -- 172.21.15.2:0/1804208042 wait complete. 2023-08-25T03:34:16.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.283+0000 7f369368f700 1 Processor -- start 2023-08-25T03:34:16.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.283+0000 7f369368f700 1 -- start start 2023-08-25T03:34:16.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.284+0000 7f369368f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3678003d70 0x7f368c16bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.284+0000 7f369368f700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f368c09c270 0x7f368c06dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.284+0000 7f369368f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f368c06e190 0x7f368c09aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.284+0000 7f369368f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c06e640 con 0x7f368c06e190 2023-08-25T03:34:16.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.284+0000 7f369368f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c0c8cf0 con 0x7f3678003d70 2023-08-25T03:34:16.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.284+0000 7f369368f700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368c0c6130 con 0x7f368c09c270 2023-08-25T03:34:16.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.285+0000 7f3691c2c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f368c06e190 0x7f368c09aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.285+0000 7f369142b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3678003d70 0x7f368c16bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.285+0000 7f3691c2c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f368c06e190 0x7f368c09aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59640/0 (socket says 172.21.15.2:59640) 2023-08-25T03:34:16.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.285+0000 7f3691c2c700 1 -- 172.21.15.2:0/3372883984 learned_addr learned my addr 172.21.15.2:0/3372883984 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:16.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.285+0000 7f3d0e016700 1 Processor -- start 2023-08-25T03:34:16.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.285+0000 7f3d0e016700 1 -- start start 2023-08-25T03:34:16.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.287+0000 7f3d0e016700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3d080d2b80 0x7f3d08165210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.287+0000 7f3d0e016700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f3d080cfaf0 0x7f3d080cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.287+0000 7f3d0e016700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d08165750 con 0x7f3d08161560 2023-08-25T03:34:16.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.287+0000 7f3d0e016700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d080d0ce0 con 0x7f3d080cfaf0 2023-08-25T03:34:16.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.287+0000 7f3d0e016700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d080d04e0 con 0x7f3d080d2b80 2023-08-25T03:34:16.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.285+0000 7f3691c2c700 1 -- 172.21.15.2:0/3372883984 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f368c09c270 msgr2=0x7f368c06dc50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-25T03:34:16.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.285+0000 7f3691c2c700 1 --2- 172.21.15.2:0/3372883984 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f368c09c270 0x7f368c06dc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.285+0000 7f3691c2c700 1 -- 172.21.15.2:0/3372883984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3678003d70 msgr2=0x7f368c16bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.285+0000 7f3691c2c700 1 --2- 172.21.15.2:0/3372883984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3678003d70 0x7f368c16bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.285+0000 7f3691c2c700 1 -- 172.21.15.2:0/3372883984 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368c0c6410 con 0x7f368c06e190 2023-08-25T03:34:16.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.287+0000 7f3d077fe700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f3d08161560 0x7f3d080b5f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46312/0 (socket says 172.21.15.2:46312) 2023-08-25T03:34:16.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.287+0000 7f3d06ffd700 1 --2- 172.21.15.2:0/78566557 >> v2:172.21.15.31:3300/0 conn(0x7f3d080d2b80 0x7f3d08165210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.287+0000 7f3d077fe700 1 -- 172.21.15.2:0/78566557 learned_addr learned my addr 172.21.15.2:0/78566557 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:16.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.287+0000 7f3691c2c700 1 --2- 172.21.15.2:0/3372883984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f368c06e190 0x7f368c09aab0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3684001460 tx=0x7f36840058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.287+0000 7f3d07fff700 1 --2- 172.21.15.2:0/78566557 >> v2:172.21.15.2:3300/0 conn(0x7f3d080cfaf0 0x7f3d080cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.288+0000 7f3681ffb700 1 -- 172.21.15.2:0/3372883984 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368400abd0 con 0x7f368c06e190 2023-08-25T03:34:16.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.288+0000 7f3d06ffd700 1 -- 172.21.15.2:0/78566557 >> v2:172.21.15.2:3300/0 conn(0x7f3d080cfaf0 msgr2=0x7f3d080cffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:16.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.288+0000 7f3d06ffd700 1 --2- 172.21.15.2:0/78566557 >> v2:172.21.15.2:3300/0 conn(0x7f3d080cfaf0 0x7f3d080cffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.288+0000 7f3d06ffd700 1 -- 172.21.15.2:0/78566557 >> v1:172.21.15.2:6789/0 conn(0x7f3d08161560 legacy=0x7f3d080b5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.288+0000 7f3d06ffd700 1 -- 172.21.15.2:0/78566557 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d08165990 con 0x7f3d080d2b80 2023-08-25T03:34:16.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.288+0000 7f3d07fff700 1 --2- 172.21.15.2:0/78566557 >> v2:172.21.15.2:3300/0 conn(0x7f3d080cfaf0 0x7f3d080cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:34:16.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.288+0000 7f3d06ffd700 1 --2- 172.21.15.2:0/78566557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d080d2b80 0x7f3d08165210 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3d0808c480 tx=0x7f3cfc009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81106c1e3614ea9a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.285+0000 7f164fb37700 1 Processor -- start 2023-08-25T03:34:16.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.285+0000 7f164fb37700 1 -- start start 2023-08-25T03:34:16.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.286+0000 7f164fb37700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16480cdfc0 0x7f164816e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.287+0000 7f164fb37700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f1648070f90 0x7f16480c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.287+0000 7f164fb37700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16480ca150 con 0x7f1648070bb0 2023-08-25T03:34:16.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.287+0000 7f164fb37700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16480ccb20 con 0x7f1648070f90 2023-08-25T03:34:16.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.287+0000 7f164fb37700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16480ca390 con 0x7f16480cdfc0 2023-08-25T03:34:16.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.287+0000 7f164d8d3700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1648070bb0 0x7f164808a0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.2:54340/0 (socket says 172.21.15.2:54340) 2023-08-25T03:34:16.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.287+0000 7f164d8d3700 1 -- 172.21.15.2:0/1577020932 learned_addr learned my addr 172.21.15.2:0/1577020932 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:16.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.288+0000 7f164c8d1700 1 -- 172.21.15.2:0/1577020932 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2557852694 0 0) 0x7f1644001110 con 0x7f1648070bb0 2023-08-25T03:34:16.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.288+0000 7f164d0d2700 1 --2- 172.21.15.2:0/1577020932 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16480cdfc0 0x7f164816e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.288+0000 7f164c8d1700 1 -- 172.21.15.2:0/1577020932 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16340035b0 con 0x7f1648070bb0 2023-08-25T03:34:16.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.288+0000 7f369368f700 1 -- 172.21.15.2:0/3372883984 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368c0c66f0 con 0x7f368c06e190 2023-08-25T03:34:16.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.288+0000 7f3681ffb700 1 -- 172.21.15.2:0/3372883984 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368400ad50 con 0x7f368c06e190 2023-08-25T03:34:16.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.288+0000 7f3681ffb700 1 -- 172.21.15.2:0/3372883984 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3684013630 con 0x7f368c06e190 2023-08-25T03:34:16.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.288+0000 7f369368f700 1 -- 172.21.15.2:0/3372883984 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368c0c6cb0 con 0x7f368c06e190 2023-08-25T03:34:16.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.289+0000 7f3d067fc700 1 -- 172.21.15.2:0/78566557 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc004230 con 0x7f3d080d2b80 2023-08-25T03:34:16.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.289+0000 7f3d067fc700 1 -- 172.21.15.2:0/78566557 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cfc004390 con 0x7f3d080d2b80 2023-08-25T03:34:16.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.289+0000 7f3d067fc700 1 -- 172.21.15.2:0/78566557 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cfc0036a0 con 0x7f3d080d2b80 2023-08-25T03:34:16.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.289+0000 7f369368f700 1 -- 172.21.15.2:0/3372883984 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f368c0c6ed0 con 0x7f368c06e190 2023-08-25T03:34:16.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.290+0000 7f3d0e016700 1 -- 172.21.15.2:0/78566557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d080d2b80 msgr2=0x7f3d08165210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.290+0000 7f3d0e016700 1 --2- 172.21.15.2:0/78566557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d080d2b80 0x7f3d08165210 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3d0808c480 tx=0x7f3cfc009b20 comp rx=0 tx=0).stop 2023-08-25T03:34:16.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.289+0000 7f164e0d4700 1 --2- 172.21.15.2:0/1577020932 >> v2:172.21.15.2:3300/0 conn(0x7f1648070f90 0x7f16480c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.289+0000 7f164e0d4700 1 -- 172.21.15.2:0/1577020932 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16480cdfc0 msgr2=0x7f164816e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:16.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.289+0000 7f164e0d4700 1 --2- 172.21.15.2:0/1577020932 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16480cdfc0 0x7f164816e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.289+0000 7f164e0d4700 1 -- 172.21.15.2:0/1577020932 >> v1:172.21.15.31:6789/0 conn(0x7f1648070bb0 legacy=0x7f164808a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.289+0000 7f164e0d4700 1 -- 172.21.15.2:0/1577020932 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16480ca670 con 0x7f1648070f90 2023-08-25T03:34:16.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.290+0000 7f3d0e016700 1 -- 172.21.15.2:0/78566557 shutdown_connections 2023-08-25T03:34:16.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.290+0000 7f3d0e016700 1 --2- 172.21.15.2:0/78566557 >> v2:172.21.15.2:3300/0 conn(0x7f3d080cfaf0 0x7f3d080cffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.290+0000 7f3d0e016700 1 --2- 172.21.15.2:0/78566557 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d080d2b80 0x7f3d08165210 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.290+0000 7f3d0e016700 1 -- 172.21.15.2:0/78566557 >> 172.21.15.2:0/78566557 conn(0x7f3d0806abe0 msgr2=0x7f3d0806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:16.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.290+0000 7f3d0e016700 1 -- 172.21.15.2:0/78566557 shutdown_connections 2023-08-25T03:34:16.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.292+0000 7f3681ffb700 1 -- 172.21.15.2:0/3372883984 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3684013790 con 0x7f368c06e190 2023-08-25T03:34:16.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.290+0000 7f3d0e016700 1 -- 172.21.15.2:0/78566557 wait complete. 2023-08-25T03:34:16.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.292+0000 7f3d0e016700 1 Processor -- start 2023-08-25T03:34:16.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.292+0000 7f3681ffb700 1 --2- 172.21.15.2:0/3372883984 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f367006a0e0 0x7f367006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.292+0000 7f3681ffb700 1 -- 172.21.15.2:0/3372883984 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3684089540 con 0x7f368c06e190 2023-08-25T03:34:16.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.293+0000 7f369142b700 1 --2- 172.21.15.2:0/3372883984 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f367006a0e0 0x7f367006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.292+0000 7f3d0e016700 1 -- start start 2023-08-25T03:34:16.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.293+0000 7f3d0e016700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d080cfaf0 0x7f3d08162ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.293+0000 7f3d0e016700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d080d2b80 0x7f3d08163000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.293+0000 7f3d0e016700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d08161560 0x7f3d0815fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.293+0000 7f3d0e016700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0813d970 con 0x7f3d080cfaf0 2023-08-25T03:34:16.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.293+0000 7f3d0e016700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d08160c10 con 0x7f3d08161560 2023-08-25T03:34:16.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.293+0000 7f3d0e016700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0813ca80 con 0x7f3d080d2b80 2023-08-25T03:34:16.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.293+0000 7f3d077fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d080cfaf0 0x7f3d08162ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.293+0000 7f3d077fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d080cfaf0 0x7f3d08162ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59658/0 (socket says 172.21.15.2:59658) 2023-08-25T03:34:16.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.293+0000 7f3d077fe700 1 -- 172.21.15.2:0/2323808192 learned_addr learned my addr 172.21.15.2:0/2323808192 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:16.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.293+0000 7f3d07fff700 1 --2- 172.21.15.2:0/2323808192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d08161560 0x7f3d0815fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.294+0000 7f3d06ffd700 1 --2- 172.21.15.2:0/2323808192 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d080d2b80 0x7f3d08163000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.294+0000 7f3d077fe700 1 -- 172.21.15.2:0/2323808192 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d080d2b80 msgr2=0x7f3d08163000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.294+0000 7f3d077fe700 1 --2- 172.21.15.2:0/2323808192 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d080d2b80 0x7f3d08163000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.294+0000 7f3d077fe700 1 -- 172.21.15.2:0/2323808192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d08161560 msgr2=0x7f3d0815fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.294+0000 7f3d077fe700 1 --2- 172.21.15.2:0/2323808192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d08161560 0x7f3d0815fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.294+0000 7f3d077fe700 1 -- 172.21.15.2:0/2323808192 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d00009a70 con 0x7f3d080cfaf0 2023-08-25T03:34:16.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.294+0000 7f3d077fe700 1 --2- 172.21.15.2:0/2323808192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d080cfaf0 0x7f3d08162ac0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800c710 tx=0x7f3cf800ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.296+0000 7f211d05b700 1 Processor -- start 2023-08-25T03:34:16.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.296+0000 7f369142b700 1 --2- 172.21.15.2:0/3372883984 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f367006a0e0 0x7f367006c5a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f368800a010 tx=0x7f368800c000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.296+0000 7f211d05b700 1 -- start start 2023-08-25T03:34:16.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.297+0000 7f211d05b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f21180cad70 0x7f211815d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.297+0000 7f211d05b700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f211815d580 0x7f21180ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.297+0000 7f211d05b700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21180709e0 con 0x7f21181570b0 2023-08-25T03:34:16.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.297+0000 7f211d05b700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21180717c0 con 0x7f21180cad70 2023-08-25T03:34:16.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.297+0000 7f211d05b700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211815da10 con 0x7f211815d580 2023-08-25T03:34:16.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.297+0000 7f211659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f21180cad70 0x7f211815d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.297+0000 7f211659c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f21180cad70 0x7f211815d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.2:50956/0 (socket says 172.21.15.2:50956) 2023-08-25T03:34:16.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.297+0000 7f211659c700 1 -- 172.21.15.2:0/1878661486 learned_addr learned my addr 172.21.15.2:0/1878661486 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:16.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.297+0000 7f211759e700 1 --2- 172.21.15.2:0/1878661486 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f211815d580 0x7f21180ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.297+0000 7f211659c700 1 -- 172.21.15.2:0/1878661486 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f211815d580 msgr2=0x7f21180ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:16.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.297+0000 7f211659c700 1 --2- 172.21.15.2:0/1878661486 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f211815d580 0x7f21180ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.297+0000 7f211659c700 1 -- 172.21.15.2:0/1878661486 >> v1:172.21.15.31:6789/0 conn(0x7f21181570b0 legacy=0x7f21180702d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:16.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.297+0000 7f211659c700 1 -- 172.21.15.2:0/1878661486 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2118157490 con 0x7f21180cad70 2023-08-25T03:34:16.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.298+0000 7f211659c700 1 --2- 172.21.15.2:0/1878661486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21180cad70 0x7f211815d040 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f210c00b970 tx=0x7f210c00bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55d14aa367b29c63 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.298+0000 7f164e0d4700 1 --2- 172.21.15.2:0/1577020932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1648070f90 0x7f16480c9c10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1640009d90 tx=0x7f164000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdcff7da49f64b25 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.298+0000 7f74ba7a6700 1 Processor -- start 2023-08-25T03:34:16.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.299+0000 7f2115d9b700 1 -- 172.21.15.2:0/1878661486 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c004230 con 0x7f21180cad70 2023-08-25T03:34:16.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.299+0000 7f2115d9b700 1 -- 172.21.15.2:0/1878661486 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f210c004390 con 0x7f21180cad70 2023-08-25T03:34:16.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.299+0000 7f2115d9b700 1 -- 172.21.15.2:0/1878661486 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c009e40 con 0x7f21180cad70 2023-08-25T03:34:16.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.299+0000 7f3ceffff700 1 -- 172.21.15.2:0/2323808192 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8004c80 con 0x7f3d080cfaf0 2023-08-25T03:34:16.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.298+0000 7f74ba7a6700 1 -- start start 2023-08-25T03:34:16.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.299+0000 7f3681ffb700 1 -- 172.21.15.2:0/3372883984 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3684054cc0 con 0x7f368c06e190 2023-08-25T03:34:16.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.299+0000 7f164c8d1700 1 -- 172.21.15.2:0/1577020932 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164000e070 con 0x7f1648070f90 2023-08-25T03:34:16.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.299+0000 7f164c8d1700 1 -- 172.21.15.2:0/1577020932 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1640003a20 con 0x7f1648070f90 2023-08-25T03:34:16.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.299+0000 7f164c8d1700 1 -- 172.21.15.2:0/1577020932 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1644001800 con 0x7f1648070f90 2023-08-25T03:34:16.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.299+0000 7f3ceffff700 1 -- 172.21.15.2:0/2323808192 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf8004de0 con 0x7f3d080cfaf0 2023-08-25T03:34:16.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.299+0000 7f3ceffff700 1 -- 172.21.15.2:0/2323808192 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8007310 con 0x7f3d080cfaf0 2023-08-25T03:34:16.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.299+0000 7f3d0e016700 1 -- 172.21.15.2:0/2323808192 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cfc0094a0 con 0x7f3d080cfaf0 2023-08-25T03:34:16.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.299+0000 7f3d0e016700 1 -- 172.21.15.2:0/2323808192 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d0813d040 con 0x7f3d080cfaf0 2023-08-25T03:34:16.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.300+0000 7f211d05b700 1 -- 172.21.15.2:0/1878661486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21180cad70 msgr2=0x7f211815d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.300+0000 7f211d05b700 1 --2- 172.21.15.2:0/1878661486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21180cad70 0x7f211815d040 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f210c00b970 tx=0x7f210c00bc80 comp rx=0 tx=0).stop 2023-08-25T03:34:16.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.300+0000 7f211d05b700 1 -- 172.21.15.2:0/1878661486 shutdown_connections 2023-08-25T03:34:16.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.300+0000 7f211d05b700 1 --2- 172.21.15.2:0/1878661486 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f211815d580 0x7f21180ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.300+0000 7f211d05b700 1 --2- 172.21.15.2:0/1878661486 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21180cad70 0x7f211815d040 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.300+0000 7f211d05b700 1 -- 172.21.15.2:0/1878661486 >> 172.21.15.2:0/1878661486 conn(0x7f21180ceec0 msgr2=0x7f21180cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:16.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.301+0000 7f74ba7a6700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f74b406c160 0x7f74b406c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.301+0000 7f74ba7a6700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b415dba0 con 0x7f74b40cdf70 2023-08-25T03:34:16.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.301+0000 7f74ba7a6700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74b40cec10 con 0x7f74b40c9db0 2023-08-25T03:34:16.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.301+0000 7f74ba7a6700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74b4150710 con 0x7f74b406c160 2023-08-25T03:34:16.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.301+0000 7f3d0e016700 1 -- 172.21.15.2:0/2323808192 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d080746d0 con 0x7f3d080cfaf0 2023-08-25T03:34:16.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.301+0000 7f164fb37700 1 -- 172.21.15.2:0/1577020932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1648070f90 msgr2=0x7f16480c9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.301+0000 7f164fb37700 1 --2- 172.21.15.2:0/1577020932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1648070f90 0x7f16480c9c10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1640009d90 tx=0x7f164000b330 comp rx=0 tx=0).stop 2023-08-25T03:34:16.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.302+0000 7f164fb37700 1 -- 172.21.15.2:0/1577020932 shutdown_connections 2023-08-25T03:34:16.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.302+0000 7f164fb37700 1 --2- 172.21.15.2:0/1577020932 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1648070f90 0x7f16480c9c10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.302+0000 7f164fb37700 1 --2- 172.21.15.2:0/1577020932 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16480cdfc0 0x7f164816e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.302+0000 7f164fb37700 1 -- 172.21.15.2:0/1577020932 >> 172.21.15.2:0/1577020932 conn(0x7f16480ceec0 msgr2=0x7f16480cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:16.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.302+0000 7f164fb37700 1 -- 172.21.15.2:0/1577020932 shutdown_connections 2023-08-25T03:34:16.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.302+0000 7f74b8d43700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f74b40cdf70 0x7f74b406dfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46340/0 (socket says 172.21.15.2:46340) 2023-08-25T03:34:16.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.302+0000 7f74b3fff700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f74b406c160 0x7f74b406c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.302+0000 7f74b8d43700 1 -- 172.21.15.2:0/2905950919 learned_addr learned my addr 172.21.15.2:0/2905950919 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:16.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.301+0000 7f211d05b700 1 -- 172.21.15.2:0/1878661486 shutdown_connections 2023-08-25T03:34:16.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.302+0000 7f164fb37700 1 -- 172.21.15.2:0/1577020932 wait complete. 2023-08-25T03:34:16.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.304+0000 7f164fb37700 1 Processor -- start 2023-08-25T03:34:16.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.302+0000 7f74b3fff700 1 -- 172.21.15.2:0/2905950919 >> v1:172.21.15.31:6789/0 conn(0x7f74b40c9db0 legacy=0x7f74b406d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.302+0000 7f74b3fff700 1 -- 172.21.15.2:0/2905950919 >> v1:172.21.15.2:6789/0 conn(0x7f74b40cdf70 legacy=0x7f74b406dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.302+0000 7f74b3fff700 1 -- 172.21.15.2:0/2905950919 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b406e6e0 con 0x7f74b406c160 2023-08-25T03:34:16.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.302+0000 7f211d05b700 1 -- 172.21.15.2:0/1878661486 wait complete. 2023-08-25T03:34:16.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.304+0000 7f211d05b700 1 Processor -- start 2023-08-25T03:34:16.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.304+0000 7f164fb37700 1 -- start start 2023-08-25T03:34:16.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.305+0000 7f164fb37700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1648070bb0 0x7f16480afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.305+0000 7f164fb37700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1648070f90 0x7f164820fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.305+0000 7f164fb37700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16480cdfc0 0x7f1648169910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.305+0000 7f164fb37700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16480c8be0 con 0x7f1648070bb0 2023-08-25T03:34:16.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.305+0000 7f164fb37700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16480b0350 con 0x7f16480cdfc0 2023-08-25T03:34:16.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.305+0000 7f164fb37700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16480c7c80 con 0x7f1648070f90 2023-08-25T03:34:16.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.306+0000 7f3ceffff700 1 -- 172.21.15.2:0/2323808192 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3cf8007470 con 0x7f3d080cfaf0 2023-08-25T03:34:16.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.306+0000 7f3ceffff700 1 --2- 172.21.15.2:0/2323808192 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3cf406a110 0x7f3cf406c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.305+0000 7f211d05b700 1 -- start start 2023-08-25T03:34:16.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.306+0000 7f211d05b700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f21180cad70 0x7f21180c1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.306+0000 7f211d05b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21181570b0 0x7f2118200370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.306+0000 7f211d05b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f211815d580 0x7f2118203650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.306+0000 7f211d05b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21181ff6a0 con 0x7f21181570b0 2023-08-25T03:34:16.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.306+0000 7f211d05b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21182008b0 con 0x7f211815d580 2023-08-25T03:34:16.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.306+0000 7f211d05b700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211814e3c0 con 0x7f21180cad70 2023-08-25T03:34:16.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.306+0000 7f3ceffff700 1 -- 172.21.15.2:0/2323808192 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3cf8087610 con 0x7f3d080cfaf0 2023-08-25T03:34:16.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.306+0000 7f164d0d2700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1648070f90 0x7f164820fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.307+0000 7f164e0d4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16480cdfc0 0x7f1648169910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.307+0000 7f164d0d2700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1648070f90 0x7f164820fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.2:59922/0 (socket says 172.21.15.2:59922) 2023-08-25T03:34:16.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.307+0000 7f164d0d2700 1 -- 172.21.15.2:0/1881449865 learned_addr learned my addr 172.21.15.2:0/1881449865 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:16.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.306+0000 7f2116d9d700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f21180cad70 0x7f21180c1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.306+0000 7f2116d9d700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f21180cad70 0x7f21180c1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.2:59930/0 (socket says 172.21.15.2:59930) 2023-08-25T03:34:16.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.306+0000 7f2116d9d700 1 -- 172.21.15.2:0/2485957011 learned_addr learned my addr 172.21.15.2:0/2485957011 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:16.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.307+0000 7f211759e700 1 --2- 172.21.15.2:0/2485957011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f211815d580 0x7f2118203650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.307+0000 7f3d06ffd700 1 --2- 172.21.15.2:0/2323808192 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3cf406a110 0x7f3cf406c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.307+0000 7f164d0d2700 1 -- 172.21.15.2:0/1881449865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16480cdfc0 msgr2=0x7f1648169910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.307+0000 7f164d0d2700 1 --2- 172.21.15.2:0/1881449865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16480cdfc0 0x7f1648169910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.307+0000 7f164d0d2700 1 -- 172.21.15.2:0/1881449865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1648070bb0 msgr2=0x7f16480afe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.307+0000 7f164d0d2700 1 --2- 172.21.15.2:0/1881449865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1648070bb0 0x7f16480afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.307+0000 7f164d0d2700 1 -- 172.21.15.2:0/1881449865 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1640009a70 con 0x7f1648070f90 2023-08-25T03:34:16.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.307+0000 7f2116d9d700 1 -- 172.21.15.2:0/2485957011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f211815d580 msgr2=0x7f2118203650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.307+0000 7f2116d9d700 1 --2- 172.21.15.2:0/2485957011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f211815d580 0x7f2118203650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.307+0000 7f2116d9d700 1 -- 172.21.15.2:0/2485957011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21181570b0 msgr2=0x7f2118200370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.307+0000 7f2116d9d700 1 --2- 172.21.15.2:0/2485957011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21181570b0 0x7f2118200370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.307+0000 7f2116d9d700 1 -- 172.21.15.2:0/2485957011 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210c009550 con 0x7f21180cad70 2023-08-25T03:34:16.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.307+0000 7f164d0d2700 1 --2- 172.21.15.2:0/1881449865 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1648070f90 0x7f164820fc30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f163800df30 tx=0x7f163800df60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.308+0000 7f2116d9d700 1 --2- 172.21.15.2:0/2485957011 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f21180cad70 0x7f21180c1b10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f211000c8c0 tx=0x7f211000cbd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.308+0000 7f3d06ffd700 1 --2- 172.21.15.2:0/2323808192 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3cf406a110 0x7f3cf406c5d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc000c00 tx=0x7f3cfc008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.308+0000 7f74b3fff700 1 --2- 172.21.15.2:0/2905950919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74b406c160 0x7f74b406c540 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f74a8007f70 tx=0x7f74a8005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d203248dd0a0a2f5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.308+0000 7f20ff7fe700 1 -- 172.21.15.2:0/2485957011 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110003e10 con 0x7f21180cad70 2023-08-25T03:34:16.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.308+0000 7f20ff7fe700 1 -- 172.21.15.2:0/2485957011 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2110005070 con 0x7f21180cad70 2023-08-25T03:34:16.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.308+0000 7f20ff7fe700 1 -- 172.21.15.2:0/2485957011 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110005420 con 0x7f21180cad70 2023-08-25T03:34:16.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.308+0000 7f211d05b700 1 -- 172.21.15.2:0/2485957011 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2118203b90 con 0x7f21180cad70 2023-08-25T03:34:16.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.308+0000 7f211d05b700 1 -- 172.21.15.2:0/2485957011 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21181ff890 con 0x7f21180cad70 2023-08-25T03:34:16.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.309+0000 7f163e7fc700 1 -- 172.21.15.2:0/1881449865 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163800db20 con 0x7f1648070f90 2023-08-25T03:34:16.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.309+0000 7f163e7fc700 1 -- 172.21.15.2:0/1881449865 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16380098f0 con 0x7f1648070f90 2023-08-25T03:34:16.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.309+0000 7f163e7fc700 1 -- 172.21.15.2:0/1881449865 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1638009c60 con 0x7f1648070f90 2023-08-25T03:34:16.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.309+0000 7f164fb37700 1 -- 172.21.15.2:0/1881449865 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16480c7f60 con 0x7f1648070f90 2023-08-25T03:34:16.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.309+0000 7f164fb37700 1 -- 172.21.15.2:0/1881449865 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16480c8520 con 0x7f1648070f90 2023-08-25T03:34:16.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.311+0000 7f163e7fc700 1 -- 172.21.15.2:0/1881449865 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1638014070 con 0x7f1648070f90 2023-08-25T03:34:16.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.311+0000 7f164fb37700 1 -- 172.21.15.2:0/1881449865 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16300052f0 con 0x7f1648070f90 2023-08-25T03:34:16.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.311+0000 7f163e7fc700 1 --2- 172.21.15.2:0/1881449865 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f162c06a1b0 0x7f162c06c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.311+0000 7f163e7fc700 1 -- 172.21.15.2:0/1881449865 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1638088480 con 0x7f1648070f90 2023-08-25T03:34:16.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.312+0000 7f164d8d3700 1 --2- 172.21.15.2:0/1881449865 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f162c06a1b0 0x7f162c06c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.312+0000 7f20ff7fe700 1 -- 172.21.15.2:0/2485957011 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2110005580 con 0x7f21180cad70 2023-08-25T03:34:16.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.312+0000 7f164d8d3700 1 --2- 172.21.15.2:0/1881449865 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f162c06a1b0 0x7f162c06c670 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1644007960 tx=0x7f16440018b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.312+0000 7f20ff7fe700 1 --2- 172.21.15.2:0/2485957011 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f210006a260 0x7f210006c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.312+0000 7f20ff7fe700 1 -- 172.21.15.2:0/2485957011 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f21100877c0 con 0x7f21180cad70 2023-08-25T03:34:16.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.312+0000 7f211659c700 1 --2- 172.21.15.2:0/2485957011 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f210006a260 0x7f210006c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.313+0000 7f3ceffff700 1 -- 172.21.15.2:0/2323808192 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3cf8052c70 con 0x7f3d080cfaf0 2023-08-25T03:34:16.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.313+0000 7f211659c700 1 --2- 172.21.15.2:0/2485957011 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f210006a260 0x7f210006c720 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f211814c8a0 tx=0x7f210c010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.314+0000 7f211d05b700 1 -- 172.21.15.2:0/2485957011 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f80052f0 con 0x7f21180cad70 2023-08-25T03:34:16.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.315+0000 7f74b2ffd700 1 -- 172.21.15.2:0/2905950919 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a8004230 con 0x7f74b406c160 2023-08-25T03:34:16.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.315+0000 7f74b2ffd700 1 -- 172.21.15.2:0/2905950919 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a8004390 con 0x7f74b406c160 2023-08-25T03:34:16.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.315+0000 7f74b2ffd700 1 -- 172.21.15.2:0/2905950919 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a800c820 con 0x7f74b406c160 2023-08-25T03:34:16.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.315+0000 7f74ba7a6700 1 -- 172.21.15.2:0/2905950919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74b406c160 msgr2=0x7f74b406c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.315+0000 7f74ba7a6700 1 --2- 172.21.15.2:0/2905950919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74b406c160 0x7f74b406c540 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f74a8007f70 tx=0x7f74a8005d00 comp rx=0 tx=0).stop 2023-08-25T03:34:16.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.316+0000 7f163e7fc700 1 -- 172.21.15.2:0/1881449865 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1638053b90 con 0x7f1648070f90 2023-08-25T03:34:16.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.317+0000 7f74ba7a6700 1 -- 172.21.15.2:0/2905950919 shutdown_connections 2023-08-25T03:34:16.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.317+0000 7f74ba7a6700 1 --2- 172.21.15.2:0/2905950919 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74b406c160 0x7f74b406c540 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.317+0000 7f74ba7a6700 1 -- 172.21.15.2:0/2905950919 >> 172.21.15.2:0/2905950919 conn(0x7f74b40cf1e0 msgr2=0x7f74b40cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:16.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.317+0000 7f74ba7a6700 1 -- 172.21.15.2:0/2905950919 shutdown_connections 2023-08-25T03:34:16.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.317+0000 7f74ba7a6700 1 -- 172.21.15.2:0/2905950919 wait complete. 2023-08-25T03:34:16.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.319+0000 7f74ba7a6700 1 Processor -- start 2023-08-25T03:34:16.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.319+0000 7f74ba7a6700 1 -- start start 2023-08-25T03:34:16.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.320+0000 7f74ba7a6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74b406c160 0x7f74b4148610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.320+0000 7f74ba7a6700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f74b40c9db0 0x7f74b40c62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.321+0000 7f74b3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74b406c160 0x7f74b4148610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.321+0000 7f74b3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74b406c160 0x7f74b4148610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59694/0 (socket says 172.21.15.2:59694) 2023-08-25T03:34:16.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.321+0000 7f20ff7fe700 1 -- 172.21.15.2:0/2485957011 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2110052eb0 con 0x7f21180cad70 2023-08-25T03:34:16.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.321+0000 7f74ba7a6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b40cdf70 0x7f74b40711a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.321+0000 7f74ba7a6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b40b1d30 con 0x7f74b406c160 2023-08-25T03:34:16.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.321+0000 7f74ba7a6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74b40c8930 con 0x7f74b40cdf70 2023-08-25T03:34:16.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.321+0000 7f74ba7a6700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b40c8e90 con 0x7f74b40c9db0 2023-08-25T03:34:16.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.321+0000 7f74b3fff700 1 -- 172.21.15.2:0/1885264488 learned_addr learned my addr 172.21.15.2:0/1885264488 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:16.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.321+0000 7f74b3fff700 1 -- 172.21.15.2:0/1885264488 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f74b40c9db0 msgr2=0x7f74b40c62b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-08-25T03:34:16.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.321+0000 7f74b3fff700 1 --2- 172.21.15.2:0/1885264488 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f74b40c9db0 0x7f74b40c62b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.321+0000 7f74b3fff700 1 -- 172.21.15.2:0/1885264488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b40cdf70 msgr2=0x7f74b40711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.321+0000 7f74b3fff700 1 --2- 172.21.15.2:0/1885264488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b40cdf70 0x7f74b40711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.321+0000 7f74b3fff700 1 -- 172.21.15.2:0/1885264488 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a8007d50 con 0x7f74b406c160 2023-08-25T03:34:16.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.322+0000 7f74b3fff700 1 --2- 172.21.15.2:0/1885264488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74b406c160 0x7f74b4148610 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f74b406e930 tx=0x7f74a80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.322+0000 7f74b0ff9700 1 -- 172.21.15.2:0/1885264488 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a801e040 con 0x7f74b406c160 2023-08-25T03:34:16.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.322+0000 7f74ba7a6700 1 -- 172.21.15.2:0/1885264488 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74b40c8b60 con 0x7f74b406c160 2023-08-25T03:34:16.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.322+0000 7f74ba7a6700 1 -- 172.21.15.2:0/1885264488 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74b40c7140 con 0x7f74b406c160 2023-08-25T03:34:16.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.322+0000 7f74b0ff9700 1 -- 172.21.15.2:0/1885264488 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74a8003bf0 con 0x7f74b406c160 2023-08-25T03:34:16.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.323+0000 7f74b0ff9700 1 -- 172.21.15.2:0/1885264488 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74a8023480 con 0x7f74b406c160 2023-08-25T03:34:16.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.325+0000 7f74b0ff9700 1 -- 172.21.15.2:0/1885264488 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f74a80235e0 con 0x7f74b406c160 2023-08-25T03:34:16.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.326+0000 7f74b0ff9700 1 --2- 172.21.15.2:0/1885264488 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f74a006a0e0 0x7f74a006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.326+0000 7f74b0ff9700 1 -- 172.21.15.2:0/1885264488 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f74a8089d30 con 0x7f74b406c160 2023-08-25T03:34:16.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.327+0000 7f74b37fe700 1 --2- 172.21.15.2:0/1885264488 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f74a006a0e0 0x7f74a006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.327+0000 7f74ba7a6700 1 -- 172.21.15.2:0/1885264488 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74b406d080 con 0x7f74b406c160 2023-08-25T03:34:16.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.327+0000 7f74b37fe700 1 --2- 172.21.15.2:0/1885264488 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f74a006a0e0 0x7f74a006c5a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f74b4159b80 tx=0x7f74a4007a40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.338+0000 7f74b0ff9700 1 -- 172.21.15.2:0/1885264488 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f74a80554e0 con 0x7f74b406c160 2023-08-25T03:34:16.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.352+0000 7f8ab165b700 1 Processor -- start 2023-08-25T03:34:16.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.352+0000 7f8ab165b700 1 -- start start 2023-08-25T03:34:16.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.353+0000 7f8ab165b700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8aac15f4d0 0x7f8aac0cae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.353+0000 7f8ab165b700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8aac0cb3c0 0x7f8aac0abd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.353+0000 7f8ab165b700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aac0c9510 con 0x7f8aac0ac2d0 2023-08-25T03:34:16.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.353+0000 7f8ab165b700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aac0c97f0 con 0x7f8aac0cb3c0 2023-08-25T03:34:16.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.353+0000 7f8ab165b700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac06ffd0 con 0x7f8aac15f4d0 2023-08-25T03:34:16.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.354+0000 7f8aaa7fc700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8aac0cb3c0 0x7f8aac0abd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.354+0000 7f8aaa7fc700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8aac0cb3c0 0x7f8aac0abd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59704/0 (socket says 172.21.15.2:59704) 2023-08-25T03:34:16.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.354+0000 7f8aaa7fc700 1 -- 172.21.15.2:0/3823135944 learned_addr learned my addr 172.21.15.2:0/3823135944 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:16.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.355+0000 7f8aaa7fc700 1 -- 172.21.15.2:0/3823135944 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8aac15f4d0 msgr2=0x7f8aac0cae80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-08-25T03:34:16.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.355+0000 7f8aaa7fc700 1 --2- 172.21.15.2:0/3823135944 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8aac15f4d0 0x7f8aac0cae80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.355+0000 7f8aaa7fc700 1 -- 172.21.15.2:0/3823135944 >> v1:172.21.15.31:6789/0 conn(0x7f8aac0ac2d0 legacy=0x7f8aac0c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:16.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.355+0000 7f8aaa7fc700 1 -- 172.21.15.2:0/3823135944 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aac0702b0 con 0x7f8aac0cb3c0 2023-08-25T03:34:16.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.355+0000 7f8aaa7fc700 1 --2- 172.21.15.2:0/3823135944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8aac0cb3c0 0x7f8aac0abd90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0098f0 tx=0x7f8a9c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=487d58af189e808a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.356+0000 7f8aa9ffb700 1 -- 172.21.15.2:0/3823135944 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c004230 con 0x7f8aac0cb3c0 2023-08-25T03:34:16.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.356+0000 7f8aa9ffb700 1 -- 172.21.15.2:0/3823135944 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a9c004390 con 0x7f8aac0cb3c0 2023-08-25T03:34:16.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.356+0000 7f8aa9ffb700 1 -- 172.21.15.2:0/3823135944 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c0036a0 con 0x7f8aac0cb3c0 2023-08-25T03:34:16.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.357+0000 7f8ab165b700 1 -- 172.21.15.2:0/3823135944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8aac0cb3c0 msgr2=0x7f8aac0abd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.357+0000 7f8ab165b700 1 --2- 172.21.15.2:0/3823135944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8aac0cb3c0 0x7f8aac0abd90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0098f0 tx=0x7f8a9c009c00 comp rx=0 tx=0).stop 2023-08-25T03:34:16.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.358+0000 7f8ab165b700 1 -- 172.21.15.2:0/3823135944 shutdown_connections 2023-08-25T03:34:16.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.358+0000 7f8ab165b700 1 --2- 172.21.15.2:0/3823135944 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8aac15f4d0 0x7f8aac0cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.358+0000 7f8ab165b700 1 --2- 172.21.15.2:0/3823135944 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8aac0cb3c0 0x7f8aac0abd90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.358+0000 7f8ab165b700 1 -- 172.21.15.2:0/3823135944 >> 172.21.15.2:0/3823135944 conn(0x7f8aac0ceec0 msgr2=0x7f8aac0cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:16.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.358+0000 7f8ab165b700 1 -- 172.21.15.2:0/3823135944 shutdown_connections 2023-08-25T03:34:16.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.358+0000 7f8ab165b700 1 -- 172.21.15.2:0/3823135944 wait complete. 2023-08-25T03:34:16.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.360+0000 7f8ab165b700 1 Processor -- start 2023-08-25T03:34:16.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.360+0000 7f8ab165b700 1 -- start start 2023-08-25T03:34:16.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.362+0000 7f8ab165b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8aac0ac2d0 0x7f8aac0af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.362+0000 7f8ab165b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8aac0cb3c0 0x7f8aac0afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.362+0000 7f8ab165b700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8aac15f4d0 0x7f8aac1459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.362+0000 7f8ab165b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac14cfb0 con 0x7f8aac0ac2d0 2023-08-25T03:34:16.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.362+0000 7f8ab165b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac14ca00 con 0x7f8aac0cb3c0 2023-08-25T03:34:16.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.362+0000 7f8ab165b700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac0b02d0 con 0x7f8aac15f4d0 2023-08-25T03:34:16.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.362+0000 7f8aaaffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8aac0ac2d0 0x7f8aac0af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.362+0000 7f8aaaffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8aac0ac2d0 0x7f8aac0af330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59708/0 (socket says 172.21.15.2:59708) 2023-08-25T03:34:16.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.362+0000 7f8aaaffd700 1 -- 172.21.15.2:0/1062395700 learned_addr learned my addr 172.21.15.2:0/1062395700 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:16.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.363+0000 7f8aaaffd700 1 -- 172.21.15.2:0/1062395700 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8aac15f4d0 msgr2=0x7f8aac1459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.363+0000 7f8aaaffd700 1 --2- 172.21.15.2:0/1062395700 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8aac15f4d0 0x7f8aac1459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.363+0000 7f8aaaffd700 1 -- 172.21.15.2:0/1062395700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8aac0cb3c0 msgr2=0x7f8aac0afd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.363+0000 7f8aaaffd700 1 --2- 172.21.15.2:0/1062395700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8aac0cb3c0 0x7f8aac0afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.363+0000 7f8aaaffd700 1 -- 172.21.15.2:0/1062395700 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9c009550 con 0x7f8aac0ac2d0 2023-08-25T03:34:16.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.363+0000 7f8aaaffd700 1 --2- 172.21.15.2:0/1062395700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8aac0ac2d0 0x7f8aac0af330 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000b470 tx=0x7f8aa000b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.371+0000 7f8a937fe700 1 -- 172.21.15.2:0/1062395700 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0011830 con 0x7f8aac0ac2d0 2023-08-25T03:34:16.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.371+0000 7f8a937fe700 1 -- 172.21.15.2:0/1062395700 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa00119b0 con 0x7f8aac0ac2d0 2023-08-25T03:34:16.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.372+0000 7f8a937fe700 1 -- 172.21.15.2:0/1062395700 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0011d60 con 0x7f8aac0ac2d0 2023-08-25T03:34:16.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.372+0000 7f8ab165b700 1 -- 172.21.15.2:0/1062395700 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aac14be10 con 0x7f8aac0ac2d0 2023-08-25T03:34:16.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.372+0000 7f8ab165b700 1 -- 172.21.15.2:0/1062395700 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aac14c3b0 con 0x7f8aac0ac2d0 2023-08-25T03:34:16.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.376+0000 7f8a937fe700 1 -- 172.21.15.2:0/1062395700 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8aa001a400 con 0x7f8aac0ac2d0 2023-08-25T03:34:16.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.377+0000 7f8a937fe700 1 --2- 172.21.15.2:0/1062395700 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8a9406a310 0x7f8a9406c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.377+0000 7f8a937fe700 1 -- 172.21.15.2:0/1062395700 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8aa0088600 con 0x7f8aac0ac2d0 2023-08-25T03:34:16.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.377+0000 7f8aaa7fc700 1 --2- 172.21.15.2:0/1062395700 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8a9406a310 0x7f8a9406c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.378+0000 7f8aaa7fc700 1 --2- 172.21.15.2:0/1062395700 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8a9406a310 0x7f8a9406c7d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8aac099c80 tx=0x7f8a9c010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.379+0000 7f8ab165b700 1 -- 172.21.15.2:0/1062395700 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a8c0052f0 con 0x7f8aac0ac2d0 2023-08-25T03:34:16.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.389+0000 7f8a937fe700 1 -- 172.21.15.2:0/1062395700 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8aa0053cd0 con 0x7f8aac0ac2d0 2023-08-25T03:34:16.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.673+0000 7f090919f700 1 -- 172.21.15.2:0/2287035650 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f08e4005870 con 0x7f09041570b0 2023-08-25T03:34:16.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.677+0000 7f08eb7fe700 1 -- 172.21.15.2:0/2287035650 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f08f8017090 con 0x7f09041570b0 2023-08-25T03:34:16.678 INFO:teuthology.orchestra.run.smithi002.stdout:55834574857 2023-08-25T03:34:16.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.678+0000 7facf301c700 1 -- 172.21.15.2:0/1065260405 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7facec16d470 con 0x7facec08a8f0 2023-08-25T03:34:16.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.679+0000 7face97fa700 1 -- 172.21.15.2:0/1065260405 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7face0015090 con 0x7facec08a8f0 2023-08-25T03:34:16.681 INFO:teuthology.orchestra.run.smithi002.stdout:60129542154 2023-08-25T03:34:16.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.681+0000 7f090919f700 1 -- 172.21.15.2:0/2287035650 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f08f006a1b0 msgr2=0x7f08f006c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.681+0000 7f090919f700 1 --2- 172.21.15.2:0/2287035650 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f08f006a1b0 0x7f08f006c670 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f08f400d370 tx=0x7f08f4012000 comp rx=0 tx=0).stop 2023-08-25T03:34:16.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.681+0000 7f090919f700 1 -- 172.21.15.2:0/2287035650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09041570b0 msgr2=0x7f090406cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.681+0000 7f090919f700 1 --2- 172.21.15.2:0/2287035650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09041570b0 0x7f090406cdf0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f08f800c4e0 tx=0x7f08f800c8a0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.681+0000 7f090919f700 1 -- 172.21.15.2:0/2287035650 shutdown_connections 2023-08-25T03:34:16.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.681+0000 7f090919f700 1 --2- 172.21.15.2:0/2287035650 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f090415d750 0x7f09041fef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.681+0000 7f090919f700 1 --2- 172.21.15.2:0/2287035650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09040cad70 0x7f09040c5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.681+0000 7f090919f700 1 --2- 172.21.15.2:0/2287035650 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f08f006a1b0 0x7f08f006c670 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.681+0000 7f090919f700 1 --2- 172.21.15.2:0/2287035650 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f09041570b0 0x7f090406cdf0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.681+0000 7f090919f700 1 -- 172.21.15.2:0/2287035650 >> 172.21.15.2:0/2287035650 conn(0x7f09040ceec0 msgr2=0x7f09040ad1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:16.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.682+0000 7f090919f700 1 -- 172.21.15.2:0/2287035650 shutdown_connections 2023-08-25T03:34:16.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.682+0000 7f090919f700 1 -- 172.21.15.2:0/2287035650 wait complete. 2023-08-25T03:34:16.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.683+0000 7facf301c700 1 -- 172.21.15.2:0/1065260405 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7facd806a0e0 msgr2=0x7facd806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.683+0000 7facf301c700 1 --2- 172.21.15.2:0/1065260405 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7facd806a0e0 0x7facd806c5a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7face400b370 tx=0x7face4010000 comp rx=0 tx=0).stop 2023-08-25T03:34:16.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.683+0000 7facf301c700 1 -- 172.21.15.2:0/1065260405 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facec08a8f0 msgr2=0x7facec0aef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.683+0000 7facf301c700 1 --2- 172.21.15.2:0/1065260405 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facec08a8f0 0x7facec0aef50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7face000b550 tx=0x7face000b910 comp rx=0 tx=0).stop 2023-08-25T03:34:16.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.683+0000 7facf301c700 1 -- 172.21.15.2:0/1065260405 shutdown_connections 2023-08-25T03:34:16.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.683+0000 7facf301c700 1 --2- 172.21.15.2:0/1065260405 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7facd806a0e0 0x7facd806c5a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.683+0000 7facf301c700 1 --2- 172.21.15.2:0/1065260405 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7facec08a8f0 0x7facec0aef50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.683+0000 7facf301c700 1 --2- 172.21.15.2:0/1065260405 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7facec070940 0x7facec099a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.683+0000 7facf301c700 1 --2- 172.21.15.2:0/1065260405 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7facec071030 0x7facec210770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.683+0000 7facf301c700 1 -- 172.21.15.2:0/1065260405 >> 172.21.15.2:0/1065260405 conn(0x7facec0ceec0 msgr2=0x7facec06dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:16.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.683+0000 7facf301c700 1 -- 172.21.15.2:0/1065260405 shutdown_connections 2023-08-25T03:34:16.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.684+0000 7facf301c700 1 -- 172.21.15.2:0/1065260405 wait complete. 2023-08-25T03:34:16.705 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.1 2023-08-25T03:34:16.706 DEBUG:teuthology.parallel:result is None 2023-08-25T03:34:16.706 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542154 for osd.6 2023-08-25T03:34:16.707 DEBUG:teuthology.parallel:result is None 2023-08-25T03:34:16.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.717+0000 7f369368f700 1 -- 172.21.15.2:0/3372883984 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f368c0ae170 con 0x7f368c06e190 2023-08-25T03:34:16.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.718+0000 7f3681ffb700 1 -- 172.21.15.2:0/3372883984 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f36840582e0 con 0x7f368c06e190 2023-08-25T03:34:16.719 INFO:teuthology.orchestra.run.smithi002.stdout:55834574856 2023-08-25T03:34:16.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.720+0000 7f369368f700 1 -- 172.21.15.2:0/3372883984 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f367006a0e0 msgr2=0x7f367006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.720+0000 7f369368f700 1 --2- 172.21.15.2:0/3372883984 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f367006a0e0 0x7f367006c5a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f368800a010 tx=0x7f368800c000 comp rx=0 tx=0).stop 2023-08-25T03:34:16.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.720+0000 7f369368f700 1 -- 172.21.15.2:0/3372883984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f368c06e190 msgr2=0x7f368c09aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.720+0000 7f369368f700 1 --2- 172.21.15.2:0/3372883984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f368c06e190 0x7f368c09aab0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3684001460 tx=0x7f36840058d0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.720+0000 7f369368f700 1 -- 172.21.15.2:0/3372883984 shutdown_connections 2023-08-25T03:34:16.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.720+0000 7f369368f700 1 --2- 172.21.15.2:0/3372883984 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f367006a0e0 0x7f367006c5a0 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f368800a010 tx=0x7f368800c000 comp rx=0 tx=0).stop 2023-08-25T03:34:16.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.720+0000 7f369368f700 1 --2- 172.21.15.2:0/3372883984 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f368c06e190 0x7f368c09aab0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.720+0000 7f369368f700 1 --2- 172.21.15.2:0/3372883984 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3678003d70 0x7f368c16bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.720+0000 7f369368f700 1 --2- 172.21.15.2:0/3372883984 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f368c09c270 0x7f368c06dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.720+0000 7f369368f700 1 -- 172.21.15.2:0/3372883984 >> 172.21.15.2:0/3372883984 conn(0x7f368c0ceec0 msgr2=0x7f368c0c9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:16.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.721+0000 7f369368f700 1 -- 172.21.15.2:0/3372883984 shutdown_connections 2023-08-25T03:34:16.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.723+0000 7f369368f700 1 -- 172.21.15.2:0/3372883984 wait complete. 2023-08-25T03:34:16.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.729+0000 7f164fb37700 1 -- 172.21.15.2:0/1881449865 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f16300055a0 con 0x7f1648070f90 2023-08-25T03:34:16.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.729+0000 7f3d0e016700 1 -- 172.21.15.2:0/2323808192 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f3d08165750 con 0x7f3d080cfaf0 2023-08-25T03:34:16.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.729+0000 7f3ceffff700 1 -- 172.21.15.2:0/2323808192 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3d08165750 con 0x7f3d080cfaf0 2023-08-25T03:34:16.731 INFO:teuthology.orchestra.run.smithi002.stdout:60129542154 2023-08-25T03:34:16.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.729+0000 7f163e7fc700 1 -- 172.21.15.2:0/1881449865 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f16380571b0 con 0x7f1648070f90 2023-08-25T03:34:16.733 INFO:teuthology.orchestra.run.smithi002.stdout:60129542154 2023-08-25T03:34:16.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.731+0000 7f3d0e016700 1 -- 172.21.15.2:0/2323808192 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3cf406a110 msgr2=0x7f3cf406c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.731+0000 7f3d0e016700 1 --2- 172.21.15.2:0/2323808192 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3cf406a110 0x7f3cf406c5d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc000c00 tx=0x7f3cfc008030 comp rx=0 tx=0).stop 2023-08-25T03:34:16.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.731+0000 7f3d0e016700 1 -- 172.21.15.2:0/2323808192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d080cfaf0 msgr2=0x7f3d08162ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.731+0000 7f3d0e016700 1 --2- 172.21.15.2:0/2323808192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d080cfaf0 0x7f3d08162ac0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800c710 tx=0x7f3cf800ca20 comp rx=0 tx=0).stop 2023-08-25T03:34:16.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.731+0000 7f3d0e016700 1 -- 172.21.15.2:0/2323808192 shutdown_connections 2023-08-25T03:34:16.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.731+0000 7f3d0e016700 1 --2- 172.21.15.2:0/2323808192 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3cf406a110 0x7f3cf406c5d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.731+0000 7f3d0e016700 1 --2- 172.21.15.2:0/2323808192 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3d080cfaf0 0x7f3d08162ac0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.731+0000 7f3d0e016700 1 --2- 172.21.15.2:0/2323808192 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d08161560 0x7f3d0815fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.731+0000 7f3d0e016700 1 --2- 172.21.15.2:0/2323808192 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d080d2b80 0x7f3d08163000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.732+0000 7f3d0e016700 1 -- 172.21.15.2:0/2323808192 >> 172.21.15.2:0/2323808192 conn(0x7f3d0806abe0 msgr2=0x7f3d08164490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:16.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.732+0000 7f3d0e016700 1 -- 172.21.15.2:0/2323808192 shutdown_connections 2023-08-25T03:34:16.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.732+0000 7f3d0e016700 1 -- 172.21.15.2:0/2323808192 wait complete. 2023-08-25T03:34:16.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.733+0000 7f164fb37700 1 -- 172.21.15.2:0/1881449865 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f162c06a1b0 msgr2=0x7f162c06c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.733+0000 7f164fb37700 1 --2- 172.21.15.2:0/1881449865 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f162c06a1b0 0x7f162c06c670 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1644007960 tx=0x7f16440018b0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.733+0000 7f164fb37700 1 -- 172.21.15.2:0/1881449865 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1648070f90 msgr2=0x7f164820fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.733+0000 7f164fb37700 1 --2- 172.21.15.2:0/1881449865 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1648070f90 0x7f164820fc30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f163800df30 tx=0x7f163800df60 comp rx=0 tx=0).stop 2023-08-25T03:34:16.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.733+0000 7f164fb37700 1 -- 172.21.15.2:0/1881449865 shutdown_connections 2023-08-25T03:34:16.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.733+0000 7f164fb37700 1 --2- 172.21.15.2:0/1881449865 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f162c06a1b0 0x7f162c06c670 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.733+0000 7f164fb37700 1 --2- 172.21.15.2:0/1881449865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1648070bb0 0x7f16480afe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.733+0000 7f164fb37700 1 --2- 172.21.15.2:0/1881449865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16480cdfc0 0x7f1648169910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.733+0000 7f164fb37700 1 --2- 172.21.15.2:0/1881449865 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1648070f90 0x7f164820fc30 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.733+0000 7f164fb37700 1 -- 172.21.15.2:0/1881449865 >> 172.21.15.2:0/1881449865 conn(0x7f16480ceec0 msgr2=0x7f16480c4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:16.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.733+0000 7f164fb37700 1 -- 172.21.15.2:0/1881449865 shutdown_connections 2023-08-25T03:34:16.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.733+0000 7f164fb37700 1 -- 172.21.15.2:0/1881449865 wait complete. 2023-08-25T03:34:16.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.740+0000 7f211d05b700 1 -- 172.21.15.2:0/2485957011 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f20f8005870 con 0x7f21180cad70 2023-08-25T03:34:16.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.740+0000 7f20ff7fe700 1 -- 172.21.15.2:0/2485957011 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2110017070 con 0x7f21180cad70 2023-08-25T03:34:16.741 INFO:teuthology.orchestra.run.smithi002.stdout:60129542153 2023-08-25T03:34:16.742 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.2 2023-08-25T03:34:16.743 DEBUG:teuthology.parallel:result is None 2023-08-25T03:34:16.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.744+0000 7f211d05b700 1 -- 172.21.15.2:0/2485957011 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f210006a260 msgr2=0x7f210006c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.744+0000 7f211d05b700 1 --2- 172.21.15.2:0/2485957011 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f210006a260 0x7f210006c720 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f211814c8a0 tx=0x7f210c010000 comp rx=0 tx=0).stop 2023-08-25T03:34:16.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.744+0000 7f211d05b700 1 -- 172.21.15.2:0/2485957011 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f21180cad70 msgr2=0x7f21180c1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.744+0000 7f211d05b700 1 --2- 172.21.15.2:0/2485957011 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f21180cad70 0x7f21180c1b10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f211000c8c0 tx=0x7f211000cbd0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.744+0000 7f211d05b700 1 -- 172.21.15.2:0/2485957011 shutdown_connections 2023-08-25T03:34:16.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.744+0000 7f211d05b700 1 --2- 172.21.15.2:0/2485957011 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f211815d580 0x7f2118203650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.744+0000 7f211d05b700 1 --2- 172.21.15.2:0/2485957011 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f21180cad70 0x7f21180c1b10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.744+0000 7f211d05b700 1 --2- 172.21.15.2:0/2485957011 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f210006a260 0x7f210006c720 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.744+0000 7f211d05b700 1 --2- 172.21.15.2:0/2485957011 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f21181570b0 0x7f2118200370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.744+0000 7f211d05b700 1 -- 172.21.15.2:0/2485957011 >> 172.21.15.2:0/2485957011 conn(0x7f21180ceec0 msgr2=0x7f21180cf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:16.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.744+0000 7f211d05b700 1 -- 172.21.15.2:0/2485957011 shutdown_connections 2023-08-25T03:34:16.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.744+0000 7f211d05b700 1 -- 172.21.15.2:0/2485957011 wait complete. 2023-08-25T03:34:16.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.745+0000 7f74ba7a6700 1 -- 172.21.15.2:0/1885264488 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f74b4072430 con 0x7f74b406c160 2023-08-25T03:34:16.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.746+0000 7f74b0ff9700 1 -- 172.21.15.2:0/1885264488 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f74a8058b00 con 0x7f74b406c160 2023-08-25T03:34:16.748 INFO:teuthology.orchestra.run.smithi002.stdout:55834574856 2023-08-25T03:34:16.750 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542154 for osd.4 2023-08-25T03:34:16.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.749+0000 7f74ba7a6700 1 -- 172.21.15.2:0/1885264488 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f74a006a0e0 msgr2=0x7f74a006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.749+0000 7f74ba7a6700 1 --2- 172.21.15.2:0/1885264488 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f74a006a0e0 0x7f74a006c5a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f74b4159b80 tx=0x7f74a4007a40 comp rx=0 tx=0).stop 2023-08-25T03:34:16.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.749+0000 7f74ba7a6700 1 -- 172.21.15.2:0/1885264488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74b406c160 msgr2=0x7f74b4148610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.749+0000 7f74ba7a6700 1 --2- 172.21.15.2:0/1885264488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74b406c160 0x7f74b4148610 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f74b406e930 tx=0x7f74a80047c0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.751 DEBUG:teuthology.parallel:result is None 2023-08-25T03:34:16.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.753+0000 7f74ba7a6700 1 -- 172.21.15.2:0/1885264488 shutdown_connections 2023-08-25T03:34:16.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.753+0000 7f74ba7a6700 1 --2- 172.21.15.2:0/1885264488 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f74a006a0e0 0x7f74a006c5a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.753+0000 7f74ba7a6700 1 --2- 172.21.15.2:0/1885264488 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f74b406c160 0x7f74b4148610 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.753+0000 7f74ba7a6700 1 --2- 172.21.15.2:0/1885264488 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74b40cdf70 0x7f74b40711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.753+0000 7f74ba7a6700 1 --2- 172.21.15.2:0/1885264488 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f74b40c9db0 0x7f74b40c62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.753+0000 7f74ba7a6700 1 -- 172.21.15.2:0/1885264488 >> 172.21.15.2:0/1885264488 conn(0x7f74b40cf1e0 msgr2=0x7f74b40af260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:16.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.753+0000 7f74ba7a6700 1 -- 172.21.15.2:0/1885264488 shutdown_connections 2023-08-25T03:34:16.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.753+0000 7f74ba7a6700 1 -- 172.21.15.2:0/1885264488 wait complete. 2023-08-25T03:34:16.756 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542153 got 60129542154 for osd.7 2023-08-25T03:34:16.756 DEBUG:teuthology.parallel:result is None 2023-08-25T03:34:16.760 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542152 got 60129542153 for osd.5 2023-08-25T03:34:16.761 DEBUG:teuthology.parallel:result is None 2023-08-25T03:34:16.767 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.0 2023-08-25T03:34:16.767 DEBUG:teuthology.parallel:result is None 2023-08-25T03:34:16.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.781+0000 7f8ab165b700 1 -- 172.21.15.2:0/1062395700 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f8a8c0055a0 con 0x7f8aac0ac2d0 2023-08-25T03:34:16.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.782+0000 7f8a937fe700 1 -- 172.21.15.2:0/1062395700 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8aa0015070 con 0x7f8aac0ac2d0 2023-08-25T03:34:16.783 INFO:teuthology.orchestra.run.smithi002.stdout:55834574857 2023-08-25T03:34:16.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.784+0000 7f8ab165b700 1 -- 172.21.15.2:0/1062395700 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8a9406a310 msgr2=0x7f8a9406c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.784+0000 7f8ab165b700 1 --2- 172.21.15.2:0/1062395700 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8a9406a310 0x7f8a9406c7d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8aac099c80 tx=0x7f8a9c010000 comp rx=0 tx=0).stop 2023-08-25T03:34:16.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.784+0000 7f8ab165b700 1 -- 172.21.15.2:0/1062395700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8aac0ac2d0 msgr2=0x7f8aac0af330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.784+0000 7f8ab165b700 1 --2- 172.21.15.2:0/1062395700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8aac0ac2d0 0x7f8aac0af330 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000b470 tx=0x7f8aa000b830 comp rx=0 tx=0).stop 2023-08-25T03:34:16.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.784+0000 7f8ab165b700 1 -- 172.21.15.2:0/1062395700 shutdown_connections 2023-08-25T03:34:16.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.784+0000 7f8ab165b700 1 --2- 172.21.15.2:0/1062395700 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8a9406a310 0x7f8a9406c7d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.784+0000 7f8ab165b700 1 --2- 172.21.15.2:0/1062395700 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8aac0ac2d0 0x7f8aac0af330 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.784+0000 7f8ab165b700 1 --2- 172.21.15.2:0/1062395700 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8aac15f4d0 0x7f8aac1459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.784+0000 7f8ab165b700 1 --2- 172.21.15.2:0/1062395700 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8aac0cb3c0 0x7f8aac0afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.784+0000 7f8ab165b700 1 -- 172.21.15.2:0/1062395700 >> 172.21.15.2:0/1062395700 conn(0x7f8aac0ceec0 msgr2=0x7f8aac0c6820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:16.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.784+0000 7f8ab165b700 1 -- 172.21.15.2:0/1062395700 shutdown_connections 2023-08-25T03:34:16.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.784+0000 7f8ab165b700 1 -- 172.21.15.2:0/1062395700 wait complete. 2023-08-25T03:34:16.797 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.3 2023-08-25T03:34:16.797 DEBUG:teuthology.parallel:result is None 2023-08-25T03:34:16.798 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-08-25T03:34:16.798 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T03:34:16.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.961+0000 7f4e74855700 1 Processor -- start 2023-08-25T03:34:16.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.961+0000 7f4e74855700 1 -- start start 2023-08-25T03:34:16.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.962+0000 7f4e74855700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4e700b19a0 0x7f4e7015e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.962+0000 7f4e74855700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4e700caea0 0x7f4e700cb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.962+0000 7f4e74855700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e70071b40 con 0x7f4e700b1d80 2023-08-25T03:34:16.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.962+0000 7f4e74855700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e700720b0 con 0x7f4e700caea0 2023-08-25T03:34:16.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.962+0000 7f4e74855700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e70071dd0 con 0x7f4e700b19a0 2023-08-25T03:34:16.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.962+0000 7f4e6dd9b700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f4e700b1d80 0x7f4e70071430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46384/0 (socket says 172.21.15.2:46384) 2023-08-25T03:34:16.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.962+0000 7f4e6dd9b700 1 -- 172.21.15.2:0/3409599104 learned_addr learned my addr 172.21.15.2:0/3409599104 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:16.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.962+0000 7f4e6ed9d700 1 --2- 172.21.15.2:0/3409599104 >> v2:172.21.15.31:3300/0 conn(0x7f4e700caea0 0x7f4e700cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.962+0000 7f4e6d59a700 1 -- 172.21.15.2:0/3409599104 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190906769 0 0) 0x7f4e70071b40 con 0x7f4e700b1d80 2023-08-25T03:34:16.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.962+0000 7f4e6e59c700 1 --2- 172.21.15.2:0/3409599104 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4e700b19a0 0x7f4e7015e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.962+0000 7f4e6d59a700 1 -- 172.21.15.2:0/3409599104 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e580057c0 con 0x7f4e700b1d80 2023-08-25T03:34:16.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.962+0000 7f4e6d59a700 1 -- 172.21.15.2:0/3409599104 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1313451099 0 0) 0x7f4e580057c0 con 0x7f4e700b1d80 2023-08-25T03:34:16.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.962+0000 7f4e6d59a700 1 -- 172.21.15.2:0/3409599104 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4e700b19a0 msgr2=0x7f4e7015e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:16.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.962+0000 7f4e6d59a700 1 --2- 172.21.15.2:0/3409599104 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4e700b19a0 0x7f4e7015e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.962+0000 7f4e6d59a700 1 -- 172.21.15.2:0/3409599104 >> v2:172.21.15.31:3300/0 conn(0x7f4e700caea0 msgr2=0x7f4e700cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:16.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.962+0000 7f4e6d59a700 1 --2- 172.21.15.2:0/3409599104 >> v2:172.21.15.31:3300/0 conn(0x7f4e700caea0 0x7f4e700cb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.962+0000 7f4e6d59a700 1 -- 172.21.15.2:0/3409599104 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7015ddb0 con 0x7f4e700b1d80 2023-08-25T03:34:16.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.962+0000 7f4e6d59a700 1 -- 172.21.15.2:0/3409599104 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f4e64001670 con 0x7f4e700b1d80 2023-08-25T03:34:16.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.962+0000 7f4e6d59a700 1 -- 172.21.15.2:0/3409599104 >> v1:172.21.15.2:6789/0 conn(0x7f4e700b1d80 legacy=0x7f4e70071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.962+0000 7f4e6d59a700 1 --2- 172.21.15.2:0/3409599104 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4e700b19a0 0x7f4e58005c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.962+0000 7f4e6d59a700 1 --2- 172.21.15.2:0/3409599104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e58006190 0x7f4e58006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.962+0000 7f4e6d59a700 1 --2- 172.21.15.2:0/3409599104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e58006fa0 0x7f4e58007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.962+0000 7f4e6d59a700 1 -- 172.21.15.2:0/3409599104 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e700720b0 con 0x7f4e58006190 2023-08-25T03:34:16.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.962+0000 7f4e6d59a700 1 -- 172.21.15.2:0/3409599104 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e70071dd0 con 0x7f4e58006fa0 2023-08-25T03:34:16.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.963+0000 7f4e6d59a700 1 -- 172.21.15.2:0/3409599104 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5800dc70 con 0x7f4e700b19a0 2023-08-25T03:34:16.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.963+0000 7f4e6dd9b700 1 --2- 172.21.15.2:0/3409599104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e58006190 0x7f4e58006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.963+0000 7f4e6e59c700 1 --2- 172.21.15.2:0/3409599104 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4e700b19a0 0x7f4e58005c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.963+0000 7f4e6ed9d700 1 --2- 172.21.15.2:0/3409599104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e58006fa0 0x7f4e58007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.963+0000 7f4e6dd9b700 1 -- 172.21.15.2:0/3409599104 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4e700b19a0 msgr2=0x7f4e58005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:16.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.963+0000 7f4e6dd9b700 1 --2- 172.21.15.2:0/3409599104 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4e700b19a0 0x7f4e58005c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.963+0000 7f4e6dd9b700 1 -- 172.21.15.2:0/3409599104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e58006fa0 msgr2=0x7f4e58007420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:16.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.963+0000 7f4e6dd9b700 1 --2- 172.21.15.2:0/3409599104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e58006fa0 0x7f4e58007420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.963+0000 7f4e6dd9b700 1 -- 172.21.15.2:0/3409599104 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e580057c0 con 0x7f4e58006190 2023-08-25T03:34:16.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.963+0000 7f4e6dd9b700 1 --2- 172.21.15.2:0/3409599104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e58006190 0x7f4e58006a60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400c3a0 tx=0x7f4e58007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b82eb5e9266087 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.963+0000 7f4e6d59a700 1 -- 172.21.15.2:0/3409599104 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64007c60 con 0x7f4e58006190 2023-08-25T03:34:16.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.963+0000 7f4e6d59a700 1 -- 172.21.15.2:0/3409599104 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e64005960 con 0x7f4e58006190 2023-08-25T03:34:16.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.963+0000 7f4e6d59a700 1 -- 172.21.15.2:0/3409599104 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6400cea0 con 0x7f4e58006190 2023-08-25T03:34:16.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.964+0000 7f4e74855700 1 -- 172.21.15.2:0/3409599104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e58006190 msgr2=0x7f4e58006a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.964+0000 7f4e74855700 1 --2- 172.21.15.2:0/3409599104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e58006190 0x7f4e58006a60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4e6400c3a0 tx=0x7f4e58007a90 comp rx=0 tx=0).stop 2023-08-25T03:34:16.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.964+0000 7f4e6ed9d700 1 -- 172.21.15.2:0/3409599104 reap_dead start 2023-08-25T03:34:16.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.964+0000 7f4e74855700 1 -- 172.21.15.2:0/3409599104 shutdown_connections 2023-08-25T03:34:16.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.964+0000 7f4e74855700 1 -- 172.21.15.2:0/3409599104 >> 172.21.15.2:0/3409599104 conn(0x7f4e700cfea0 msgr2=0x7f4e700d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:16.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.964+0000 7f4e74855700 1 -- 172.21.15.2:0/3409599104 shutdown_connections 2023-08-25T03:34:16.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.964+0000 7f4e74855700 1 -- 172.21.15.2:0/3409599104 wait complete. 2023-08-25T03:34:16.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.965+0000 7f4e74855700 1 Processor -- start 2023-08-25T03:34:16.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.965+0000 7f4e74855700 1 -- start start 2023-08-25T03:34:16.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.966+0000 7f4e74855700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4e58006190 0x7f4e7015c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.966+0000 7f4e74855700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e700c9880 0x7f4e70153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.966+0000 7f4e74855700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e700c88d0 0x7f4e7006b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.966+0000 7f4e74855700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e700c9d00 con 0x7f4e700c88d0 2023-08-25T03:34:16.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.966+0000 7f4e74855700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7015caa0 con 0x7f4e700c9880 2023-08-25T03:34:16.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.966+0000 7f4e74855700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7015cd80 con 0x7f4e58006190 2023-08-25T03:34:16.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.966+0000 7f4e6ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e700c88d0 0x7f4e7006b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.966+0000 7f4e6ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e700c88d0 0x7f4e7006b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59732/0 (socket says 172.21.15.2:59732) 2023-08-25T03:34:16.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.966+0000 7f4e6ed9d700 1 -- 172.21.15.2:0/477080804 learned_addr learned my addr 172.21.15.2:0/477080804 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:16.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.966+0000 7f4e6dd9b700 1 --2- 172.21.15.2:0/477080804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e700c9880 0x7f4e70153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.966+0000 7f4e6e59c700 1 --2- 172.21.15.2:0/477080804 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4e58006190 0x7f4e7015c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.966+0000 7f4e6ed9d700 1 -- 172.21.15.2:0/477080804 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4e58006190 msgr2=0x7f4e7015c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.966+0000 7f4e6ed9d700 1 --2- 172.21.15.2:0/477080804 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4e58006190 0x7f4e7015c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.966+0000 7f4e6ed9d700 1 -- 172.21.15.2:0/477080804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e700c9880 msgr2=0x7f4e70153d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:16.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.966+0000 7f4e6ed9d700 1 --2- 172.21.15.2:0/477080804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e700c9880 0x7f4e70153d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:16.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.966+0000 7f4e6ed9d700 1 -- 172.21.15.2:0/477080804 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e700c80d0 con 0x7f4e700c88d0 2023-08-25T03:34:16.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.966+0000 7f4e6ed9d700 1 --2- 172.21.15.2:0/477080804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e700c88d0 0x7f4e7006b390 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4e7006d300 tx=0x7f4e68011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.967+0000 7f4e5effd700 1 -- 172.21.15.2:0/477080804 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6800e040 con 0x7f4e700c88d0 2023-08-25T03:34:16.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.967+0000 7f4e5effd700 1 -- 172.21.15.2:0/477080804 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6800f040 con 0x7f4e700c88d0 2023-08-25T03:34:16.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.967+0000 7f4e5effd700 1 -- 172.21.15.2:0/477080804 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6800fc40 con 0x7f4e700c88d0 2023-08-25T03:34:16.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.967+0000 7f4e74855700 1 -- 172.21.15.2:0/477080804 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e700c83b0 con 0x7f4e700c88d0 2023-08-25T03:34:16.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.967+0000 7f4e74855700 1 -- 172.21.15.2:0/477080804 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e700c8d80 con 0x7f4e700c88d0 2023-08-25T03:34:16.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.967+0000 7f4e74855700 1 -- 172.21.15.2:0/477080804 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e70066f70 con 0x7f4e700c88d0 2023-08-25T03:34:16.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.968+0000 7f4e5effd700 1 -- 172.21.15.2:0/477080804 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4e68022460 con 0x7f4e700c88d0 2023-08-25T03:34:16.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.968+0000 7f4e5effd700 1 --2- 172.21.15.2:0/477080804 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f4e5006a0e0 0x7f4e5006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:16.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.968+0000 7f4e5effd700 1 -- 172.21.15.2:0/477080804 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4e68088a50 con 0x7f4e700c88d0 2023-08-25T03:34:16.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.968+0000 7f4e6e59c700 1 --2- 172.21.15.2:0/477080804 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f4e5006a0e0 0x7f4e5006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:16.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.969+0000 7f4e6e59c700 1 --2- 172.21.15.2:0/477080804 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f4e5006a0e0 0x7f4e5006c5a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009b00 tx=0x7f4e60009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:16.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:16.973+0000 7f4e5effd700 1 -- 172.21.15.2:0/477080804 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4e68057bd0 con 0x7f4e700c88d0 2023-08-25T03:34:17.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.168+0000 7f4e74855700 1 -- 172.21.15.2:0/477080804 --> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4e7015e370 con 0x7f4e5006a0e0 2023-08-25T03:34:17.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.177+0000 7f4e5effd700 1 -- 172.21.15.2:0/477080804 <== mgr.4108 v2:172.21.15.2:6832/3528779779 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286680 (secure 0 0 0) 0x7f4e60019070 con 0x7f4e5006a0e0 2023-08-25T03:34:17.181 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:17.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.180+0000 7f4e74855700 1 -- 172.21.15.2:0/477080804 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f4e5006a0e0 msgr2=0x7f4e5006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:17.184 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.180+0000 7f4e74855700 1 --2- 172.21.15.2:0/477080804 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f4e5006a0e0 0x7f4e5006c5a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4e60009b00 tx=0x7f4e60009210 comp rx=0 tx=0).stop 2023-08-25T03:34:17.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.180+0000 7f4e74855700 1 -- 172.21.15.2:0/477080804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e700c88d0 msgr2=0x7f4e7006b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:17.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.180+0000 7f4e74855700 1 --2- 172.21.15.2:0/477080804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e700c88d0 0x7f4e7006b390 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4e7006d300 tx=0x7f4e68011ed0 comp rx=0 tx=0).stop 2023-08-25T03:34:17.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.180+0000 7f4e74855700 1 -- 172.21.15.2:0/477080804 shutdown_connections 2023-08-25T03:34:17.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.180+0000 7f4e74855700 1 --2- 172.21.15.2:0/477080804 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f4e5006a0e0 0x7f4e5006c5a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:17.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.180+0000 7f4e74855700 1 --2- 172.21.15.2:0/477080804 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4e700c88d0 0x7f4e7006b390 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:17.185 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.180+0000 7f4e74855700 1 --2- 172.21.15.2:0/477080804 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4e58006190 0x7f4e7015c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:17.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.180+0000 7f4e74855700 1 --2- 172.21.15.2:0/477080804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e700c9880 0x7f4e70153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:17.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.180+0000 7f4e74855700 1 -- 172.21.15.2:0/477080804 >> 172.21.15.2:0/477080804 conn(0x7f4e700cfea0 msgr2=0x7f4e70157b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:17.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.180+0000 7f4e74855700 1 -- 172.21.15.2:0/477080804 shutdown_connections 2023-08-25T03:34:17.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.180+0000 7f4e74855700 1 -- 172.21.15.2:0/477080804 wait complete. 2023-08-25T03:34:17.186 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-08-25T03:34:17.193 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-08-25T03:34:15.506497+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219352,"kb_used_data":3176,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754755368,"statfs":{"total":773094113280,"available":772869496832,"internally_reserved":0,"allocated":3252224,"data_stored":1930756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.897410"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611410+0000","last_change":"2023-08-25T03:34:03.628457+0000","last_active":"2023-08-25T03:34:04.611410+0000","last_peered":"2023-08-25T03:34:04.611410+0000","last_clean":"2023-08-25T03:34:04.611410+0000","last_became_active":"2023-08-25T03:34:03.627977+0000","last_became_peered":"2023-08-25T03:34:03.627977+0000","last_unstale":"2023-08-25T03:34:04.611410+0000","last_undegraded":"2023-08-25T03:34:04.611410+0000","last_fullsized":"2023-08-25T03:34:04.611410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:01:15.550047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611582+0000","last_change":"2023-08-25T03:34:03.616304+0000","last_active":"2023-08-25T03:34:04.611582+0000","last_peered":"2023-08-25T03:34:04.611582+0000","last_clean":"2023-08-25T03:34:04.611582+0000","last_became_active":"2023-08-25T03:34:03.615917+0000","last_became_peered":"2023-08-25T03:34:03.615917+0000","last_unstale":"2023-08-25T03:34:04.611582+0000","last_undegraded":"2023-08-25T03:34:04.611582+0000","last_fullsized":"2023-08-25T03:34:04.611582+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:08.250998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348968+0000","last_change":"2023-08-25T03:34:03.622799+0000","last_active":"2023-08-25T03:34:05.348968+0000","last_peered":"2023-08-25T03:34:05.348968+0000","last_clean":"2023-08-25T03:34:05.348968+0000","last_became_active":"2023-08-25T03:34:03.622602+0000","last_became_peered":"2023-08-25T03:34:03.622602+0000","last_unstale":"2023-08-25T03:34:05.348968+0000","last_undegraded":"2023-08-25T03:34:05.348968+0000","last_fullsized":"2023-08-25T03:34:05.348968+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:45:56.260357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.613438+0000","last_change":"2023-08-25T03:34:03.615254+0000","last_active":"2023-08-25T03:34:04.613438+0000","last_peered":"2023-08-25T03:34:04.613438+0000","last_clean":"2023-08-25T03:34:04.613438+0000","last_became_active":"2023-08-25T03:34:03.614917+0000","last_became_peered":"2023-08-25T03:34:03.614917+0000","last_unstale":"2023-08-25T03:34:04.613438+0000","last_undegraded":"2023-08-25T03:34:04.613438+0000","last_fullsized":"2023-08-25T03:34:04.613438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:43:29.368622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169198+0000","last_change":"2023-08-25T03:34:03.626638+0000","last_active":"2023-08-25T03:34:05.169198+0000","last_peered":"2023-08-25T03:34:05.169198+0000","last_clean":"2023-08-25T03:34:05.169198+0000","last_became_active":"2023-08-25T03:34:03.626317+0000","last_became_peered":"2023-08-25T03:34:03.626317+0000","last_unstale":"2023-08-25T03:34:05.169198+0000","last_undegraded":"2023-08-25T03:34:05.169198+0000","last_fullsized":"2023-08-25T03:34:05.169198+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:10:04.553228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348619+0000","last_change":"2023-08-25T03:34:03.619323+0000","last_active":"2023-08-25T03:34:05.348619+0000","last_peered":"2023-08-25T03:34:05.348619+0000","last_clean":"2023-08-25T03:34:05.348619+0000","last_became_active":"2023-08-25T03:34:03.618985+0000","last_became_peered":"2023-08-25T03:34:03.618985+0000","last_unstale":"2023-08-25T03:34:05.348619+0000","last_undegraded":"2023-08-25T03:34:05.348619+0000","last_fullsized":"2023-08-25T03:34:05.348619+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:02:02.415298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.263861+0000","last_change":"2023-08-25T03:34:03.617493+0000","last_active":"2023-08-25T03:34:05.263861+0000","last_peered":"2023-08-25T03:34:05.263861+0000","last_clean":"2023-08-25T03:34:05.263861+0000","last_became_active":"2023-08-25T03:34:03.617167+0000","last_became_peered":"2023-08-25T03:34:03.617167+0000","last_unstale":"2023-08-25T03:34:05.263861+0000","last_undegraded":"2023-08-25T03:34:05.263861+0000","last_fullsized":"2023-08-25T03:34:05.263861+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:57:59.924993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611074+0000","last_change":"2023-08-25T03:34:03.625710+0000","last_active":"2023-08-25T03:34:04.611074+0000","last_peered":"2023-08-25T03:34:04.611074+0000","last_clean":"2023-08-25T03:34:04.611074+0000","last_became_active":"2023-08-25T03:34:03.625404+0000","last_became_peered":"2023-08-25T03:34:03.625404+0000","last_unstale":"2023-08-25T03:34:04.611074+0000","last_undegraded":"2023-08-25T03:34:04.611074+0000","last_fullsized":"2023-08-25T03:34:04.611074+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:02:02.415298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169221+0000","last_change":"2023-08-25T03:34:03.626723+0000","last_active":"2023-08-25T03:34:05.169221+0000","last_peered":"2023-08-25T03:34:05.169221+0000","last_clean":"2023-08-25T03:34:05.169221+0000","last_became_active":"2023-08-25T03:34:03.626419+0000","last_became_peered":"2023-08-25T03:34:03.626419+0000","last_unstale":"2023-08-25T03:34:05.169221+0000","last_undegraded":"2023-08-25T03:34:05.169221+0000","last_fullsized":"2023-08-25T03:34:05.169221+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:09.006588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611787+0000","last_change":"2023-08-25T03:34:03.618247+0000","last_active":"2023-08-25T03:34:04.611787+0000","last_peered":"2023-08-25T03:34:04.611787+0000","last_clean":"2023-08-25T03:34:04.611787+0000","last_became_active":"2023-08-25T03:34:03.617830+0000","last_became_peered":"2023-08-25T03:34:03.617830+0000","last_unstale":"2023-08-25T03:34:04.611787+0000","last_undegraded":"2023-08-25T03:34:04.611787+0000","last_fullsized":"2023-08-25T03:34:04.611787+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:01:41.454184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349005+0000","last_change":"2023-08-25T03:34:03.622380+0000","last_active":"2023-08-25T03:34:05.349005+0000","last_peered":"2023-08-25T03:34:05.349005+0000","last_clean":"2023-08-25T03:34:05.349005+0000","last_became_active":"2023-08-25T03:34:03.619691+0000","last_became_peered":"2023-08-25T03:34:03.619691+0000","last_unstale":"2023-08-25T03:34:05.349005+0000","last_undegraded":"2023-08-25T03:34:05.349005+0000","last_fullsized":"2023-08-25T03:34:05.349005+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:08.250998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611500+0000","last_change":"2023-08-25T03:34:03.624204+0000","last_active":"2023-08-25T03:34:04.611500+0000","last_peered":"2023-08-25T03:34:04.611500+0000","last_clean":"2023-08-25T03:34:04.611500+0000","last_became_active":"2023-08-25T03:34:03.623945+0000","last_became_peered":"2023-08-25T03:34:03.623945+0000","last_unstale":"2023-08-25T03:34:04.611500+0000","last_undegraded":"2023-08-25T03:34:04.611500+0000","last_fullsized":"2023-08-25T03:34:04.611500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:25:43.507515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611336+0000","last_change":"2023-08-25T03:34:03.622941+0000","last_active":"2023-08-25T03:34:04.611336+0000","last_peered":"2023-08-25T03:34:04.611336+0000","last_clean":"2023-08-25T03:34:04.611336+0000","last_became_active":"2023-08-25T03:34:03.622627+0000","last_became_peered":"2023-08-25T03:34:03.622627+0000","last_unstale":"2023-08-25T03:34:04.611336+0000","last_undegraded":"2023-08-25T03:34:04.611336+0000","last_fullsized":"2023-08-25T03:34:04.611336+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:23:49.310992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611493+0000","last_change":"2023-08-25T03:34:03.616557+0000","last_active":"2023-08-25T03:34:04.611493+0000","last_peered":"2023-08-25T03:34:04.611493+0000","last_clean":"2023-08-25T03:34:04.611493+0000","last_became_active":"2023-08-25T03:34:03.615850+0000","last_became_peered":"2023-08-25T03:34:03.615850+0000","last_unstale":"2023-08-25T03:34:04.611493+0000","last_undegraded":"2023-08-25T03:34:04.611493+0000","last_fullsized":"2023-08-25T03:34:04.611493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:10:04.553228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264238+0000","last_change":"2023-08-25T03:34:03.617319+0000","last_active":"2023-08-25T03:34:05.264238+0000","last_peered":"2023-08-25T03:34:05.264238+0000","last_clean":"2023-08-25T03:34:05.264238+0000","last_became_active":"2023-08-25T03:34:03.617012+0000","last_became_peered":"2023-08-25T03:34:03.617012+0000","last_unstale":"2023-08-25T03:34:05.264238+0000","last_undegraded":"2023-08-25T03:34:05.264238+0000","last_fullsized":"2023-08-25T03:34:05.264238+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:00:42.442989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348569+0000","last_change":"2023-08-25T03:34:03.615370+0000","last_active":"2023-08-25T03:34:05.348569+0000","last_peered":"2023-08-25T03:34:05.348569+0000","last_clean":"2023-08-25T03:34:05.348569+0000","last_became_active":"2023-08-25T03:34:03.615076+0000","last_became_peered":"2023-08-25T03:34:03.615076+0000","last_unstale":"2023-08-25T03:34:05.348569+0000","last_undegraded":"2023-08-25T03:34:05.348569+0000","last_fullsized":"2023-08-25T03:34:05.348569+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:49:11.917056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169218+0000","last_change":"2023-08-25T03:34:03.627117+0000","last_active":"2023-08-25T03:34:05.169218+0000","last_peered":"2023-08-25T03:34:05.169218+0000","last_clean":"2023-08-25T03:34:05.169218+0000","last_became_active":"2023-08-25T03:34:03.626473+0000","last_became_peered":"2023-08-25T03:34:03.626473+0000","last_unstale":"2023-08-25T03:34:05.169218+0000","last_undegraded":"2023-08-25T03:34:05.169218+0000","last_fullsized":"2023-08-25T03:34:05.169218+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:52:26.311165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349494+0000","last_change":"2023-08-25T03:34:03.618966+0000","last_active":"2023-08-25T03:34:05.349494+0000","last_peered":"2023-08-25T03:34:05.349494+0000","last_clean":"2023-08-25T03:34:05.349494+0000","last_became_active":"2023-08-25T03:34:03.618636+0000","last_became_peered":"2023-08-25T03:34:03.618636+0000","last_unstale":"2023-08-25T03:34:05.349494+0000","last_undegraded":"2023-08-25T03:34:05.349494+0000","last_fullsized":"2023-08-25T03:34:05.349494+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:33:29.470663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611519+0000","last_change":"2023-08-25T03:34:03.605314+0000","last_active":"2023-08-25T03:34:04.611519+0000","last_peered":"2023-08-25T03:34:04.611519+0000","last_clean":"2023-08-25T03:34:04.611519+0000","last_became_active":"2023-08-25T03:34:03.605008+0000","last_became_peered":"2023-08-25T03:34:03.605008+0000","last_unstale":"2023-08-25T03:34:04.611519+0000","last_undegraded":"2023-08-25T03:34:04.611519+0000","last_fullsized":"2023-08-25T03:34:04.611519+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:47:39.963816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349001+0000","last_change":"2023-08-25T03:34:03.621319+0000","last_active":"2023-08-25T03:34:05.349001+0000","last_peered":"2023-08-25T03:34:05.349001+0000","last_clean":"2023-08-25T03:34:05.349001+0000","last_became_active":"2023-08-25T03:34:03.619115+0000","last_became_peered":"2023-08-25T03:34:03.619115+0000","last_unstale":"2023-08-25T03:34:05.349001+0000","last_undegraded":"2023-08-25T03:34:05.349001+0000","last_fullsized":"2023-08-25T03:34:05.349001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:23:05.976771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264207+0000","last_change":"2023-08-25T03:34:03.622785+0000","last_active":"2023-08-25T03:34:05.264207+0000","last_peered":"2023-08-25T03:34:05.264207+0000","last_clean":"2023-08-25T03:34:05.264207+0000","last_became_active":"2023-08-25T03:34:03.622469+0000","last_became_peered":"2023-08-25T03:34:03.622469+0000","last_unstale":"2023-08-25T03:34:05.264207+0000","last_undegraded":"2023-08-25T03:34:05.264207+0000","last_fullsized":"2023-08-25T03:34:05.264207+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:35:42.993608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.168950+0000","last_change":"2023-08-25T03:34:03.620545+0000","last_active":"2023-08-25T03:34:05.168950+0000","last_peered":"2023-08-25T03:34:05.168950+0000","last_clean":"2023-08-25T03:34:05.168950+0000","last_became_active":"2023-08-25T03:34:03.619698+0000","last_became_peered":"2023-08-25T03:34:03.619698+0000","last_unstale":"2023-08-25T03:34:05.168950+0000","last_undegraded":"2023-08-25T03:34:05.168950+0000","last_fullsized":"2023-08-25T03:34:05.168950+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:41:33.001585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349094+0000","last_change":"2023-08-25T03:34:03.631189+0000","last_active":"2023-08-25T03:34:05.349094+0000","last_peered":"2023-08-25T03:34:05.349094+0000","last_clean":"2023-08-25T03:34:05.349094+0000","last_became_active":"2023-08-25T03:34:03.630807+0000","last_became_peered":"2023-08-25T03:34:03.630807+0000","last_unstale":"2023-08-25T03:34:05.349094+0000","last_undegraded":"2023-08-25T03:34:05.349094+0000","last_fullsized":"2023-08-25T03:34:05.349094+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:41:33.001585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.263844+0000","last_change":"2023-08-25T03:34:03.623150+0000","last_active":"2023-08-25T03:34:05.263844+0000","last_peered":"2023-08-25T03:34:05.263844+0000","last_clean":"2023-08-25T03:34:05.263844+0000","last_became_active":"2023-08-25T03:34:03.622796+0000","last_became_peered":"2023-08-25T03:34:03.622796+0000","last_unstale":"2023-08-25T03:34:05.263844+0000","last_undegraded":"2023-08-25T03:34:05.263844+0000","last_fullsized":"2023-08-25T03:34:05.263844+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:01:41.454184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349917+0000","last_change":"2023-08-25T03:34:03.620031+0000","last_active":"2023-08-25T03:34:05.349917+0000","last_peered":"2023-08-25T03:34:05.349917+0000","last_clean":"2023-08-25T03:34:05.349917+0000","last_became_active":"2023-08-25T03:34:03.619738+0000","last_became_peered":"2023-08-25T03:34:03.619738+0000","last_unstale":"2023-08-25T03:34:05.349917+0000","last_undegraded":"2023-08-25T03:34:05.349917+0000","last_fullsized":"2023-08-25T03:34:05.349917+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:08.250998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349705+0000","last_change":"2023-08-25T03:34:03.632252+0000","last_active":"2023-08-25T03:34:05.349705+0000","last_peered":"2023-08-25T03:34:05.349705+0000","last_clean":"2023-08-25T03:34:05.349705+0000","last_became_active":"2023-08-25T03:34:03.631975+0000","last_became_peered":"2023-08-25T03:34:03.631975+0000","last_unstale":"2023-08-25T03:34:05.349705+0000","last_undegraded":"2023-08-25T03:34:05.349705+0000","last_fullsized":"2023-08-25T03:34:05.349705+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:22:25.881167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611728+0000","last_change":"2023-08-25T03:34:03.613385+0000","last_active":"2023-08-25T03:34:04.611728+0000","last_peered":"2023-08-25T03:34:04.611728+0000","last_clean":"2023-08-25T03:34:04.611728+0000","last_became_active":"2023-08-25T03:34:03.613110+0000","last_became_peered":"2023-08-25T03:34:03.613110+0000","last_unstale":"2023-08-25T03:34:04.611728+0000","last_undegraded":"2023-08-25T03:34:04.611728+0000","last_fullsized":"2023-08-25T03:34:04.611728+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:03:06.448800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348876+0000","last_change":"2023-08-25T03:34:02.608606+0000","last_active":"2023-08-25T03:34:05.348876+0000","last_peered":"2023-08-25T03:34:05.348876+0000","last_clean":"2023-08-25T03:34:05.348876+0000","last_became_active":"2023-08-25T03:34:02.608298+0000","last_became_peered":"2023-08-25T03:34:02.608298+0000","last_unstale":"2023-08-25T03:34:05.348876+0000","last_undegraded":"2023-08-25T03:34:05.348876+0000","last_fullsized":"2023-08-25T03:34:05.348876+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:52:25.303751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.350009+0000","last_change":"2023-08-25T03:34:03.619019+0000","last_active":"2023-08-25T03:34:05.350009+0000","last_peered":"2023-08-25T03:34:05.350009+0000","last_clean":"2023-08-25T03:34:05.350009+0000","last_became_active":"2023-08-25T03:34:03.618719+0000","last_became_peered":"2023-08-25T03:34:03.618719+0000","last_unstale":"2023-08-25T03:34:05.350009+0000","last_undegraded":"2023-08-25T03:34:05.350009+0000","last_fullsized":"2023-08-25T03:34:05.350009+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:23:05.976771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349898+0000","last_change":"2023-08-25T03:34:02.599366+0000","last_active":"2023-08-25T03:34:05.349898+0000","last_peered":"2023-08-25T03:34:05.349898+0000","last_clean":"2023-08-25T03:34:05.349898+0000","last_became_active":"2023-08-25T03:34:02.599105+0000","last_became_peered":"2023-08-25T03:34:02.599105+0000","last_unstale":"2023-08-25T03:34:05.349898+0000","last_undegraded":"2023-08-25T03:34:05.349898+0000","last_fullsized":"2023-08-25T03:34:05.349898+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:03:05.441386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169298+0000","last_change":"2023-08-25T03:34:03.615711+0000","last_active":"2023-08-25T03:34:05.169298+0000","last_peered":"2023-08-25T03:34:05.169298+0000","last_clean":"2023-08-25T03:34:05.169298+0000","last_became_active":"2023-08-25T03:34:03.615315+0000","last_became_peered":"2023-08-25T03:34:03.615315+0000","last_unstale":"2023-08-25T03:34:05.169298+0000","last_undegraded":"2023-08-25T03:34:05.169298+0000","last_fullsized":"2023-08-25T03:34:05.169298+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:03:06.448800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349079+0000","last_change":"2023-08-25T03:34:02.608397+0000","last_active":"2023-08-25T03:34:05.349079+0000","last_peered":"2023-08-25T03:34:05.349079+0000","last_clean":"2023-08-25T03:34:05.349079+0000","last_became_active":"2023-08-25T03:34:02.608067+0000","last_became_peered":"2023-08-25T03:34:02.608067+0000","last_unstale":"2023-08-25T03:34:05.349079+0000","last_undegraded":"2023-08-25T03:34:05.349079+0000","last_fullsized":"2023-08-25T03:34:05.349079+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:07.243584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612775+0000","last_change":"2023-08-25T03:34:03.613614+0000","last_active":"2023-08-25T03:34:04.612775+0000","last_peered":"2023-08-25T03:34:04.612775+0000","last_clean":"2023-08-25T03:34:04.612775+0000","last_became_active":"2023-08-25T03:34:03.613291+0000","last_became_peered":"2023-08-25T03:34:03.613291+0000","last_unstale":"2023-08-25T03:34:04.612775+0000","last_undegraded":"2023-08-25T03:34:04.612775+0000","last_fullsized":"2023-08-25T03:34:04.612775+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:38:56.062178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169421+0000","last_change":"2023-08-25T03:34:02.603475+0000","last_active":"2023-08-25T03:34:05.169421+0000","last_peered":"2023-08-25T03:34:05.169421+0000","last_clean":"2023-08-25T03:34:05.169421+0000","last_became_active":"2023-08-25T03:34:02.603129+0000","last_became_peered":"2023-08-25T03:34:02.603129+0000","last_unstale":"2023-08-25T03:34:05.169421+0000","last_undegraded":"2023-08-25T03:34:05.169421+0000","last_fullsized":"2023-08-25T03:34:05.169421+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:27:14.892441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349004+0000","last_change":"2023-08-25T03:34:03.615004+0000","last_active":"2023-08-25T03:34:05.349004+0000","last_peered":"2023-08-25T03:34:05.349004+0000","last_clean":"2023-08-25T03:34:05.349004+0000","last_became_active":"2023-08-25T03:34:03.614699+0000","last_became_peered":"2023-08-25T03:34:03.614699+0000","last_unstale":"2023-08-25T03:34:05.349004+0000","last_undegraded":"2023-08-25T03:34:05.349004+0000","last_fullsized":"2023-08-25T03:34:05.349004+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:02:02.415298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611685+0000","last_change":"2023-08-25T03:34:02.597247+0000","last_active":"2023-08-25T03:34:04.611685+0000","last_peered":"2023-08-25T03:34:04.611685+0000","last_clean":"2023-08-25T03:34:04.611685+0000","last_became_active":"2023-08-25T03:34:02.596968+0000","last_became_peered":"2023-08-25T03:34:02.596968+0000","last_unstale":"2023-08-25T03:34:04.611685+0000","last_undegraded":"2023-08-25T03:34:04.611685+0000","last_fullsized":"2023-08-25T03:34:04.611685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:23:04.969357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349570+0000","last_change":"2023-08-25T03:34:03.623532+0000","last_active":"2023-08-25T03:34:05.349570+0000","last_peered":"2023-08-25T03:34:05.349570+0000","last_clean":"2023-08-25T03:34:05.349570+0000","last_became_active":"2023-08-25T03:34:03.622546+0000","last_became_peered":"2023-08-25T03:34:03.622546+0000","last_unstale":"2023-08-25T03:34:05.349570+0000","last_undegraded":"2023-08-25T03:34:05.349570+0000","last_fullsized":"2023-08-25T03:34:05.349570+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:22:25.881167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264179+0000","last_change":"2023-08-25T03:34:02.603880+0000","last_active":"2023-08-25T03:34:05.264179+0000","last_peered":"2023-08-25T03:34:05.264179+0000","last_clean":"2023-08-25T03:34:05.264179+0000","last_became_active":"2023-08-25T03:34:02.603502+0000","last_became_peered":"2023-08-25T03:34:02.603502+0000","last_unstale":"2023-08-25T03:34:05.264179+0000","last_undegraded":"2023-08-25T03:34:05.264179+0000","last_fullsized":"2023-08-25T03:34:05.264179+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:39:33.908387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.350317+0000","last_change":"2023-08-25T03:34:03.614556+0000","last_active":"2023-08-25T03:34:05.350317+0000","last_peered":"2023-08-25T03:34:05.350317+0000","last_clean":"2023-08-25T03:34:05.350317+0000","last_became_active":"2023-08-25T03:34:03.614260+0000","last_became_peered":"2023-08-25T03:34:03.614260+0000","last_unstale":"2023-08-25T03:34:05.350317+0000","last_undegraded":"2023-08-25T03:34:05.350317+0000","last_fullsized":"2023-08-25T03:34:05.350317+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T03:41:42.153100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'6","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349211+0000","last_change":"2023-08-25T03:34:02.606875+0000","last_active":"2023-08-25T03:34:05.349211+0000","last_peered":"2023-08-25T03:34:05.349211+0000","last_clean":"2023-08-25T03:34:05.349211+0000","last_became_active":"2023-08-25T03:34:02.606567+0000","last_became_peered":"2023-08-25T03:34:02.606567+0000","last_unstale":"2023-08-25T03:34:05.349211+0000","last_undegraded":"2023-08-25T03:34:05.349211+0000","last_fullsized":"2023-08-25T03:34:05.349211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:27:14.892441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348768+0000","last_change":"2023-08-25T03:34:03.619847+0000","last_active":"2023-08-25T03:34:05.348768+0000","last_peered":"2023-08-25T03:34:05.348768+0000","last_clean":"2023-08-25T03:34:05.348768+0000","last_became_active":"2023-08-25T03:34:03.619046+0000","last_became_peered":"2023-08-25T03:34:03.619046+0000","last_unstale":"2023-08-25T03:34:05.348768+0000","last_undegraded":"2023-08-25T03:34:05.348768+0000","last_fullsized":"2023-08-25T03:34:05.348768+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:22:25.881167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264047+0000","last_change":"2023-08-25T03:34:02.604424+0000","last_active":"2023-08-25T03:34:05.264047+0000","last_peered":"2023-08-25T03:34:05.264047+0000","last_clean":"2023-08-25T03:34:05.264047+0000","last_became_active":"2023-08-25T03:34:02.604175+0000","last_became_peered":"2023-08-25T03:34:02.604175+0000","last_unstale":"2023-08-25T03:34:05.264047+0000","last_undegraded":"2023-08-25T03:34:05.264047+0000","last_fullsized":"2023-08-25T03:34:05.264047+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:45:55.252943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349790+0000","last_change":"2023-08-25T03:34:03.622054+0000","last_active":"2023-08-25T03:34:05.349790+0000","last_peered":"2023-08-25T03:34:05.349790+0000","last_clean":"2023-08-25T03:34:05.349790+0000","last_became_active":"2023-08-25T03:34:03.619122+0000","last_became_peered":"2023-08-25T03:34:03.619122+0000","last_unstale":"2023-08-25T03:34:05.349790+0000","last_undegraded":"2023-08-25T03:34:05.349790+0000","last_fullsized":"2023-08-25T03:34:05.349790+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:09.006588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.168841+0000","last_change":"2023-08-25T03:34:02.606889+0000","last_active":"2023-08-25T03:34:05.168841+0000","last_peered":"2023-08-25T03:34:05.168841+0000","last_clean":"2023-08-25T03:34:05.168841+0000","last_became_active":"2023-08-25T03:34:02.606424+0000","last_became_peered":"2023-08-25T03:34:02.606424+0000","last_unstale":"2023-08-25T03:34:05.168841+0000","last_undegraded":"2023-08-25T03:34:05.168841+0000","last_fullsized":"2023-08-25T03:34:05.168841+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:02:01.407883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611980+0000","last_change":"2023-08-25T03:34:03.613888+0000","last_active":"2023-08-25T03:34:04.611980+0000","last_peered":"2023-08-25T03:34:04.611980+0000","last_clean":"2023-08-25T03:34:04.611980+0000","last_became_active":"2023-08-25T03:34:03.612829+0000","last_became_peered":"2023-08-25T03:34:03.612829+0000","last_unstale":"2023-08-25T03:34:04.611980+0000","last_undegraded":"2023-08-25T03:34:04.611980+0000","last_fullsized":"2023-08-25T03:34:04.611980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:27:15.899855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.263949+0000","last_change":"2023-08-25T03:34:02.606279+0000","last_active":"2023-08-25T03:34:05.263949+0000","last_peered":"2023-08-25T03:34:05.263949+0000","last_clean":"2023-08-25T03:34:05.263949+0000","last_became_active":"2023-08-25T03:34:02.606005+0000","last_became_peered":"2023-08-25T03:34:02.606005+0000","last_unstale":"2023-08-25T03:34:05.263949+0000","last_undegraded":"2023-08-25T03:34:05.263949+0000","last_fullsized":"2023-08-25T03:34:05.263949+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T03:41:41.145686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169244+0000","last_change":"2023-08-25T03:34:03.627203+0000","last_active":"2023-08-25T03:34:05.169244+0000","last_peered":"2023-08-25T03:34:05.169244+0000","last_clean":"2023-08-25T03:34:05.169244+0000","last_became_active":"2023-08-25T03:34:03.626901+0000","last_became_peered":"2023-08-25T03:34:03.626901+0000","last_unstale":"2023-08-25T03:34:05.169244+0000","last_undegraded":"2023-08-25T03:34:05.169244+0000","last_fullsized":"2023-08-25T03:34:05.169244+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:23:05.976771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612301+0000","last_change":"2023-08-25T03:34:02.608430+0000","last_active":"2023-08-25T03:34:04.612301+0000","last_peered":"2023-08-25T03:34:04.612301+0000","last_clean":"2023-08-25T03:34:04.612301+0000","last_became_active":"2023-08-25T03:34:02.608086+0000","last_became_peered":"2023-08-25T03:34:02.608086+0000","last_unstale":"2023-08-25T03:34:04.612301+0000","last_undegraded":"2023-08-25T03:34:04.612301+0000","last_fullsized":"2023-08-25T03:34:04.612301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:23:48.303578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.350395+0000","last_change":"2023-08-25T03:34:03.624700+0000","last_active":"2023-08-25T03:34:05.350395+0000","last_peered":"2023-08-25T03:34:05.350395+0000","last_clean":"2023-08-25T03:34:05.350395+0000","last_became_active":"2023-08-25T03:34:03.624180+0000","last_became_peered":"2023-08-25T03:34:03.624180+0000","last_unstale":"2023-08-25T03:34:05.350395+0000","last_undegraded":"2023-08-25T03:34:05.350395+0000","last_fullsized":"2023-08-25T03:34:05.350395+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:27:15.899855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264384+0000","last_change":"2023-08-25T03:34:02.604587+0000","last_active":"2023-08-25T03:34:05.264384+0000","last_peered":"2023-08-25T03:34:05.264384+0000","last_clean":"2023-08-25T03:34:05.264384+0000","last_became_active":"2023-08-25T03:34:02.604290+0000","last_became_peered":"2023-08-25T03:34:02.604290+0000","last_unstale":"2023-08-25T03:34:05.264384+0000","last_undegraded":"2023-08-25T03:34:05.264384+0000","last_fullsized":"2023-08-25T03:34:05.264384+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:07:41.306323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611079+0000","last_change":"2023-08-25T03:34:03.613959+0000","last_active":"2023-08-25T03:34:04.611079+0000","last_peered":"2023-08-25T03:34:04.611079+0000","last_clean":"2023-08-25T03:34:04.611079+0000","last_became_active":"2023-08-25T03:34:03.613667+0000","last_became_peered":"2023-08-25T03:34:03.613667+0000","last_unstale":"2023-08-25T03:34:04.611079+0000","last_undegraded":"2023-08-25T03:34:04.611079+0000","last_fullsized":"2023-08-25T03:34:04.611079+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:41:33.001585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264371+0000","last_change":"2023-08-25T03:34:02.604585+0000","last_active":"2023-08-25T03:34:05.264371+0000","last_peered":"2023-08-25T03:34:05.264371+0000","last_clean":"2023-08-25T03:34:05.264371+0000","last_became_active":"2023-08-25T03:34:02.604255+0000","last_became_peered":"2023-08-25T03:34:02.604255+0000","last_unstale":"2023-08-25T03:34:05.264371+0000","last_undegraded":"2023-08-25T03:34:05.264371+0000","last_fullsized":"2023-08-25T03:34:05.264371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:31:14.838181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348736+0000","last_change":"2023-08-25T03:34:03.624731+0000","last_active":"2023-08-25T03:34:05.348736+0000","last_peered":"2023-08-25T03:34:05.348736+0000","last_clean":"2023-08-25T03:34:05.348736+0000","last_became_active":"2023-08-25T03:34:03.619454+0000","last_became_peered":"2023-08-25T03:34:03.619454+0000","last_unstale":"2023-08-25T03:34:05.348736+0000","last_undegraded":"2023-08-25T03:34:05.348736+0000","last_fullsized":"2023-08-25T03:34:05.348736+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:05:25.658159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264522+0000","last_change":"2023-08-25T03:34:02.605137+0000","last_active":"2023-08-25T03:34:05.264522+0000","last_peered":"2023-08-25T03:34:05.264522+0000","last_clean":"2023-08-25T03:34:05.264522+0000","last_became_active":"2023-08-25T03:34:02.604846+0000","last_became_peered":"2023-08-25T03:34:02.604846+0000","last_unstale":"2023-08-25T03:34:05.264522+0000","last_undegraded":"2023-08-25T03:34:05.264522+0000","last_fullsized":"2023-08-25T03:34:05.264522+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:44:39.385392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612781+0000","last_change":"2023-08-25T03:34:03.614885+0000","last_active":"2023-08-25T03:34:04.612781+0000","last_peered":"2023-08-25T03:34:04.612781+0000","last_clean":"2023-08-25T03:34:04.612781+0000","last_became_active":"2023-08-25T03:34:03.614420+0000","last_became_peered":"2023-08-25T03:34:03.614420+0000","last_unstale":"2023-08-25T03:34:04.612781+0000","last_undegraded":"2023-08-25T03:34:04.612781+0000","last_fullsized":"2023-08-25T03:34:04.612781+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:47:27.124952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611601+0000","last_change":"2023-08-25T03:34:02.598840+0000","last_active":"2023-08-25T03:34:04.611601+0000","last_peered":"2023-08-25T03:34:04.611601+0000","last_clean":"2023-08-25T03:34:04.611601+0000","last_became_active":"2023-08-25T03:34:02.598544+0000","last_became_peered":"2023-08-25T03:34:02.598544+0000","last_unstale":"2023-08-25T03:34:04.611601+0000","last_undegraded":"2023-08-25T03:34:04.611601+0000","last_fullsized":"2023-08-25T03:34:04.611601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:52:25.303751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.168983+0000","last_change":"2023-08-25T03:34:03.616738+0000","last_active":"2023-08-25T03:34:05.168983+0000","last_peered":"2023-08-25T03:34:05.168983+0000","last_clean":"2023-08-25T03:34:05.168983+0000","last_became_active":"2023-08-25T03:34:03.616450+0000","last_became_peered":"2023-08-25T03:34:03.616450+0000","last_unstale":"2023-08-25T03:34:05.168983+0000","last_undegraded":"2023-08-25T03:34:05.168983+0000","last_fullsized":"2023-08-25T03:34:05.168983+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:05:25.658159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349607+0000","last_change":"2023-08-25T03:34:02.605001+0000","last_active":"2023-08-25T03:34:05.349607+0000","last_peered":"2023-08-25T03:34:05.349607+0000","last_clean":"2023-08-25T03:34:05.349607+0000","last_became_active":"2023-08-25T03:34:02.604686+0000","last_became_peered":"2023-08-25T03:34:02.604686+0000","last_unstale":"2023-08-25T03:34:05.349607+0000","last_undegraded":"2023-08-25T03:34:05.349607+0000","last_fullsized":"2023-08-25T03:34:05.349607+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:10:03.545814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349133+0000","last_change":"2023-08-25T03:34:03.626817+0000","last_active":"2023-08-25T03:34:05.349133+0000","last_peered":"2023-08-25T03:34:05.349133+0000","last_clean":"2023-08-25T03:34:05.349133+0000","last_became_active":"2023-08-25T03:34:03.626483+0000","last_became_peered":"2023-08-25T03:34:03.626483+0000","last_unstale":"2023-08-25T03:34:05.349133+0000","last_undegraded":"2023-08-25T03:34:05.349133+0000","last_fullsized":"2023-08-25T03:34:05.349133+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:01:41.454184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612643+0000","last_change":"2023-08-25T03:34:02.601925+0000","last_active":"2023-08-25T03:34:04.612643+0000","last_peered":"2023-08-25T03:34:04.612643+0000","last_clean":"2023-08-25T03:34:04.612643+0000","last_became_active":"2023-08-25T03:34:02.601552+0000","last_became_peered":"2023-08-25T03:34:02.601552+0000","last_unstale":"2023-08-25T03:34:04.612643+0000","last_undegraded":"2023-08-25T03:34:04.612643+0000","last_fullsized":"2023-08-25T03:34:04.612643+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:47:38.956402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349296+0000","last_change":"2023-08-25T03:34:03.627975+0000","last_active":"2023-08-25T03:34:05.349296+0000","last_peered":"2023-08-25T03:34:05.349296+0000","last_clean":"2023-08-25T03:34:05.349296+0000","last_became_active":"2023-08-25T03:34:03.627489+0000","last_became_peered":"2023-08-25T03:34:03.627489+0000","last_unstale":"2023-08-25T03:34:05.349296+0000","last_undegraded":"2023-08-25T03:34:05.349296+0000","last_fullsized":"2023-08-25T03:34:05.349296+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:57:59.924993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348882+0000","last_change":"2023-08-25T03:34:02.603347+0000","last_active":"2023-08-25T03:34:05.348882+0000","last_peered":"2023-08-25T03:34:05.348882+0000","last_clean":"2023-08-25T03:34:05.348882+0000","last_became_active":"2023-08-25T03:34:02.603010+0000","last_became_peered":"2023-08-25T03:34:02.603010+0000","last_unstale":"2023-08-25T03:34:05.348882+0000","last_undegraded":"2023-08-25T03:34:05.348882+0000","last_fullsized":"2023-08-25T03:34:05.348882+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:07.999174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349594+0000","last_change":"2023-08-25T03:34:03.616962+0000","last_active":"2023-08-25T03:34:05.349594+0000","last_peered":"2023-08-25T03:34:05.349594+0000","last_clean":"2023-08-25T03:34:05.349594+0000","last_became_active":"2023-08-25T03:34:03.616656+0000","last_became_peered":"2023-08-25T03:34:03.616656+0000","last_unstale":"2023-08-25T03:34:05.349594+0000","last_undegraded":"2023-08-25T03:34:05.349594+0000","last_fullsized":"2023-08-25T03:34:05.349594+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:39:34.915801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348904+0000","last_change":"2023-08-25T03:34:02.603820+0000","last_active":"2023-08-25T03:34:05.348904+0000","last_peered":"2023-08-25T03:34:05.348904+0000","last_clean":"2023-08-25T03:34:05.348904+0000","last_became_active":"2023-08-25T03:34:02.603493+0000","last_became_peered":"2023-08-25T03:34:02.603493+0000","last_unstale":"2023-08-25T03:34:05.348904+0000","last_undegraded":"2023-08-25T03:34:05.348904+0000","last_fullsized":"2023-08-25T03:34:05.348904+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:47:12.826390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.350183+0000","last_change":"2023-08-25T03:34:03.625076+0000","last_active":"2023-08-25T03:34:05.350183+0000","last_peered":"2023-08-25T03:34:05.350183+0000","last_clean":"2023-08-25T03:34:05.350183+0000","last_became_active":"2023-08-25T03:34:03.624812+0000","last_became_peered":"2023-08-25T03:34:03.624812+0000","last_unstale":"2023-08-25T03:34:05.350183+0000","last_undegraded":"2023-08-25T03:34:05.350183+0000","last_fullsized":"2023-08-25T03:34:05.350183+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:27:15.899855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349701+0000","last_change":"2023-08-25T03:33:57.570267+0000","last_active":"2023-08-25T03:34:05.349701+0000","last_peered":"2023-08-25T03:34:05.349701+0000","last_clean":"2023-08-25T03:34:05.349701+0000","last_became_active":"2023-08-25T03:33:57.569882+0000","last_became_peered":"2023-08-25T03:33:57.569882+0000","last_unstale":"2023-08-25T03:34:05.349701+0000","last_undegraded":"2023-08-25T03:34:05.349701+0000","last_fullsized":"2023-08-25T03:34:05.349701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:09:58.529844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612129+0000","last_change":"2023-08-25T03:34:02.600129+0000","last_active":"2023-08-25T03:34:04.612129+0000","last_peered":"2023-08-25T03:34:04.612129+0000","last_clean":"2023-08-25T03:34:04.612129+0000","last_became_active":"2023-08-25T03:34:02.599762+0000","last_became_peered":"2023-08-25T03:34:02.599762+0000","last_unstale":"2023-08-25T03:34:04.612129+0000","last_undegraded":"2023-08-25T03:34:04.612129+0000","last_fullsized":"2023-08-25T03:34:04.612129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T03:41:41.145686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349944+0000","last_change":"2023-08-25T03:34:03.616199+0000","last_active":"2023-08-25T03:34:05.349944+0000","last_peered":"2023-08-25T03:34:05.349944+0000","last_clean":"2023-08-25T03:34:05.349944+0000","last_became_active":"2023-08-25T03:34:03.613500+0000","last_became_peered":"2023-08-25T03:34:03.613500+0000","last_unstale":"2023-08-25T03:34:05.349944+0000","last_undegraded":"2023-08-25T03:34:05.349944+0000","last_fullsized":"2023-08-25T03:34:05.349944+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:23:05.976771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611287+0000","last_change":"2023-08-25T03:33:57.569595+0000","last_active":"2023-08-25T03:34:04.611287+0000","last_peered":"2023-08-25T03:34:04.611287+0000","last_clean":"2023-08-25T03:34:04.611287+0000","last_became_active":"2023-08-25T03:33:57.569307+0000","last_became_peered":"2023-08-25T03:33:57.569307+0000","last_unstale":"2023-08-25T03:34:04.611287+0000","last_undegraded":"2023-08-25T03:34:04.611287+0000","last_fullsized":"2023-08-25T03:34:04.611287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:05:19.634775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349072+0000","last_change":"2023-08-25T03:34:02.605404+0000","last_active":"2023-08-25T03:34:05.349072+0000","last_peered":"2023-08-25T03:34:05.349072+0000","last_clean":"2023-08-25T03:34:05.349072+0000","last_became_active":"2023-08-25T03:34:02.604806+0000","last_became_peered":"2023-08-25T03:34:02.604806+0000","last_unstale":"2023-08-25T03:34:05.349072+0000","last_undegraded":"2023-08-25T03:34:05.349072+0000","last_fullsized":"2023-08-25T03:34:05.349072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:03:05.441386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349865+0000","last_change":"2023-08-25T03:34:03.632131+0000","last_active":"2023-08-25T03:34:05.349865+0000","last_peered":"2023-08-25T03:34:05.349865+0000","last_clean":"2023-08-25T03:34:05.349865+0000","last_became_active":"2023-08-25T03:34:03.631825+0000","last_became_peered":"2023-08-25T03:34:03.631825+0000","last_unstale":"2023-08-25T03:34:05.349865+0000","last_undegraded":"2023-08-25T03:34:05.349865+0000","last_fullsized":"2023-08-25T03:34:05.349865+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:09.006588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611334+0000","last_change":"2023-08-25T03:33:57.886704+0000","last_active":"2023-08-25T03:34:04.611334+0000","last_peered":"2023-08-25T03:34:04.611334+0000","last_clean":"2023-08-25T03:34:04.611334+0000","last_became_active":"2023-08-25T03:33:57.886294+0000","last_became_peered":"2023-08-25T03:33:57.886294+0000","last_unstale":"2023-08-25T03:34:04.611334+0000","last_undegraded":"2023-08-25T03:34:04.611334+0000","last_fullsized":"2023-08-25T03:34:04.611334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:15:31.545146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349162+0000","last_change":"2023-08-25T03:34:02.589214+0000","last_active":"2023-08-25T03:34:05.349162+0000","last_peered":"2023-08-25T03:34:05.349162+0000","last_clean":"2023-08-25T03:34:05.349162+0000","last_became_active":"2023-08-25T03:34:02.588607+0000","last_became_peered":"2023-08-25T03:34:02.588607+0000","last_unstale":"2023-08-25T03:34:05.349162+0000","last_undegraded":"2023-08-25T03:34:05.349162+0000","last_fullsized":"2023-08-25T03:34:05.349162+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:02:01.407883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.350043+0000","last_change":"2023-08-25T03:34:03.613767+0000","last_active":"2023-08-25T03:34:05.350043+0000","last_peered":"2023-08-25T03:34:05.350043+0000","last_clean":"2023-08-25T03:34:05.350043+0000","last_became_active":"2023-08-25T03:34:03.613433+0000","last_became_peered":"2023-08-25T03:34:03.613433+0000","last_unstale":"2023-08-25T03:34:05.350043+0000","last_undegraded":"2023-08-25T03:34:05.350043+0000","last_fullsized":"2023-08-25T03:34:05.350043+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:01:41.454184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611389+0000","last_change":"2023-08-25T03:33:57.570657+0000","last_active":"2023-08-25T03:34:04.611389+0000","last_peered":"2023-08-25T03:34:04.611389+0000","last_clean":"2023-08-25T03:34:04.611389+0000","last_became_active":"2023-08-25T03:33:57.570425+0000","last_became_peered":"2023-08-25T03:33:57.570425+0000","last_unstale":"2023-08-25T03:34:04.611389+0000","last_undegraded":"2023-08-25T03:34:04.611389+0000","last_fullsized":"2023-08-25T03:34:04.611389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:22:19.857784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":175,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.263861+0000","last_change":"2023-08-25T03:33:51.259898+0000","last_active":"2023-08-25T03:34:05.263861+0000","last_peered":"2023-08-25T03:34:05.263861+0000","last_clean":"2023-08-25T03:34:05.263861+0000","last_became_active":"2023-08-25T03:33:51.258780+0000","last_became_peered":"2023-08-25T03:33:51.258780+0000","last_unstale":"2023-08-25T03:34:05.263861+0000","last_undegraded":"2023-08-25T03:34:05.263861+0000","last_fullsized":"2023-08-25T03:34:05.263861+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:49.666858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:49.666858+0000","last_clean_scrub_stamp":"2023-08-25T03:33:49.666858+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:27:02.988993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349202+0000","last_change":"2023-08-25T03:34:02.598738+0000","last_active":"2023-08-25T03:34:05.349202+0000","last_peered":"2023-08-25T03:34:05.349202+0000","last_clean":"2023-08-25T03:34:05.349202+0000","last_became_active":"2023-08-25T03:34:02.598219+0000","last_became_peered":"2023-08-25T03:34:02.598219+0000","last_unstale":"2023-08-25T03:34:05.349202+0000","last_undegraded":"2023-08-25T03:34:05.349202+0000","last_fullsized":"2023-08-25T03:34:05.349202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:05:24.650744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612566+0000","last_change":"2023-08-25T03:34:03.624616+0000","last_active":"2023-08-25T03:34:04.612566+0000","last_peered":"2023-08-25T03:34:04.612566+0000","last_clean":"2023-08-25T03:34:04.612566+0000","last_became_active":"2023-08-25T03:34:03.624330+0000","last_became_peered":"2023-08-25T03:34:03.624330+0000","last_unstale":"2023-08-25T03:34:04.612566+0000","last_undegraded":"2023-08-25T03:34:04.612566+0000","last_fullsized":"2023-08-25T03:34:04.612566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:25:43.507515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264117+0000","last_change":"2023-08-25T03:33:57.571821+0000","last_active":"2023-08-25T03:34:05.264117+0000","last_peered":"2023-08-25T03:34:05.264117+0000","last_clean":"2023-08-25T03:34:05.264117+0000","last_became_active":"2023-08-25T03:33:57.571528+0000","last_became_peered":"2023-08-25T03:33:57.571528+0000","last_unstale":"2023-08-25T03:34:05.264117+0000","last_undegraded":"2023-08-25T03:34:05.264117+0000","last_fullsized":"2023-08-25T03:34:05.264117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:33:23.447280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.168795+0000","last_change":"2023-08-25T03:34:02.602275+0000","last_active":"2023-08-25T03:34:05.168795+0000","last_peered":"2023-08-25T03:34:05.168795+0000","last_clean":"2023-08-25T03:34:05.168795+0000","last_became_active":"2023-08-25T03:34:02.601960+0000","last_became_peered":"2023-08-25T03:34:02.601960+0000","last_unstale":"2023-08-25T03:34:05.168795+0000","last_undegraded":"2023-08-25T03:34:05.168795+0000","last_fullsized":"2023-08-25T03:34:05.168795+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:47:12.826390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169447+0000","last_change":"2023-08-25T03:34:03.612794+0000","last_active":"2023-08-25T03:34:05.169447+0000","last_peered":"2023-08-25T03:34:05.169447+0000","last_clean":"2023-08-25T03:34:05.169447+0000","last_became_active":"2023-08-25T03:34:03.612474+0000","last_became_peered":"2023-08-25T03:34:03.612474+0000","last_unstale":"2023-08-25T03:34:05.169447+0000","last_undegraded":"2023-08-25T03:34:05.169447+0000","last_fullsized":"2023-08-25T03:34:05.169447+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T03:41:42.153100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264364+0000","last_change":"2023-08-25T03:33:57.571730+0000","last_active":"2023-08-25T03:34:05.264364+0000","last_peered":"2023-08-25T03:34:05.264364+0000","last_clean":"2023-08-25T03:34:05.264364+0000","last_became_active":"2023-08-25T03:33:57.571442+0000","last_became_peered":"2023-08-25T03:33:57.571442+0000","last_unstale":"2023-08-25T03:34:05.264364+0000","last_undegraded":"2023-08-25T03:34:05.264364+0000","last_fullsized":"2023-08-25T03:34:05.264364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T03:49:14.356320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349890+0000","last_change":"2023-08-25T03:34:02.596876+0000","last_active":"2023-08-25T03:34:05.349890+0000","last_peered":"2023-08-25T03:34:05.349890+0000","last_clean":"2023-08-25T03:34:05.349890+0000","last_became_active":"2023-08-25T03:34:02.596556+0000","last_became_peered":"2023-08-25T03:34:02.596556+0000","last_unstale":"2023-08-25T03:34:05.349890+0000","last_undegraded":"2023-08-25T03:34:05.349890+0000","last_fullsized":"2023-08-25T03:34:05.349890+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:52:25.303751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169267+0000","last_change":"2023-08-25T03:34:03.616290+0000","last_active":"2023-08-25T03:34:05.169267+0000","last_peered":"2023-08-25T03:34:05.169267+0000","last_clean":"2023-08-25T03:34:05.169267+0000","last_became_active":"2023-08-25T03:34:03.616009+0000","last_became_peered":"2023-08-25T03:34:03.616009+0000","last_unstale":"2023-08-25T03:34:05.169267+0000","last_undegraded":"2023-08-25T03:34:05.169267+0000","last_fullsized":"2023-08-25T03:34:05.169267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:08.250998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.350304+0000","last_change":"2023-08-25T03:34:02.604771+0000","last_active":"2023-08-25T03:34:05.350304+0000","last_peered":"2023-08-25T03:34:05.350304+0000","last_clean":"2023-08-25T03:34:05.350304+0000","last_became_active":"2023-08-25T03:34:02.604509+0000","last_became_peered":"2023-08-25T03:34:02.604509+0000","last_unstale":"2023-08-25T03:34:05.350304+0000","last_undegraded":"2023-08-25T03:34:05.350304+0000","last_fullsized":"2023-08-25T03:34:05.350304+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:01:40.446770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349627+0000","last_change":"2023-08-25T03:33:57.570506+0000","last_active":"2023-08-25T03:34:05.349627+0000","last_peered":"2023-08-25T03:34:05.349627+0000","last_clean":"2023-08-25T03:34:05.349627+0000","last_became_active":"2023-08-25T03:33:57.570171+0000","last_became_peered":"2023-08-25T03:33:57.570171+0000","last_unstale":"2023-08-25T03:34:05.349627+0000","last_undegraded":"2023-08-25T03:34:05.349627+0000","last_fullsized":"2023-08-25T03:34:05.349627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:05:19.634775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348810+0000","last_change":"2023-08-25T03:34:03.622270+0000","last_active":"2023-08-25T03:34:05.348810+0000","last_peered":"2023-08-25T03:34:05.348810+0000","last_clean":"2023-08-25T03:34:05.348810+0000","last_became_active":"2023-08-25T03:34:03.621168+0000","last_became_peered":"2023-08-25T03:34:03.621168+0000","last_unstale":"2023-08-25T03:34:05.348810+0000","last_undegraded":"2023-08-25T03:34:05.348810+0000","last_fullsized":"2023-08-25T03:34:05.348810+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:59:37.641683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612720+0000","last_change":"2023-08-25T03:33:57.570598+0000","last_active":"2023-08-25T03:34:04.612720+0000","last_peered":"2023-08-25T03:34:04.612720+0000","last_clean":"2023-08-25T03:34:04.612720+0000","last_became_active":"2023-08-25T03:33:57.570215+0000","last_became_peered":"2023-08-25T03:33:57.570215+0000","last_unstale":"2023-08-25T03:34:04.612720+0000","last_undegraded":"2023-08-25T03:34:04.612720+0000","last_fullsized":"2023-08-25T03:34:04.612720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:40:56.425085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169511+0000","last_change":"2023-08-25T03:34:02.605323+0000","last_active":"2023-08-25T03:34:05.169511+0000","last_peered":"2023-08-25T03:34:05.169511+0000","last_clean":"2023-08-25T03:34:05.169511+0000","last_became_active":"2023-08-25T03:34:02.605111+0000","last_became_peered":"2023-08-25T03:34:02.605111+0000","last_unstale":"2023-08-25T03:34:05.169511+0000","last_undegraded":"2023-08-25T03:34:05.169511+0000","last_fullsized":"2023-08-25T03:34:05.169511+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:57:58.917579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348697+0000","last_change":"2023-08-25T03:34:03.624353+0000","last_active":"2023-08-25T03:34:05.348697+0000","last_peered":"2023-08-25T03:34:05.348697+0000","last_clean":"2023-08-25T03:34:05.348697+0000","last_became_active":"2023-08-25T03:34:03.618468+0000","last_became_peered":"2023-08-25T03:34:03.618468+0000","last_unstale":"2023-08-25T03:34:05.348697+0000","last_undegraded":"2023-08-25T03:34:05.348697+0000","last_fullsized":"2023-08-25T03:34:05.348697+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:41:33.001585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169012+0000","last_change":"2023-08-25T03:34:02.603419+0000","last_active":"2023-08-25T03:34:05.169012+0000","last_peered":"2023-08-25T03:34:05.169012+0000","last_clean":"2023-08-25T03:34:05.169012+0000","last_became_active":"2023-08-25T03:34:02.603082+0000","last_became_peered":"2023-08-25T03:34:02.603082+0000","last_unstale":"2023-08-25T03:34:05.169012+0000","last_undegraded":"2023-08-25T03:34:05.169012+0000","last_fullsized":"2023-08-25T03:34:05.169012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:22:24.873753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611520+0000","last_change":"2023-08-25T03:34:03.618585+0000","last_active":"2023-08-25T03:34:04.611520+0000","last_peered":"2023-08-25T03:34:04.611520+0000","last_clean":"2023-08-25T03:34:04.611520+0000","last_became_active":"2023-08-25T03:34:03.618291+0000","last_became_peered":"2023-08-25T03:34:03.618291+0000","last_unstale":"2023-08-25T03:34:04.611520+0000","last_undegraded":"2023-08-25T03:34:04.611520+0000","last_fullsized":"2023-08-25T03:34:04.611520+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:41:02.448469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611545+0000","last_change":"2023-08-25T03:34:02.601201+0000","last_active":"2023-08-25T03:34:04.611545+0000","last_peered":"2023-08-25T03:34:04.611545+0000","last_clean":"2023-08-25T03:34:04.611545+0000","last_became_active":"2023-08-25T03:34:02.600904+0000","last_became_peered":"2023-08-25T03:34:02.600904+0000","last_unstale":"2023-08-25T03:34:04.611545+0000","last_undegraded":"2023-08-25T03:34:04.611545+0000","last_fullsized":"2023-08-25T03:34:04.611545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:07.999174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349241+0000","last_change":"2023-08-25T03:34:03.626366+0000","last_active":"2023-08-25T03:34:05.349241+0000","last_peered":"2023-08-25T03:34:05.349241+0000","last_clean":"2023-08-25T03:34:05.349241+0000","last_became_active":"2023-08-25T03:34:03.625496+0000","last_became_peered":"2023-08-25T03:34:03.625496+0000","last_unstale":"2023-08-25T03:34:05.349241+0000","last_undegraded":"2023-08-25T03:34:05.349241+0000","last_fullsized":"2023-08-25T03:34:05.349241+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T03:41:42.153100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612260+0000","last_change":"2023-08-25T03:34:02.603423+0000","last_active":"2023-08-25T03:34:04.612260+0000","last_peered":"2023-08-25T03:34:04.612260+0000","last_clean":"2023-08-25T03:34:04.612260+0000","last_became_active":"2023-08-25T03:34:02.603077+0000","last_became_peered":"2023-08-25T03:34:02.603077+0000","last_unstale":"2023-08-25T03:34:04.612260+0000","last_undegraded":"2023-08-25T03:34:04.612260+0000","last_fullsized":"2023-08-25T03:34:04.612260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:57:58.917579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348880+0000","last_change":"2023-08-25T03:34:03.622038+0000","last_active":"2023-08-25T03:34:05.348880+0000","last_peered":"2023-08-25T03:34:05.348880+0000","last_clean":"2023-08-25T03:34:05.348880+0000","last_became_active":"2023-08-25T03:34:03.621747+0000","last_became_peered":"2023-08-25T03:34:03.621747+0000","last_unstale":"2023-08-25T03:34:05.348880+0000","last_undegraded":"2023-08-25T03:34:05.348880+0000","last_fullsized":"2023-08-25T03:34:05.348880+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:10:04.553228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.350130+0000","last_change":"2023-08-25T03:34:02.598272+0000","last_active":"2023-08-25T03:34:05.350130+0000","last_peered":"2023-08-25T03:34:05.350130+0000","last_clean":"2023-08-25T03:34:05.350130+0000","last_became_active":"2023-08-25T03:34:02.597630+0000","last_became_peered":"2023-08-25T03:34:02.597630+0000","last_unstale":"2023-08-25T03:34:05.350130+0000","last_undegraded":"2023-08-25T03:34:05.350130+0000","last_fullsized":"2023-08-25T03:34:05.350130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:03:05.441386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348872+0000","last_change":"2023-08-25T03:34:03.628379+0000","last_active":"2023-08-25T03:34:05.348872+0000","last_peered":"2023-08-25T03:34:05.348872+0000","last_clean":"2023-08-25T03:34:05.348872+0000","last_became_active":"2023-08-25T03:34:03.627870+0000","last_became_peered":"2023-08-25T03:34:03.627870+0000","last_unstale":"2023-08-25T03:34:05.348872+0000","last_undegraded":"2023-08-25T03:34:05.348872+0000","last_fullsized":"2023-08-25T03:34:05.348872+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:49:11.917056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169428+0000","last_change":"2023-08-25T03:34:02.597748+0000","last_active":"2023-08-25T03:34:05.169428+0000","last_peered":"2023-08-25T03:34:05.169428+0000","last_clean":"2023-08-25T03:34:05.169428+0000","last_became_active":"2023-08-25T03:34:02.597456+0000","last_became_peered":"2023-08-25T03:34:02.597456+0000","last_unstale":"2023-08-25T03:34:05.169428+0000","last_undegraded":"2023-08-25T03:34:05.169428+0000","last_fullsized":"2023-08-25T03:34:05.169428+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:01:40.446770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348450+0000","last_change":"2023-08-25T03:34:03.619886+0000","last_active":"2023-08-25T03:34:05.348450+0000","last_peered":"2023-08-25T03:34:05.348450+0000","last_clean":"2023-08-25T03:34:05.348450+0000","last_became_active":"2023-08-25T03:34:03.618445+0000","last_became_peered":"2023-08-25T03:34:03.618445+0000","last_unstale":"2023-08-25T03:34:05.348450+0000","last_undegraded":"2023-08-25T03:34:05.348450+0000","last_fullsized":"2023-08-25T03:34:05.348450+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:59.004627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612396+0000","last_change":"2023-08-25T03:34:03.612902+0000","last_active":"2023-08-25T03:34:04.612396+0000","last_peered":"2023-08-25T03:34:04.612396+0000","last_clean":"2023-08-25T03:34:04.612396+0000","last_became_active":"2023-08-25T03:34:03.612582+0000","last_became_peered":"2023-08-25T03:34:03.612582+0000","last_unstale":"2023-08-25T03:34:04.612396+0000","last_undegraded":"2023-08-25T03:34:04.612396+0000","last_fullsized":"2023-08-25T03:34:04.612396+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:01:15.550047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348872+0000","last_change":"2023-08-25T03:34:03.623559+0000","last_active":"2023-08-25T03:34:05.348872+0000","last_peered":"2023-08-25T03:34:05.348872+0000","last_clean":"2023-08-25T03:34:05.348872+0000","last_became_active":"2023-08-25T03:34:03.622782+0000","last_became_peered":"2023-08-25T03:34:03.622782+0000","last_unstale":"2023-08-25T03:34:05.348872+0000","last_undegraded":"2023-08-25T03:34:05.348872+0000","last_fullsized":"2023-08-25T03:34:05.348872+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:59.004627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348614+0000","last_change":"2023-08-25T03:34:03.622024+0000","last_active":"2023-08-25T03:34:05.348614+0000","last_peered":"2023-08-25T03:34:05.348614+0000","last_clean":"2023-08-25T03:34:05.348614+0000","last_became_active":"2023-08-25T03:34:03.619673+0000","last_became_peered":"2023-08-25T03:34:03.619673+0000","last_unstale":"2023-08-25T03:34:05.348614+0000","last_undegraded":"2023-08-25T03:34:05.348614+0000","last_fullsized":"2023-08-25T03:34:05.348614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:47:13.833805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349801+0000","last_change":"2023-08-25T03:34:03.621260+0000","last_active":"2023-08-25T03:34:05.349801+0000","last_peered":"2023-08-25T03:34:05.349801+0000","last_clean":"2023-08-25T03:34:05.349801+0000","last_became_active":"2023-08-25T03:34:03.620890+0000","last_became_peered":"2023-08-25T03:34:03.620890+0000","last_unstale":"2023-08-25T03:34:05.349801+0000","last_undegraded":"2023-08-25T03:34:05.349801+0000","last_fullsized":"2023-08-25T03:34:05.349801+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:52:26.311165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349539+0000","last_change":"2023-08-25T03:34:03.615228+0000","last_active":"2023-08-25T03:34:05.349539+0000","last_peered":"2023-08-25T03:34:05.349539+0000","last_clean":"2023-08-25T03:34:05.349539+0000","last_became_active":"2023-08-25T03:34:03.613949+0000","last_became_peered":"2023-08-25T03:34:03.613949+0000","last_unstale":"2023-08-25T03:34:05.349539+0000","last_undegraded":"2023-08-25T03:34:05.349539+0000","last_fullsized":"2023-08-25T03:34:05.349539+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:41:33.001585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542154,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":872448,"data_stored":685346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]}]},{"osd":6,"up_from":14,"seq":60129542154,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":249856,"data_stored":93150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":249856,"data_stored":93150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.6120000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.0950000000000006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4820000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.4400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.5229999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.5590000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":847872,"data_stored":684100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.19600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":249856,"data_stored":93150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":93240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":4,"up_from":14,"seq":60129542154,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":94818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":93802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-25T03:34:17.201 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T03:34:17.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.357+0000 7fe646899700 1 Processor -- start 2023-08-25T03:34:17.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.357+0000 7fe646899700 1 -- start start 2023-08-25T03:34:17.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.358+0000 7fe646899700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe6400ccd50 0x7fe6400cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:17.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.358+0000 7fe646899700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6400cd6e0 con 0x7fe64015a410 2023-08-25T03:34:17.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.358+0000 7fe646899700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe640071dc0 con 0x7fe64015a030 2023-08-25T03:34:17.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.358+0000 7fe646899700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6400720a0 con 0x7fe6400ccd50 2023-08-25T03:34:17.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.358+0000 7fe63f7fe700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fe64015a410 0x7fe6400ce170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46404/0 (socket says 172.21.15.2:46404) 2023-08-25T03:34:17.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.358+0000 7fe63f7fe700 1 -- 172.21.15.2:0/851696182 learned_addr learned my addr 172.21.15.2:0/851696182 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:17.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.359+0000 7fe644e36700 1 --2- 172.21.15.2:0/851696182 >> v2:172.21.15.31:3300/0 conn(0x7fe6400ccd50 0x7fe6400cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:17.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.359+0000 7fe63effd700 1 -- 172.21.15.2:0/851696182 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3232145701 0 0) 0x7fe628001110 con 0x7fe64015a410 2023-08-25T03:34:17.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.359+0000 7fe63effd700 1 -- 172.21.15.2:0/851696182 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe62c0035b0 con 0x7fe64015a410 2023-08-25T03:34:17.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.359+0000 7fe644e36700 1 -- 172.21.15.2:0/851696182 >> v1:172.21.15.31:6789/0 conn(0x7fe64015a030 legacy=0x7fe6400cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:17.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.359+0000 7fe644e36700 1 -- 172.21.15.2:0/851696182 >> v1:172.21.15.2:6789/0 conn(0x7fe64015a410 legacy=0x7fe6400ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:17.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.359+0000 7fe644e36700 1 -- 172.21.15.2:0/851696182 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6400cb4b0 con 0x7fe6400ccd50 2023-08-25T03:34:17.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.359+0000 7fe644e36700 1 --2- 172.21.15.2:0/851696182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6400ccd50 0x7fe6400cd1a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe638009d90 tx=0x7fe63800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec4d23af4792a21b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:17.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.359+0000 7fe63effd700 1 -- 172.21.15.2:0/851696182 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63800e070 con 0x7fe6400ccd50 2023-08-25T03:34:17.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.359+0000 7fe63effd700 1 -- 172.21.15.2:0/851696182 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe63800bb20 con 0x7fe6400ccd50 2023-08-25T03:34:17.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.359+0000 7fe63effd700 1 -- 172.21.15.2:0/851696182 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe638012430 con 0x7fe6400ccd50 2023-08-25T03:34:17.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.360+0000 7fe646899700 1 -- 172.21.15.2:0/851696182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6400ccd50 msgr2=0x7fe6400cd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:17.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.360+0000 7fe646899700 1 --2- 172.21.15.2:0/851696182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6400ccd50 0x7fe6400cd1a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe638009d90 tx=0x7fe63800b330 comp rx=0 tx=0).stop 2023-08-25T03:34:17.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.360+0000 7fe646899700 1 -- 172.21.15.2:0/851696182 shutdown_connections 2023-08-25T03:34:17.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.360+0000 7fe646899700 1 --2- 172.21.15.2:0/851696182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6400ccd50 0x7fe6400cd1a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:17.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.360+0000 7fe646899700 1 -- 172.21.15.2:0/851696182 >> 172.21.15.2:0/851696182 conn(0x7fe6400b1cf0 msgr2=0x7fe6400b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:17.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.360+0000 7fe646899700 1 -- 172.21.15.2:0/851696182 shutdown_connections 2023-08-25T03:34:17.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.360+0000 7fe646899700 1 -- 172.21.15.2:0/851696182 wait complete. 2023-08-25T03:34:17.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.361+0000 7fe646899700 1 Processor -- start 2023-08-25T03:34:17.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.361+0000 7fe646899700 1 -- start start 2023-08-25T03:34:17.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.362+0000 7fe646899700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6400ccd50 0x7fe6400c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:17.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.362+0000 7fe646899700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe64015a030 0x7fe64015c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:17.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.362+0000 7fe646899700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe64015a410 0x7fe6400ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:17.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.362+0000 7fe646899700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6400ace20 con 0x7fe64015a410 2023-08-25T03:34:17.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.362+0000 7fe646899700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6400ad010 con 0x7fe6400ccd50 2023-08-25T03:34:17.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.362+0000 7fe646899700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6400c4c40 con 0x7fe64015a030 2023-08-25T03:34:17.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.362+0000 7fe644e36700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe64015a410 0x7fe6400ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:17.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.362+0000 7fe644e36700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe64015a410 0x7fe6400ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59746/0 (socket says 172.21.15.2:59746) 2023-08-25T03:34:17.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.362+0000 7fe63ffff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6400ccd50 0x7fe6400c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:17.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.362+0000 7fe644e36700 1 -- 172.21.15.2:0/3175608037 learned_addr learned my addr 172.21.15.2:0/3175608037 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:17.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.362+0000 7fe63f7fe700 1 --2- 172.21.15.2:0/3175608037 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe64015a030 0x7fe64015c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:17.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.362+0000 7fe644e36700 1 -- 172.21.15.2:0/3175608037 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe64015a030 msgr2=0x7fe64015c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:17.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.362+0000 7fe644e36700 1 --2- 172.21.15.2:0/3175608037 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe64015a030 0x7fe64015c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:17.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.362+0000 7fe644e36700 1 -- 172.21.15.2:0/3175608037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6400ccd50 msgr2=0x7fe6400c64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:17.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.362+0000 7fe644e36700 1 --2- 172.21.15.2:0/3175608037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6400ccd50 0x7fe6400c64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:17.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.362+0000 7fe644e36700 1 -- 172.21.15.2:0/3175608037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe638009a70 con 0x7fe64015a410 2023-08-25T03:34:17.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.363+0000 7fe63ffff700 1 --2- 172.21.15.2:0/3175608037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6400ccd50 0x7fe6400c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:34:17.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.363+0000 7fe644e36700 1 --2- 172.21.15.2:0/3175608037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe64015a410 0x7fe6400ac8e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe6400b2bd0 tx=0x7fe638004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:17.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.363+0000 7fe63cff9700 1 -- 172.21.15.2:0/3175608037 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe638012430 con 0x7fe64015a410 2023-08-25T03:34:17.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.363+0000 7fe63cff9700 1 -- 172.21.15.2:0/3175608037 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe638020070 con 0x7fe64015a410 2023-08-25T03:34:17.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.363+0000 7fe646899700 1 -- 172.21.15.2:0/3175608037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6400c4f20 con 0x7fe64015a410 2023-08-25T03:34:17.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.363+0000 7fe63cff9700 1 -- 172.21.15.2:0/3175608037 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe638004280 con 0x7fe64015a410 2023-08-25T03:34:17.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.363+0000 7fe646899700 1 -- 172.21.15.2:0/3175608037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6400c54e0 con 0x7fe64015a410 2023-08-25T03:34:17.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.364+0000 7fe646899700 1 -- 172.21.15.2:0/3175608037 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe640066f70 con 0x7fe64015a410 2023-08-25T03:34:17.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.364+0000 7fe63cff9700 1 -- 172.21.15.2:0/3175608037 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe6380043e0 con 0x7fe64015a410 2023-08-25T03:34:17.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.365+0000 7fe63cff9700 1 --2- 172.21.15.2:0/3175608037 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe62006a0e0 0x7fe62006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:17.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.365+0000 7fe63ffff700 1 --2- 172.21.15.2:0/3175608037 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe62006a0e0 0x7fe62006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:17.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.365+0000 7fe63cff9700 1 -- 172.21.15.2:0/3175608037 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe638022070 con 0x7fe64015a410 2023-08-25T03:34:17.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.365+0000 7fe63ffff700 1 --2- 172.21.15.2:0/3175608037 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe62006a0e0 0x7fe62006c5a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe630007fd0 tx=0x7fe630009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:17.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.369+0000 7fe63cff9700 1 -- 172.21.15.2:0/3175608037 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe638055ca0 con 0x7fe64015a410 2023-08-25T03:34:17.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.567+0000 7fe646899700 1 -- 172.21.15.2:0/3175608037 --> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe6400cc3c0 con 0x7fe62006a0e0 2023-08-25T03:34:17.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.575+0000 7fe63cff9700 1 -- 172.21.15.2:0/3175608037 <== mgr.4108 v2:172.21.15.2:6832/3528779779 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286680 (secure 0 0 0) 0x7fe630001950 con 0x7fe62006a0e0 2023-08-25T03:34:17.578 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:17.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.578+0000 7fe646899700 1 -- 172.21.15.2:0/3175608037 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe62006a0e0 msgr2=0x7fe62006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:17.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.578+0000 7fe646899700 1 --2- 172.21.15.2:0/3175608037 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe62006a0e0 0x7fe62006c5a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe630007fd0 tx=0x7fe630009040 comp rx=0 tx=0).stop 2023-08-25T03:34:17.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.578+0000 7fe646899700 1 -- 172.21.15.2:0/3175608037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe64015a410 msgr2=0x7fe6400ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:17.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.578+0000 7fe646899700 1 --2- 172.21.15.2:0/3175608037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe64015a410 0x7fe6400ac8e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe6400b2bd0 tx=0x7fe638004ad0 comp rx=0 tx=0).stop 2023-08-25T03:34:17.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.578+0000 7fe646899700 1 -- 172.21.15.2:0/3175608037 shutdown_connections 2023-08-25T03:34:17.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.578+0000 7fe646899700 1 --2- 172.21.15.2:0/3175608037 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe62006a0e0 0x7fe62006c5a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:17.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.578+0000 7fe646899700 1 --2- 172.21.15.2:0/3175608037 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe64015a410 0x7fe6400ac8e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:17.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.578+0000 7fe646899700 1 --2- 172.21.15.2:0/3175608037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe6400ccd50 0x7fe6400c64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:17.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.578+0000 7fe646899700 1 --2- 172.21.15.2:0/3175608037 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe64015a030 0x7fe64015c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:17.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.578+0000 7fe646899700 1 -- 172.21.15.2:0/3175608037 >> 172.21.15.2:0/3175608037 conn(0x7fe6400b1cf0 msgr2=0x7fe6401595b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:17.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.578+0000 7fe646899700 1 -- 172.21.15.2:0/3175608037 shutdown_connections 2023-08-25T03:34:17.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.578+0000 7fe646899700 1 -- 172.21.15.2:0/3175608037 wait complete. 2023-08-25T03:34:17.583 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-08-25T03:34:17.591 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2023-08-25T03:34:17.507091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219352,"kb_used_data":3176,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754755368,"statfs":{"total":773094113280,"available":772869496832,"internally_reserved":0,"allocated":3252224,"data_stored":1930756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002820"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611410+0000","last_change":"2023-08-25T03:34:03.628457+0000","last_active":"2023-08-25T03:34:04.611410+0000","last_peered":"2023-08-25T03:34:04.611410+0000","last_clean":"2023-08-25T03:34:04.611410+0000","last_became_active":"2023-08-25T03:34:03.627977+0000","last_became_peered":"2023-08-25T03:34:03.627977+0000","last_unstale":"2023-08-25T03:34:04.611410+0000","last_undegraded":"2023-08-25T03:34:04.611410+0000","last_fullsized":"2023-08-25T03:34:04.611410+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:01:15.550047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611582+0000","last_change":"2023-08-25T03:34:03.616304+0000","last_active":"2023-08-25T03:34:04.611582+0000","last_peered":"2023-08-25T03:34:04.611582+0000","last_clean":"2023-08-25T03:34:04.611582+0000","last_became_active":"2023-08-25T03:34:03.615917+0000","last_became_peered":"2023-08-25T03:34:03.615917+0000","last_unstale":"2023-08-25T03:34:04.611582+0000","last_undegraded":"2023-08-25T03:34:04.611582+0000","last_fullsized":"2023-08-25T03:34:04.611582+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:08.250998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348968+0000","last_change":"2023-08-25T03:34:03.622799+0000","last_active":"2023-08-25T03:34:05.348968+0000","last_peered":"2023-08-25T03:34:05.348968+0000","last_clean":"2023-08-25T03:34:05.348968+0000","last_became_active":"2023-08-25T03:34:03.622602+0000","last_became_peered":"2023-08-25T03:34:03.622602+0000","last_unstale":"2023-08-25T03:34:05.348968+0000","last_undegraded":"2023-08-25T03:34:05.348968+0000","last_fullsized":"2023-08-25T03:34:05.348968+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:45:56.260357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.613438+0000","last_change":"2023-08-25T03:34:03.615254+0000","last_active":"2023-08-25T03:34:04.613438+0000","last_peered":"2023-08-25T03:34:04.613438+0000","last_clean":"2023-08-25T03:34:04.613438+0000","last_became_active":"2023-08-25T03:34:03.614917+0000","last_became_peered":"2023-08-25T03:34:03.614917+0000","last_unstale":"2023-08-25T03:34:04.613438+0000","last_undegraded":"2023-08-25T03:34:04.613438+0000","last_fullsized":"2023-08-25T03:34:04.613438+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:43:29.368622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169198+0000","last_change":"2023-08-25T03:34:03.626638+0000","last_active":"2023-08-25T03:34:05.169198+0000","last_peered":"2023-08-25T03:34:05.169198+0000","last_clean":"2023-08-25T03:34:05.169198+0000","last_became_active":"2023-08-25T03:34:03.626317+0000","last_became_peered":"2023-08-25T03:34:03.626317+0000","last_unstale":"2023-08-25T03:34:05.169198+0000","last_undegraded":"2023-08-25T03:34:05.169198+0000","last_fullsized":"2023-08-25T03:34:05.169198+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:10:04.553228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348619+0000","last_change":"2023-08-25T03:34:03.619323+0000","last_active":"2023-08-25T03:34:05.348619+0000","last_peered":"2023-08-25T03:34:05.348619+0000","last_clean":"2023-08-25T03:34:05.348619+0000","last_became_active":"2023-08-25T03:34:03.618985+0000","last_became_peered":"2023-08-25T03:34:03.618985+0000","last_unstale":"2023-08-25T03:34:05.348619+0000","last_undegraded":"2023-08-25T03:34:05.348619+0000","last_fullsized":"2023-08-25T03:34:05.348619+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:02:02.415298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.263861+0000","last_change":"2023-08-25T03:34:03.617493+0000","last_active":"2023-08-25T03:34:05.263861+0000","last_peered":"2023-08-25T03:34:05.263861+0000","last_clean":"2023-08-25T03:34:05.263861+0000","last_became_active":"2023-08-25T03:34:03.617167+0000","last_became_peered":"2023-08-25T03:34:03.617167+0000","last_unstale":"2023-08-25T03:34:05.263861+0000","last_undegraded":"2023-08-25T03:34:05.263861+0000","last_fullsized":"2023-08-25T03:34:05.263861+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:57:59.924993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611074+0000","last_change":"2023-08-25T03:34:03.625710+0000","last_active":"2023-08-25T03:34:04.611074+0000","last_peered":"2023-08-25T03:34:04.611074+0000","last_clean":"2023-08-25T03:34:04.611074+0000","last_became_active":"2023-08-25T03:34:03.625404+0000","last_became_peered":"2023-08-25T03:34:03.625404+0000","last_unstale":"2023-08-25T03:34:04.611074+0000","last_undegraded":"2023-08-25T03:34:04.611074+0000","last_fullsized":"2023-08-25T03:34:04.611074+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:02:02.415298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169221+0000","last_change":"2023-08-25T03:34:03.626723+0000","last_active":"2023-08-25T03:34:05.169221+0000","last_peered":"2023-08-25T03:34:05.169221+0000","last_clean":"2023-08-25T03:34:05.169221+0000","last_became_active":"2023-08-25T03:34:03.626419+0000","last_became_peered":"2023-08-25T03:34:03.626419+0000","last_unstale":"2023-08-25T03:34:05.169221+0000","last_undegraded":"2023-08-25T03:34:05.169221+0000","last_fullsized":"2023-08-25T03:34:05.169221+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:09.006588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611787+0000","last_change":"2023-08-25T03:34:03.618247+0000","last_active":"2023-08-25T03:34:04.611787+0000","last_peered":"2023-08-25T03:34:04.611787+0000","last_clean":"2023-08-25T03:34:04.611787+0000","last_became_active":"2023-08-25T03:34:03.617830+0000","last_became_peered":"2023-08-25T03:34:03.617830+0000","last_unstale":"2023-08-25T03:34:04.611787+0000","last_undegraded":"2023-08-25T03:34:04.611787+0000","last_fullsized":"2023-08-25T03:34:04.611787+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:01:41.454184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349005+0000","last_change":"2023-08-25T03:34:03.622380+0000","last_active":"2023-08-25T03:34:05.349005+0000","last_peered":"2023-08-25T03:34:05.349005+0000","last_clean":"2023-08-25T03:34:05.349005+0000","last_became_active":"2023-08-25T03:34:03.619691+0000","last_became_peered":"2023-08-25T03:34:03.619691+0000","last_unstale":"2023-08-25T03:34:05.349005+0000","last_undegraded":"2023-08-25T03:34:05.349005+0000","last_fullsized":"2023-08-25T03:34:05.349005+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:08.250998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611500+0000","last_change":"2023-08-25T03:34:03.624204+0000","last_active":"2023-08-25T03:34:04.611500+0000","last_peered":"2023-08-25T03:34:04.611500+0000","last_clean":"2023-08-25T03:34:04.611500+0000","last_became_active":"2023-08-25T03:34:03.623945+0000","last_became_peered":"2023-08-25T03:34:03.623945+0000","last_unstale":"2023-08-25T03:34:04.611500+0000","last_undegraded":"2023-08-25T03:34:04.611500+0000","last_fullsized":"2023-08-25T03:34:04.611500+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:25:43.507515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611336+0000","last_change":"2023-08-25T03:34:03.622941+0000","last_active":"2023-08-25T03:34:04.611336+0000","last_peered":"2023-08-25T03:34:04.611336+0000","last_clean":"2023-08-25T03:34:04.611336+0000","last_became_active":"2023-08-25T03:34:03.622627+0000","last_became_peered":"2023-08-25T03:34:03.622627+0000","last_unstale":"2023-08-25T03:34:04.611336+0000","last_undegraded":"2023-08-25T03:34:04.611336+0000","last_fullsized":"2023-08-25T03:34:04.611336+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:23:49.310992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611493+0000","last_change":"2023-08-25T03:34:03.616557+0000","last_active":"2023-08-25T03:34:04.611493+0000","last_peered":"2023-08-25T03:34:04.611493+0000","last_clean":"2023-08-25T03:34:04.611493+0000","last_became_active":"2023-08-25T03:34:03.615850+0000","last_became_peered":"2023-08-25T03:34:03.615850+0000","last_unstale":"2023-08-25T03:34:04.611493+0000","last_undegraded":"2023-08-25T03:34:04.611493+0000","last_fullsized":"2023-08-25T03:34:04.611493+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:10:04.553228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264238+0000","last_change":"2023-08-25T03:34:03.617319+0000","last_active":"2023-08-25T03:34:05.264238+0000","last_peered":"2023-08-25T03:34:05.264238+0000","last_clean":"2023-08-25T03:34:05.264238+0000","last_became_active":"2023-08-25T03:34:03.617012+0000","last_became_peered":"2023-08-25T03:34:03.617012+0000","last_unstale":"2023-08-25T03:34:05.264238+0000","last_undegraded":"2023-08-25T03:34:05.264238+0000","last_fullsized":"2023-08-25T03:34:05.264238+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:00:42.442989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348569+0000","last_change":"2023-08-25T03:34:03.615370+0000","last_active":"2023-08-25T03:34:05.348569+0000","last_peered":"2023-08-25T03:34:05.348569+0000","last_clean":"2023-08-25T03:34:05.348569+0000","last_became_active":"2023-08-25T03:34:03.615076+0000","last_became_peered":"2023-08-25T03:34:03.615076+0000","last_unstale":"2023-08-25T03:34:05.348569+0000","last_undegraded":"2023-08-25T03:34:05.348569+0000","last_fullsized":"2023-08-25T03:34:05.348569+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:49:11.917056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169218+0000","last_change":"2023-08-25T03:34:03.627117+0000","last_active":"2023-08-25T03:34:05.169218+0000","last_peered":"2023-08-25T03:34:05.169218+0000","last_clean":"2023-08-25T03:34:05.169218+0000","last_became_active":"2023-08-25T03:34:03.626473+0000","last_became_peered":"2023-08-25T03:34:03.626473+0000","last_unstale":"2023-08-25T03:34:05.169218+0000","last_undegraded":"2023-08-25T03:34:05.169218+0000","last_fullsized":"2023-08-25T03:34:05.169218+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:52:26.311165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349494+0000","last_change":"2023-08-25T03:34:03.618966+0000","last_active":"2023-08-25T03:34:05.349494+0000","last_peered":"2023-08-25T03:34:05.349494+0000","last_clean":"2023-08-25T03:34:05.349494+0000","last_became_active":"2023-08-25T03:34:03.618636+0000","last_became_peered":"2023-08-25T03:34:03.618636+0000","last_unstale":"2023-08-25T03:34:05.349494+0000","last_undegraded":"2023-08-25T03:34:05.349494+0000","last_fullsized":"2023-08-25T03:34:05.349494+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:33:29.470663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611519+0000","last_change":"2023-08-25T03:34:03.605314+0000","last_active":"2023-08-25T03:34:04.611519+0000","last_peered":"2023-08-25T03:34:04.611519+0000","last_clean":"2023-08-25T03:34:04.611519+0000","last_became_active":"2023-08-25T03:34:03.605008+0000","last_became_peered":"2023-08-25T03:34:03.605008+0000","last_unstale":"2023-08-25T03:34:04.611519+0000","last_undegraded":"2023-08-25T03:34:04.611519+0000","last_fullsized":"2023-08-25T03:34:04.611519+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:47:39.963816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349001+0000","last_change":"2023-08-25T03:34:03.621319+0000","last_active":"2023-08-25T03:34:05.349001+0000","last_peered":"2023-08-25T03:34:05.349001+0000","last_clean":"2023-08-25T03:34:05.349001+0000","last_became_active":"2023-08-25T03:34:03.619115+0000","last_became_peered":"2023-08-25T03:34:03.619115+0000","last_unstale":"2023-08-25T03:34:05.349001+0000","last_undegraded":"2023-08-25T03:34:05.349001+0000","last_fullsized":"2023-08-25T03:34:05.349001+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:23:05.976771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264207+0000","last_change":"2023-08-25T03:34:03.622785+0000","last_active":"2023-08-25T03:34:05.264207+0000","last_peered":"2023-08-25T03:34:05.264207+0000","last_clean":"2023-08-25T03:34:05.264207+0000","last_became_active":"2023-08-25T03:34:03.622469+0000","last_became_peered":"2023-08-25T03:34:03.622469+0000","last_unstale":"2023-08-25T03:34:05.264207+0000","last_undegraded":"2023-08-25T03:34:05.264207+0000","last_fullsized":"2023-08-25T03:34:05.264207+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:35:42.993608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.168950+0000","last_change":"2023-08-25T03:34:03.620545+0000","last_active":"2023-08-25T03:34:05.168950+0000","last_peered":"2023-08-25T03:34:05.168950+0000","last_clean":"2023-08-25T03:34:05.168950+0000","last_became_active":"2023-08-25T03:34:03.619698+0000","last_became_peered":"2023-08-25T03:34:03.619698+0000","last_unstale":"2023-08-25T03:34:05.168950+0000","last_undegraded":"2023-08-25T03:34:05.168950+0000","last_fullsized":"2023-08-25T03:34:05.168950+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:41:33.001585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349094+0000","last_change":"2023-08-25T03:34:03.631189+0000","last_active":"2023-08-25T03:34:05.349094+0000","last_peered":"2023-08-25T03:34:05.349094+0000","last_clean":"2023-08-25T03:34:05.349094+0000","last_became_active":"2023-08-25T03:34:03.630807+0000","last_became_peered":"2023-08-25T03:34:03.630807+0000","last_unstale":"2023-08-25T03:34:05.349094+0000","last_undegraded":"2023-08-25T03:34:05.349094+0000","last_fullsized":"2023-08-25T03:34:05.349094+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:41:33.001585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.263844+0000","last_change":"2023-08-25T03:34:03.623150+0000","last_active":"2023-08-25T03:34:05.263844+0000","last_peered":"2023-08-25T03:34:05.263844+0000","last_clean":"2023-08-25T03:34:05.263844+0000","last_became_active":"2023-08-25T03:34:03.622796+0000","last_became_peered":"2023-08-25T03:34:03.622796+0000","last_unstale":"2023-08-25T03:34:05.263844+0000","last_undegraded":"2023-08-25T03:34:05.263844+0000","last_fullsized":"2023-08-25T03:34:05.263844+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:01:41.454184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349917+0000","last_change":"2023-08-25T03:34:03.620031+0000","last_active":"2023-08-25T03:34:05.349917+0000","last_peered":"2023-08-25T03:34:05.349917+0000","last_clean":"2023-08-25T03:34:05.349917+0000","last_became_active":"2023-08-25T03:34:03.619738+0000","last_became_peered":"2023-08-25T03:34:03.619738+0000","last_unstale":"2023-08-25T03:34:05.349917+0000","last_undegraded":"2023-08-25T03:34:05.349917+0000","last_fullsized":"2023-08-25T03:34:05.349917+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:08.250998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349705+0000","last_change":"2023-08-25T03:34:03.632252+0000","last_active":"2023-08-25T03:34:05.349705+0000","last_peered":"2023-08-25T03:34:05.349705+0000","last_clean":"2023-08-25T03:34:05.349705+0000","last_became_active":"2023-08-25T03:34:03.631975+0000","last_became_peered":"2023-08-25T03:34:03.631975+0000","last_unstale":"2023-08-25T03:34:05.349705+0000","last_undegraded":"2023-08-25T03:34:05.349705+0000","last_fullsized":"2023-08-25T03:34:05.349705+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:22:25.881167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611728+0000","last_change":"2023-08-25T03:34:03.613385+0000","last_active":"2023-08-25T03:34:04.611728+0000","last_peered":"2023-08-25T03:34:04.611728+0000","last_clean":"2023-08-25T03:34:04.611728+0000","last_became_active":"2023-08-25T03:34:03.613110+0000","last_became_peered":"2023-08-25T03:34:03.613110+0000","last_unstale":"2023-08-25T03:34:04.611728+0000","last_undegraded":"2023-08-25T03:34:04.611728+0000","last_fullsized":"2023-08-25T03:34:04.611728+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:03:06.448800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348876+0000","last_change":"2023-08-25T03:34:02.608606+0000","last_active":"2023-08-25T03:34:05.348876+0000","last_peered":"2023-08-25T03:34:05.348876+0000","last_clean":"2023-08-25T03:34:05.348876+0000","last_became_active":"2023-08-25T03:34:02.608298+0000","last_became_peered":"2023-08-25T03:34:02.608298+0000","last_unstale":"2023-08-25T03:34:05.348876+0000","last_undegraded":"2023-08-25T03:34:05.348876+0000","last_fullsized":"2023-08-25T03:34:05.348876+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:52:25.303751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.350009+0000","last_change":"2023-08-25T03:34:03.619019+0000","last_active":"2023-08-25T03:34:05.350009+0000","last_peered":"2023-08-25T03:34:05.350009+0000","last_clean":"2023-08-25T03:34:05.350009+0000","last_became_active":"2023-08-25T03:34:03.618719+0000","last_became_peered":"2023-08-25T03:34:03.618719+0000","last_unstale":"2023-08-25T03:34:05.350009+0000","last_undegraded":"2023-08-25T03:34:05.350009+0000","last_fullsized":"2023-08-25T03:34:05.350009+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:23:05.976771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349898+0000","last_change":"2023-08-25T03:34:02.599366+0000","last_active":"2023-08-25T03:34:05.349898+0000","last_peered":"2023-08-25T03:34:05.349898+0000","last_clean":"2023-08-25T03:34:05.349898+0000","last_became_active":"2023-08-25T03:34:02.599105+0000","last_became_peered":"2023-08-25T03:34:02.599105+0000","last_unstale":"2023-08-25T03:34:05.349898+0000","last_undegraded":"2023-08-25T03:34:05.349898+0000","last_fullsized":"2023-08-25T03:34:05.349898+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:03:05.441386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169298+0000","last_change":"2023-08-25T03:34:03.615711+0000","last_active":"2023-08-25T03:34:05.169298+0000","last_peered":"2023-08-25T03:34:05.169298+0000","last_clean":"2023-08-25T03:34:05.169298+0000","last_became_active":"2023-08-25T03:34:03.615315+0000","last_became_peered":"2023-08-25T03:34:03.615315+0000","last_unstale":"2023-08-25T03:34:05.169298+0000","last_undegraded":"2023-08-25T03:34:05.169298+0000","last_fullsized":"2023-08-25T03:34:05.169298+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:03:06.448800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349079+0000","last_change":"2023-08-25T03:34:02.608397+0000","last_active":"2023-08-25T03:34:05.349079+0000","last_peered":"2023-08-25T03:34:05.349079+0000","last_clean":"2023-08-25T03:34:05.349079+0000","last_became_active":"2023-08-25T03:34:02.608067+0000","last_became_peered":"2023-08-25T03:34:02.608067+0000","last_unstale":"2023-08-25T03:34:05.349079+0000","last_undegraded":"2023-08-25T03:34:05.349079+0000","last_fullsized":"2023-08-25T03:34:05.349079+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:07.243584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612775+0000","last_change":"2023-08-25T03:34:03.613614+0000","last_active":"2023-08-25T03:34:04.612775+0000","last_peered":"2023-08-25T03:34:04.612775+0000","last_clean":"2023-08-25T03:34:04.612775+0000","last_became_active":"2023-08-25T03:34:03.613291+0000","last_became_peered":"2023-08-25T03:34:03.613291+0000","last_unstale":"2023-08-25T03:34:04.612775+0000","last_undegraded":"2023-08-25T03:34:04.612775+0000","last_fullsized":"2023-08-25T03:34:04.612775+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:38:56.062178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169421+0000","last_change":"2023-08-25T03:34:02.603475+0000","last_active":"2023-08-25T03:34:05.169421+0000","last_peered":"2023-08-25T03:34:05.169421+0000","last_clean":"2023-08-25T03:34:05.169421+0000","last_became_active":"2023-08-25T03:34:02.603129+0000","last_became_peered":"2023-08-25T03:34:02.603129+0000","last_unstale":"2023-08-25T03:34:05.169421+0000","last_undegraded":"2023-08-25T03:34:05.169421+0000","last_fullsized":"2023-08-25T03:34:05.169421+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:27:14.892441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349004+0000","last_change":"2023-08-25T03:34:03.615004+0000","last_active":"2023-08-25T03:34:05.349004+0000","last_peered":"2023-08-25T03:34:05.349004+0000","last_clean":"2023-08-25T03:34:05.349004+0000","last_became_active":"2023-08-25T03:34:03.614699+0000","last_became_peered":"2023-08-25T03:34:03.614699+0000","last_unstale":"2023-08-25T03:34:05.349004+0000","last_undegraded":"2023-08-25T03:34:05.349004+0000","last_fullsized":"2023-08-25T03:34:05.349004+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:02:02.415298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611685+0000","last_change":"2023-08-25T03:34:02.597247+0000","last_active":"2023-08-25T03:34:04.611685+0000","last_peered":"2023-08-25T03:34:04.611685+0000","last_clean":"2023-08-25T03:34:04.611685+0000","last_became_active":"2023-08-25T03:34:02.596968+0000","last_became_peered":"2023-08-25T03:34:02.596968+0000","last_unstale":"2023-08-25T03:34:04.611685+0000","last_undegraded":"2023-08-25T03:34:04.611685+0000","last_fullsized":"2023-08-25T03:34:04.611685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:23:04.969357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349570+0000","last_change":"2023-08-25T03:34:03.623532+0000","last_active":"2023-08-25T03:34:05.349570+0000","last_peered":"2023-08-25T03:34:05.349570+0000","last_clean":"2023-08-25T03:34:05.349570+0000","last_became_active":"2023-08-25T03:34:03.622546+0000","last_became_peered":"2023-08-25T03:34:03.622546+0000","last_unstale":"2023-08-25T03:34:05.349570+0000","last_undegraded":"2023-08-25T03:34:05.349570+0000","last_fullsized":"2023-08-25T03:34:05.349570+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:22:25.881167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"23'3","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264179+0000","last_change":"2023-08-25T03:34:02.603880+0000","last_active":"2023-08-25T03:34:05.264179+0000","last_peered":"2023-08-25T03:34:05.264179+0000","last_clean":"2023-08-25T03:34:05.264179+0000","last_became_active":"2023-08-25T03:34:02.603502+0000","last_became_peered":"2023-08-25T03:34:02.603502+0000","last_unstale":"2023-08-25T03:34:05.264179+0000","last_undegraded":"2023-08-25T03:34:05.264179+0000","last_fullsized":"2023-08-25T03:34:05.264179+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:39:33.908387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.350317+0000","last_change":"2023-08-25T03:34:03.614556+0000","last_active":"2023-08-25T03:34:05.350317+0000","last_peered":"2023-08-25T03:34:05.350317+0000","last_clean":"2023-08-25T03:34:05.350317+0000","last_became_active":"2023-08-25T03:34:03.614260+0000","last_became_peered":"2023-08-25T03:34:03.614260+0000","last_unstale":"2023-08-25T03:34:05.350317+0000","last_undegraded":"2023-08-25T03:34:05.350317+0000","last_fullsized":"2023-08-25T03:34:05.350317+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T03:41:42.153100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"23'6","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349211+0000","last_change":"2023-08-25T03:34:02.606875+0000","last_active":"2023-08-25T03:34:05.349211+0000","last_peered":"2023-08-25T03:34:05.349211+0000","last_clean":"2023-08-25T03:34:05.349211+0000","last_became_active":"2023-08-25T03:34:02.606567+0000","last_became_peered":"2023-08-25T03:34:02.606567+0000","last_unstale":"2023-08-25T03:34:05.349211+0000","last_undegraded":"2023-08-25T03:34:05.349211+0000","last_fullsized":"2023-08-25T03:34:05.349211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:27:14.892441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348768+0000","last_change":"2023-08-25T03:34:03.619847+0000","last_active":"2023-08-25T03:34:05.348768+0000","last_peered":"2023-08-25T03:34:05.348768+0000","last_clean":"2023-08-25T03:34:05.348768+0000","last_became_active":"2023-08-25T03:34:03.619046+0000","last_became_peered":"2023-08-25T03:34:03.619046+0000","last_unstale":"2023-08-25T03:34:05.348768+0000","last_undegraded":"2023-08-25T03:34:05.348768+0000","last_fullsized":"2023-08-25T03:34:05.348768+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:22:25.881167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264047+0000","last_change":"2023-08-25T03:34:02.604424+0000","last_active":"2023-08-25T03:34:05.264047+0000","last_peered":"2023-08-25T03:34:05.264047+0000","last_clean":"2023-08-25T03:34:05.264047+0000","last_became_active":"2023-08-25T03:34:02.604175+0000","last_became_peered":"2023-08-25T03:34:02.604175+0000","last_unstale":"2023-08-25T03:34:05.264047+0000","last_undegraded":"2023-08-25T03:34:05.264047+0000","last_fullsized":"2023-08-25T03:34:05.264047+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:45:55.252943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349790+0000","last_change":"2023-08-25T03:34:03.622054+0000","last_active":"2023-08-25T03:34:05.349790+0000","last_peered":"2023-08-25T03:34:05.349790+0000","last_clean":"2023-08-25T03:34:05.349790+0000","last_became_active":"2023-08-25T03:34:03.619122+0000","last_became_peered":"2023-08-25T03:34:03.619122+0000","last_unstale":"2023-08-25T03:34:05.349790+0000","last_undegraded":"2023-08-25T03:34:05.349790+0000","last_fullsized":"2023-08-25T03:34:05.349790+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:09.006588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.168841+0000","last_change":"2023-08-25T03:34:02.606889+0000","last_active":"2023-08-25T03:34:05.168841+0000","last_peered":"2023-08-25T03:34:05.168841+0000","last_clean":"2023-08-25T03:34:05.168841+0000","last_became_active":"2023-08-25T03:34:02.606424+0000","last_became_peered":"2023-08-25T03:34:02.606424+0000","last_unstale":"2023-08-25T03:34:05.168841+0000","last_undegraded":"2023-08-25T03:34:05.168841+0000","last_fullsized":"2023-08-25T03:34:05.168841+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:02:01.407883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611980+0000","last_change":"2023-08-25T03:34:03.613888+0000","last_active":"2023-08-25T03:34:04.611980+0000","last_peered":"2023-08-25T03:34:04.611980+0000","last_clean":"2023-08-25T03:34:04.611980+0000","last_became_active":"2023-08-25T03:34:03.612829+0000","last_became_peered":"2023-08-25T03:34:03.612829+0000","last_unstale":"2023-08-25T03:34:04.611980+0000","last_undegraded":"2023-08-25T03:34:04.611980+0000","last_fullsized":"2023-08-25T03:34:04.611980+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:27:15.899855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.263949+0000","last_change":"2023-08-25T03:34:02.606279+0000","last_active":"2023-08-25T03:34:05.263949+0000","last_peered":"2023-08-25T03:34:05.263949+0000","last_clean":"2023-08-25T03:34:05.263949+0000","last_became_active":"2023-08-25T03:34:02.606005+0000","last_became_peered":"2023-08-25T03:34:02.606005+0000","last_unstale":"2023-08-25T03:34:05.263949+0000","last_undegraded":"2023-08-25T03:34:05.263949+0000","last_fullsized":"2023-08-25T03:34:05.263949+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T03:41:41.145686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169244+0000","last_change":"2023-08-25T03:34:03.627203+0000","last_active":"2023-08-25T03:34:05.169244+0000","last_peered":"2023-08-25T03:34:05.169244+0000","last_clean":"2023-08-25T03:34:05.169244+0000","last_became_active":"2023-08-25T03:34:03.626901+0000","last_became_peered":"2023-08-25T03:34:03.626901+0000","last_unstale":"2023-08-25T03:34:05.169244+0000","last_undegraded":"2023-08-25T03:34:05.169244+0000","last_fullsized":"2023-08-25T03:34:05.169244+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:23:05.976771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"23'8","reported_seq":21,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612301+0000","last_change":"2023-08-25T03:34:02.608430+0000","last_active":"2023-08-25T03:34:04.612301+0000","last_peered":"2023-08-25T03:34:04.612301+0000","last_clean":"2023-08-25T03:34:04.612301+0000","last_became_active":"2023-08-25T03:34:02.608086+0000","last_became_peered":"2023-08-25T03:34:02.608086+0000","last_unstale":"2023-08-25T03:34:04.612301+0000","last_undegraded":"2023-08-25T03:34:04.612301+0000","last_fullsized":"2023-08-25T03:34:04.612301+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:23:48.303578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.350395+0000","last_change":"2023-08-25T03:34:03.624700+0000","last_active":"2023-08-25T03:34:05.350395+0000","last_peered":"2023-08-25T03:34:05.350395+0000","last_clean":"2023-08-25T03:34:05.350395+0000","last_became_active":"2023-08-25T03:34:03.624180+0000","last_became_peered":"2023-08-25T03:34:03.624180+0000","last_unstale":"2023-08-25T03:34:05.350395+0000","last_undegraded":"2023-08-25T03:34:05.350395+0000","last_fullsized":"2023-08-25T03:34:05.350395+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:27:15.899855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264384+0000","last_change":"2023-08-25T03:34:02.604587+0000","last_active":"2023-08-25T03:34:05.264384+0000","last_peered":"2023-08-25T03:34:05.264384+0000","last_clean":"2023-08-25T03:34:05.264384+0000","last_became_active":"2023-08-25T03:34:02.604290+0000","last_became_peered":"2023-08-25T03:34:02.604290+0000","last_unstale":"2023-08-25T03:34:05.264384+0000","last_undegraded":"2023-08-25T03:34:05.264384+0000","last_fullsized":"2023-08-25T03:34:05.264384+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:07:41.306323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611079+0000","last_change":"2023-08-25T03:34:03.613959+0000","last_active":"2023-08-25T03:34:04.611079+0000","last_peered":"2023-08-25T03:34:04.611079+0000","last_clean":"2023-08-25T03:34:04.611079+0000","last_became_active":"2023-08-25T03:34:03.613667+0000","last_became_peered":"2023-08-25T03:34:03.613667+0000","last_unstale":"2023-08-25T03:34:04.611079+0000","last_undegraded":"2023-08-25T03:34:04.611079+0000","last_fullsized":"2023-08-25T03:34:04.611079+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:41:33.001585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264371+0000","last_change":"2023-08-25T03:34:02.604585+0000","last_active":"2023-08-25T03:34:05.264371+0000","last_peered":"2023-08-25T03:34:05.264371+0000","last_clean":"2023-08-25T03:34:05.264371+0000","last_became_active":"2023-08-25T03:34:02.604255+0000","last_became_peered":"2023-08-25T03:34:02.604255+0000","last_unstale":"2023-08-25T03:34:05.264371+0000","last_undegraded":"2023-08-25T03:34:05.264371+0000","last_fullsized":"2023-08-25T03:34:05.264371+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:31:14.838181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348736+0000","last_change":"2023-08-25T03:34:03.624731+0000","last_active":"2023-08-25T03:34:05.348736+0000","last_peered":"2023-08-25T03:34:05.348736+0000","last_clean":"2023-08-25T03:34:05.348736+0000","last_became_active":"2023-08-25T03:34:03.619454+0000","last_became_peered":"2023-08-25T03:34:03.619454+0000","last_unstale":"2023-08-25T03:34:05.348736+0000","last_undegraded":"2023-08-25T03:34:05.348736+0000","last_fullsized":"2023-08-25T03:34:05.348736+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:05:25.658159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264522+0000","last_change":"2023-08-25T03:34:02.605137+0000","last_active":"2023-08-25T03:34:05.264522+0000","last_peered":"2023-08-25T03:34:05.264522+0000","last_clean":"2023-08-25T03:34:05.264522+0000","last_became_active":"2023-08-25T03:34:02.604846+0000","last_became_peered":"2023-08-25T03:34:02.604846+0000","last_unstale":"2023-08-25T03:34:05.264522+0000","last_undegraded":"2023-08-25T03:34:05.264522+0000","last_fullsized":"2023-08-25T03:34:05.264522+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:44:39.385392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612781+0000","last_change":"2023-08-25T03:34:03.614885+0000","last_active":"2023-08-25T03:34:04.612781+0000","last_peered":"2023-08-25T03:34:04.612781+0000","last_clean":"2023-08-25T03:34:04.612781+0000","last_became_active":"2023-08-25T03:34:03.614420+0000","last_became_peered":"2023-08-25T03:34:03.614420+0000","last_unstale":"2023-08-25T03:34:04.612781+0000","last_undegraded":"2023-08-25T03:34:04.612781+0000","last_fullsized":"2023-08-25T03:34:04.612781+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:47:27.124952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611601+0000","last_change":"2023-08-25T03:34:02.598840+0000","last_active":"2023-08-25T03:34:04.611601+0000","last_peered":"2023-08-25T03:34:04.611601+0000","last_clean":"2023-08-25T03:34:04.611601+0000","last_became_active":"2023-08-25T03:34:02.598544+0000","last_became_peered":"2023-08-25T03:34:02.598544+0000","last_unstale":"2023-08-25T03:34:04.611601+0000","last_undegraded":"2023-08-25T03:34:04.611601+0000","last_fullsized":"2023-08-25T03:34:04.611601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:52:25.303751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.168983+0000","last_change":"2023-08-25T03:34:03.616738+0000","last_active":"2023-08-25T03:34:05.168983+0000","last_peered":"2023-08-25T03:34:05.168983+0000","last_clean":"2023-08-25T03:34:05.168983+0000","last_became_active":"2023-08-25T03:34:03.616450+0000","last_became_peered":"2023-08-25T03:34:03.616450+0000","last_unstale":"2023-08-25T03:34:05.168983+0000","last_undegraded":"2023-08-25T03:34:05.168983+0000","last_fullsized":"2023-08-25T03:34:05.168983+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:05:25.658159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349607+0000","last_change":"2023-08-25T03:34:02.605001+0000","last_active":"2023-08-25T03:34:05.349607+0000","last_peered":"2023-08-25T03:34:05.349607+0000","last_clean":"2023-08-25T03:34:05.349607+0000","last_became_active":"2023-08-25T03:34:02.604686+0000","last_became_peered":"2023-08-25T03:34:02.604686+0000","last_unstale":"2023-08-25T03:34:05.349607+0000","last_undegraded":"2023-08-25T03:34:05.349607+0000","last_fullsized":"2023-08-25T03:34:05.349607+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:10:03.545814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349133+0000","last_change":"2023-08-25T03:34:03.626817+0000","last_active":"2023-08-25T03:34:05.349133+0000","last_peered":"2023-08-25T03:34:05.349133+0000","last_clean":"2023-08-25T03:34:05.349133+0000","last_became_active":"2023-08-25T03:34:03.626483+0000","last_became_peered":"2023-08-25T03:34:03.626483+0000","last_unstale":"2023-08-25T03:34:05.349133+0000","last_undegraded":"2023-08-25T03:34:05.349133+0000","last_fullsized":"2023-08-25T03:34:05.349133+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:01:41.454184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612643+0000","last_change":"2023-08-25T03:34:02.601925+0000","last_active":"2023-08-25T03:34:04.612643+0000","last_peered":"2023-08-25T03:34:04.612643+0000","last_clean":"2023-08-25T03:34:04.612643+0000","last_became_active":"2023-08-25T03:34:02.601552+0000","last_became_peered":"2023-08-25T03:34:02.601552+0000","last_unstale":"2023-08-25T03:34:04.612643+0000","last_undegraded":"2023-08-25T03:34:04.612643+0000","last_fullsized":"2023-08-25T03:34:04.612643+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:47:38.956402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349296+0000","last_change":"2023-08-25T03:34:03.627975+0000","last_active":"2023-08-25T03:34:05.349296+0000","last_peered":"2023-08-25T03:34:05.349296+0000","last_clean":"2023-08-25T03:34:05.349296+0000","last_became_active":"2023-08-25T03:34:03.627489+0000","last_became_peered":"2023-08-25T03:34:03.627489+0000","last_unstale":"2023-08-25T03:34:05.349296+0000","last_undegraded":"2023-08-25T03:34:05.349296+0000","last_fullsized":"2023-08-25T03:34:05.349296+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:57:59.924993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348882+0000","last_change":"2023-08-25T03:34:02.603347+0000","last_active":"2023-08-25T03:34:05.348882+0000","last_peered":"2023-08-25T03:34:05.348882+0000","last_clean":"2023-08-25T03:34:05.348882+0000","last_became_active":"2023-08-25T03:34:02.603010+0000","last_became_peered":"2023-08-25T03:34:02.603010+0000","last_unstale":"2023-08-25T03:34:05.348882+0000","last_undegraded":"2023-08-25T03:34:05.348882+0000","last_fullsized":"2023-08-25T03:34:05.348882+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:07.999174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349594+0000","last_change":"2023-08-25T03:34:03.616962+0000","last_active":"2023-08-25T03:34:05.349594+0000","last_peered":"2023-08-25T03:34:05.349594+0000","last_clean":"2023-08-25T03:34:05.349594+0000","last_became_active":"2023-08-25T03:34:03.616656+0000","last_became_peered":"2023-08-25T03:34:03.616656+0000","last_unstale":"2023-08-25T03:34:05.349594+0000","last_undegraded":"2023-08-25T03:34:05.349594+0000","last_fullsized":"2023-08-25T03:34:05.349594+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:39:34.915801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348904+0000","last_change":"2023-08-25T03:34:02.603820+0000","last_active":"2023-08-25T03:34:05.348904+0000","last_peered":"2023-08-25T03:34:05.348904+0000","last_clean":"2023-08-25T03:34:05.348904+0000","last_became_active":"2023-08-25T03:34:02.603493+0000","last_became_peered":"2023-08-25T03:34:02.603493+0000","last_unstale":"2023-08-25T03:34:05.348904+0000","last_undegraded":"2023-08-25T03:34:05.348904+0000","last_fullsized":"2023-08-25T03:34:05.348904+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:47:12.826390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.350183+0000","last_change":"2023-08-25T03:34:03.625076+0000","last_active":"2023-08-25T03:34:05.350183+0000","last_peered":"2023-08-25T03:34:05.350183+0000","last_clean":"2023-08-25T03:34:05.350183+0000","last_became_active":"2023-08-25T03:34:03.624812+0000","last_became_peered":"2023-08-25T03:34:03.624812+0000","last_unstale":"2023-08-25T03:34:05.350183+0000","last_undegraded":"2023-08-25T03:34:05.350183+0000","last_fullsized":"2023-08-25T03:34:05.350183+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:27:15.899855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349701+0000","last_change":"2023-08-25T03:33:57.570267+0000","last_active":"2023-08-25T03:34:05.349701+0000","last_peered":"2023-08-25T03:34:05.349701+0000","last_clean":"2023-08-25T03:34:05.349701+0000","last_became_active":"2023-08-25T03:33:57.569882+0000","last_became_peered":"2023-08-25T03:33:57.569882+0000","last_unstale":"2023-08-25T03:34:05.349701+0000","last_undegraded":"2023-08-25T03:34:05.349701+0000","last_fullsized":"2023-08-25T03:34:05.349701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:09:58.529844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612129+0000","last_change":"2023-08-25T03:34:02.600129+0000","last_active":"2023-08-25T03:34:04.612129+0000","last_peered":"2023-08-25T03:34:04.612129+0000","last_clean":"2023-08-25T03:34:04.612129+0000","last_became_active":"2023-08-25T03:34:02.599762+0000","last_became_peered":"2023-08-25T03:34:02.599762+0000","last_unstale":"2023-08-25T03:34:04.612129+0000","last_undegraded":"2023-08-25T03:34:04.612129+0000","last_fullsized":"2023-08-25T03:34:04.612129+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T03:41:41.145686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349944+0000","last_change":"2023-08-25T03:34:03.616199+0000","last_active":"2023-08-25T03:34:05.349944+0000","last_peered":"2023-08-25T03:34:05.349944+0000","last_clean":"2023-08-25T03:34:05.349944+0000","last_became_active":"2023-08-25T03:34:03.613500+0000","last_became_peered":"2023-08-25T03:34:03.613500+0000","last_unstale":"2023-08-25T03:34:05.349944+0000","last_undegraded":"2023-08-25T03:34:05.349944+0000","last_fullsized":"2023-08-25T03:34:05.349944+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:23:05.976771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611287+0000","last_change":"2023-08-25T03:33:57.569595+0000","last_active":"2023-08-25T03:34:04.611287+0000","last_peered":"2023-08-25T03:34:04.611287+0000","last_clean":"2023-08-25T03:34:04.611287+0000","last_became_active":"2023-08-25T03:33:57.569307+0000","last_became_peered":"2023-08-25T03:33:57.569307+0000","last_unstale":"2023-08-25T03:34:04.611287+0000","last_undegraded":"2023-08-25T03:34:04.611287+0000","last_fullsized":"2023-08-25T03:34:04.611287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:05:19.634775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349072+0000","last_change":"2023-08-25T03:34:02.605404+0000","last_active":"2023-08-25T03:34:05.349072+0000","last_peered":"2023-08-25T03:34:05.349072+0000","last_clean":"2023-08-25T03:34:05.349072+0000","last_became_active":"2023-08-25T03:34:02.604806+0000","last_became_peered":"2023-08-25T03:34:02.604806+0000","last_unstale":"2023-08-25T03:34:05.349072+0000","last_undegraded":"2023-08-25T03:34:05.349072+0000","last_fullsized":"2023-08-25T03:34:05.349072+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:03:05.441386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349865+0000","last_change":"2023-08-25T03:34:03.632131+0000","last_active":"2023-08-25T03:34:05.349865+0000","last_peered":"2023-08-25T03:34:05.349865+0000","last_clean":"2023-08-25T03:34:05.349865+0000","last_became_active":"2023-08-25T03:34:03.631825+0000","last_became_peered":"2023-08-25T03:34:03.631825+0000","last_unstale":"2023-08-25T03:34:05.349865+0000","last_undegraded":"2023-08-25T03:34:05.349865+0000","last_fullsized":"2023-08-25T03:34:05.349865+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:09.006588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611334+0000","last_change":"2023-08-25T03:33:57.886704+0000","last_active":"2023-08-25T03:34:04.611334+0000","last_peered":"2023-08-25T03:34:04.611334+0000","last_clean":"2023-08-25T03:34:04.611334+0000","last_became_active":"2023-08-25T03:33:57.886294+0000","last_became_peered":"2023-08-25T03:33:57.886294+0000","last_unstale":"2023-08-25T03:34:04.611334+0000","last_undegraded":"2023-08-25T03:34:04.611334+0000","last_fullsized":"2023-08-25T03:34:04.611334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:15:31.545146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349162+0000","last_change":"2023-08-25T03:34:02.589214+0000","last_active":"2023-08-25T03:34:05.349162+0000","last_peered":"2023-08-25T03:34:05.349162+0000","last_clean":"2023-08-25T03:34:05.349162+0000","last_became_active":"2023-08-25T03:34:02.588607+0000","last_became_peered":"2023-08-25T03:34:02.588607+0000","last_unstale":"2023-08-25T03:34:05.349162+0000","last_undegraded":"2023-08-25T03:34:05.349162+0000","last_fullsized":"2023-08-25T03:34:05.349162+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:02:01.407883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.350043+0000","last_change":"2023-08-25T03:34:03.613767+0000","last_active":"2023-08-25T03:34:05.350043+0000","last_peered":"2023-08-25T03:34:05.350043+0000","last_clean":"2023-08-25T03:34:05.350043+0000","last_became_active":"2023-08-25T03:34:03.613433+0000","last_became_peered":"2023-08-25T03:34:03.613433+0000","last_unstale":"2023-08-25T03:34:05.350043+0000","last_undegraded":"2023-08-25T03:34:05.350043+0000","last_fullsized":"2023-08-25T03:34:05.350043+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:01:41.454184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611389+0000","last_change":"2023-08-25T03:33:57.570657+0000","last_active":"2023-08-25T03:34:04.611389+0000","last_peered":"2023-08-25T03:34:04.611389+0000","last_clean":"2023-08-25T03:34:04.611389+0000","last_became_active":"2023-08-25T03:33:57.570425+0000","last_became_peered":"2023-08-25T03:33:57.570425+0000","last_unstale":"2023-08-25T03:34:04.611389+0000","last_undegraded":"2023-08-25T03:34:04.611389+0000","last_fullsized":"2023-08-25T03:34:04.611389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:22:19.857784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":175,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.263861+0000","last_change":"2023-08-25T03:33:51.259898+0000","last_active":"2023-08-25T03:34:05.263861+0000","last_peered":"2023-08-25T03:34:05.263861+0000","last_clean":"2023-08-25T03:34:05.263861+0000","last_became_active":"2023-08-25T03:33:51.258780+0000","last_became_peered":"2023-08-25T03:33:51.258780+0000","last_unstale":"2023-08-25T03:34:05.263861+0000","last_undegraded":"2023-08-25T03:34:05.263861+0000","last_fullsized":"2023-08-25T03:34:05.263861+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:49.666858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:49.666858+0000","last_clean_scrub_stamp":"2023-08-25T03:33:49.666858+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:27:02.988993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349202+0000","last_change":"2023-08-25T03:34:02.598738+0000","last_active":"2023-08-25T03:34:05.349202+0000","last_peered":"2023-08-25T03:34:05.349202+0000","last_clean":"2023-08-25T03:34:05.349202+0000","last_became_active":"2023-08-25T03:34:02.598219+0000","last_became_peered":"2023-08-25T03:34:02.598219+0000","last_unstale":"2023-08-25T03:34:05.349202+0000","last_undegraded":"2023-08-25T03:34:05.349202+0000","last_fullsized":"2023-08-25T03:34:05.349202+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:05:24.650744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612566+0000","last_change":"2023-08-25T03:34:03.624616+0000","last_active":"2023-08-25T03:34:04.612566+0000","last_peered":"2023-08-25T03:34:04.612566+0000","last_clean":"2023-08-25T03:34:04.612566+0000","last_became_active":"2023-08-25T03:34:03.624330+0000","last_became_peered":"2023-08-25T03:34:03.624330+0000","last_unstale":"2023-08-25T03:34:04.612566+0000","last_undegraded":"2023-08-25T03:34:04.612566+0000","last_fullsized":"2023-08-25T03:34:04.612566+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:25:43.507515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264117+0000","last_change":"2023-08-25T03:33:57.571821+0000","last_active":"2023-08-25T03:34:05.264117+0000","last_peered":"2023-08-25T03:34:05.264117+0000","last_clean":"2023-08-25T03:34:05.264117+0000","last_became_active":"2023-08-25T03:33:57.571528+0000","last_became_peered":"2023-08-25T03:33:57.571528+0000","last_unstale":"2023-08-25T03:34:05.264117+0000","last_undegraded":"2023-08-25T03:34:05.264117+0000","last_fullsized":"2023-08-25T03:34:05.264117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:33:23.447280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.168795+0000","last_change":"2023-08-25T03:34:02.602275+0000","last_active":"2023-08-25T03:34:05.168795+0000","last_peered":"2023-08-25T03:34:05.168795+0000","last_clean":"2023-08-25T03:34:05.168795+0000","last_became_active":"2023-08-25T03:34:02.601960+0000","last_became_peered":"2023-08-25T03:34:02.601960+0000","last_unstale":"2023-08-25T03:34:05.168795+0000","last_undegraded":"2023-08-25T03:34:05.168795+0000","last_fullsized":"2023-08-25T03:34:05.168795+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:47:12.826390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169447+0000","last_change":"2023-08-25T03:34:03.612794+0000","last_active":"2023-08-25T03:34:05.169447+0000","last_peered":"2023-08-25T03:34:05.169447+0000","last_clean":"2023-08-25T03:34:05.169447+0000","last_became_active":"2023-08-25T03:34:03.612474+0000","last_became_peered":"2023-08-25T03:34:03.612474+0000","last_unstale":"2023-08-25T03:34:05.169447+0000","last_undegraded":"2023-08-25T03:34:05.169447+0000","last_fullsized":"2023-08-25T03:34:05.169447+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T03:41:42.153100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.264364+0000","last_change":"2023-08-25T03:33:57.571730+0000","last_active":"2023-08-25T03:34:05.264364+0000","last_peered":"2023-08-25T03:34:05.264364+0000","last_clean":"2023-08-25T03:34:05.264364+0000","last_became_active":"2023-08-25T03:33:57.571442+0000","last_became_peered":"2023-08-25T03:33:57.571442+0000","last_unstale":"2023-08-25T03:34:05.264364+0000","last_undegraded":"2023-08-25T03:34:05.264364+0000","last_fullsized":"2023-08-25T03:34:05.264364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T03:49:14.356320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349890+0000","last_change":"2023-08-25T03:34:02.596876+0000","last_active":"2023-08-25T03:34:05.349890+0000","last_peered":"2023-08-25T03:34:05.349890+0000","last_clean":"2023-08-25T03:34:05.349890+0000","last_became_active":"2023-08-25T03:34:02.596556+0000","last_became_peered":"2023-08-25T03:34:02.596556+0000","last_unstale":"2023-08-25T03:34:05.349890+0000","last_undegraded":"2023-08-25T03:34:05.349890+0000","last_fullsized":"2023-08-25T03:34:05.349890+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:52:25.303751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169267+0000","last_change":"2023-08-25T03:34:03.616290+0000","last_active":"2023-08-25T03:34:05.169267+0000","last_peered":"2023-08-25T03:34:05.169267+0000","last_clean":"2023-08-25T03:34:05.169267+0000","last_became_active":"2023-08-25T03:34:03.616009+0000","last_became_peered":"2023-08-25T03:34:03.616009+0000","last_unstale":"2023-08-25T03:34:05.169267+0000","last_undegraded":"2023-08-25T03:34:05.169267+0000","last_fullsized":"2023-08-25T03:34:05.169267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:08.250998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.350304+0000","last_change":"2023-08-25T03:34:02.604771+0000","last_active":"2023-08-25T03:34:05.350304+0000","last_peered":"2023-08-25T03:34:05.350304+0000","last_clean":"2023-08-25T03:34:05.350304+0000","last_became_active":"2023-08-25T03:34:02.604509+0000","last_became_peered":"2023-08-25T03:34:02.604509+0000","last_unstale":"2023-08-25T03:34:05.350304+0000","last_undegraded":"2023-08-25T03:34:05.350304+0000","last_fullsized":"2023-08-25T03:34:05.350304+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:01:40.446770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349627+0000","last_change":"2023-08-25T03:33:57.570506+0000","last_active":"2023-08-25T03:34:05.349627+0000","last_peered":"2023-08-25T03:34:05.349627+0000","last_clean":"2023-08-25T03:34:05.349627+0000","last_became_active":"2023-08-25T03:33:57.570171+0000","last_became_peered":"2023-08-25T03:33:57.570171+0000","last_unstale":"2023-08-25T03:34:05.349627+0000","last_undegraded":"2023-08-25T03:34:05.349627+0000","last_fullsized":"2023-08-25T03:34:05.349627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:05:19.634775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348810+0000","last_change":"2023-08-25T03:34:03.622270+0000","last_active":"2023-08-25T03:34:05.348810+0000","last_peered":"2023-08-25T03:34:05.348810+0000","last_clean":"2023-08-25T03:34:05.348810+0000","last_became_active":"2023-08-25T03:34:03.621168+0000","last_became_peered":"2023-08-25T03:34:03.621168+0000","last_unstale":"2023-08-25T03:34:05.348810+0000","last_undegraded":"2023-08-25T03:34:05.348810+0000","last_fullsized":"2023-08-25T03:34:05.348810+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:59:37.641683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612720+0000","last_change":"2023-08-25T03:33:57.570598+0000","last_active":"2023-08-25T03:34:04.612720+0000","last_peered":"2023-08-25T03:34:04.612720+0000","last_clean":"2023-08-25T03:34:04.612720+0000","last_became_active":"2023-08-25T03:33:57.570215+0000","last_became_peered":"2023-08-25T03:33:57.570215+0000","last_unstale":"2023-08-25T03:34:04.612720+0000","last_undegraded":"2023-08-25T03:34:04.612720+0000","last_fullsized":"2023-08-25T03:34:04.612720+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:40:56.425085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169511+0000","last_change":"2023-08-25T03:34:02.605323+0000","last_active":"2023-08-25T03:34:05.169511+0000","last_peered":"2023-08-25T03:34:05.169511+0000","last_clean":"2023-08-25T03:34:05.169511+0000","last_became_active":"2023-08-25T03:34:02.605111+0000","last_became_peered":"2023-08-25T03:34:02.605111+0000","last_unstale":"2023-08-25T03:34:05.169511+0000","last_undegraded":"2023-08-25T03:34:05.169511+0000","last_fullsized":"2023-08-25T03:34:05.169511+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:57:58.917579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348697+0000","last_change":"2023-08-25T03:34:03.624353+0000","last_active":"2023-08-25T03:34:05.348697+0000","last_peered":"2023-08-25T03:34:05.348697+0000","last_clean":"2023-08-25T03:34:05.348697+0000","last_became_active":"2023-08-25T03:34:03.618468+0000","last_became_peered":"2023-08-25T03:34:03.618468+0000","last_unstale":"2023-08-25T03:34:05.348697+0000","last_undegraded":"2023-08-25T03:34:05.348697+0000","last_fullsized":"2023-08-25T03:34:05.348697+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:41:33.001585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169012+0000","last_change":"2023-08-25T03:34:02.603419+0000","last_active":"2023-08-25T03:34:05.169012+0000","last_peered":"2023-08-25T03:34:05.169012+0000","last_clean":"2023-08-25T03:34:05.169012+0000","last_became_active":"2023-08-25T03:34:02.603082+0000","last_became_peered":"2023-08-25T03:34:02.603082+0000","last_unstale":"2023-08-25T03:34:05.169012+0000","last_undegraded":"2023-08-25T03:34:05.169012+0000","last_fullsized":"2023-08-25T03:34:05.169012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:22:24.873753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611520+0000","last_change":"2023-08-25T03:34:03.618585+0000","last_active":"2023-08-25T03:34:04.611520+0000","last_peered":"2023-08-25T03:34:04.611520+0000","last_clean":"2023-08-25T03:34:04.611520+0000","last_became_active":"2023-08-25T03:34:03.618291+0000","last_became_peered":"2023-08-25T03:34:03.618291+0000","last_unstale":"2023-08-25T03:34:04.611520+0000","last_undegraded":"2023-08-25T03:34:04.611520+0000","last_fullsized":"2023-08-25T03:34:04.611520+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:41:02.448469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.611545+0000","last_change":"2023-08-25T03:34:02.601201+0000","last_active":"2023-08-25T03:34:04.611545+0000","last_peered":"2023-08-25T03:34:04.611545+0000","last_clean":"2023-08-25T03:34:04.611545+0000","last_became_active":"2023-08-25T03:34:02.600904+0000","last_became_peered":"2023-08-25T03:34:02.600904+0000","last_unstale":"2023-08-25T03:34:04.611545+0000","last_undegraded":"2023-08-25T03:34:04.611545+0000","last_fullsized":"2023-08-25T03:34:04.611545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:07.999174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349241+0000","last_change":"2023-08-25T03:34:03.626366+0000","last_active":"2023-08-25T03:34:05.349241+0000","last_peered":"2023-08-25T03:34:05.349241+0000","last_clean":"2023-08-25T03:34:05.349241+0000","last_became_active":"2023-08-25T03:34:03.625496+0000","last_became_peered":"2023-08-25T03:34:03.625496+0000","last_unstale":"2023-08-25T03:34:05.349241+0000","last_undegraded":"2023-08-25T03:34:05.349241+0000","last_fullsized":"2023-08-25T03:34:05.349241+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T03:41:42.153100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"23'2","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612260+0000","last_change":"2023-08-25T03:34:02.603423+0000","last_active":"2023-08-25T03:34:04.612260+0000","last_peered":"2023-08-25T03:34:04.612260+0000","last_clean":"2023-08-25T03:34:04.612260+0000","last_became_active":"2023-08-25T03:34:02.603077+0000","last_became_peered":"2023-08-25T03:34:02.603077+0000","last_unstale":"2023-08-25T03:34:04.612260+0000","last_undegraded":"2023-08-25T03:34:04.612260+0000","last_fullsized":"2023-08-25T03:34:04.612260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:57:58.917579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348880+0000","last_change":"2023-08-25T03:34:03.622038+0000","last_active":"2023-08-25T03:34:05.348880+0000","last_peered":"2023-08-25T03:34:05.348880+0000","last_clean":"2023-08-25T03:34:05.348880+0000","last_became_active":"2023-08-25T03:34:03.621747+0000","last_became_peered":"2023-08-25T03:34:03.621747+0000","last_unstale":"2023-08-25T03:34:05.348880+0000","last_undegraded":"2023-08-25T03:34:05.348880+0000","last_fullsized":"2023-08-25T03:34:05.348880+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:10:04.553228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"23'1","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.350130+0000","last_change":"2023-08-25T03:34:02.598272+0000","last_active":"2023-08-25T03:34:05.350130+0000","last_peered":"2023-08-25T03:34:05.350130+0000","last_clean":"2023-08-25T03:34:05.350130+0000","last_became_active":"2023-08-25T03:34:02.597630+0000","last_became_peered":"2023-08-25T03:34:02.597630+0000","last_unstale":"2023-08-25T03:34:05.350130+0000","last_undegraded":"2023-08-25T03:34:05.350130+0000","last_fullsized":"2023-08-25T03:34:05.350130+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:03:05.441386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348872+0000","last_change":"2023-08-25T03:34:03.628379+0000","last_active":"2023-08-25T03:34:05.348872+0000","last_peered":"2023-08-25T03:34:05.348872+0000","last_clean":"2023-08-25T03:34:05.348872+0000","last_became_active":"2023-08-25T03:34:03.627870+0000","last_became_peered":"2023-08-25T03:34:03.627870+0000","last_unstale":"2023-08-25T03:34:05.348872+0000","last_undegraded":"2023-08-25T03:34:05.348872+0000","last_fullsized":"2023-08-25T03:34:05.348872+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:49:11.917056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"23'4","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.169428+0000","last_change":"2023-08-25T03:34:02.597748+0000","last_active":"2023-08-25T03:34:05.169428+0000","last_peered":"2023-08-25T03:34:05.169428+0000","last_clean":"2023-08-25T03:34:05.169428+0000","last_became_active":"2023-08-25T03:34:02.597456+0000","last_became_peered":"2023-08-25T03:34:02.597456+0000","last_unstale":"2023-08-25T03:34:05.169428+0000","last_undegraded":"2023-08-25T03:34:05.169428+0000","last_fullsized":"2023-08-25T03:34:05.169428+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:01.570307+0000","last_clean_scrub_stamp":"2023-08-25T03:34:01.570307+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:01:40.446770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348450+0000","last_change":"2023-08-25T03:34:03.619886+0000","last_active":"2023-08-25T03:34:05.348450+0000","last_peered":"2023-08-25T03:34:05.348450+0000","last_clean":"2023-08-25T03:34:05.348450+0000","last_became_active":"2023-08-25T03:34:03.618445+0000","last_became_peered":"2023-08-25T03:34:03.618445+0000","last_unstale":"2023-08-25T03:34:05.348450+0000","last_undegraded":"2023-08-25T03:34:05.348450+0000","last_fullsized":"2023-08-25T03:34:05.348450+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:59.004627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:04.612396+0000","last_change":"2023-08-25T03:34:03.612902+0000","last_active":"2023-08-25T03:34:04.612396+0000","last_peered":"2023-08-25T03:34:04.612396+0000","last_clean":"2023-08-25T03:34:04.612396+0000","last_became_active":"2023-08-25T03:34:03.612582+0000","last_became_peered":"2023-08-25T03:34:03.612582+0000","last_unstale":"2023-08-25T03:34:04.612396+0000","last_undegraded":"2023-08-25T03:34:04.612396+0000","last_fullsized":"2023-08-25T03:34:04.612396+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:01:15.550047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348872+0000","last_change":"2023-08-25T03:34:03.623559+0000","last_active":"2023-08-25T03:34:05.348872+0000","last_peered":"2023-08-25T03:34:05.348872+0000","last_clean":"2023-08-25T03:34:05.348872+0000","last_became_active":"2023-08-25T03:34:03.622782+0000","last_became_peered":"2023-08-25T03:34:03.622782+0000","last_unstale":"2023-08-25T03:34:05.348872+0000","last_undegraded":"2023-08-25T03:34:05.348872+0000","last_fullsized":"2023-08-25T03:34:05.348872+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:46:59.004627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.348614+0000","last_change":"2023-08-25T03:34:03.622024+0000","last_active":"2023-08-25T03:34:05.348614+0000","last_peered":"2023-08-25T03:34:05.348614+0000","last_clean":"2023-08-25T03:34:05.348614+0000","last_became_active":"2023-08-25T03:34:03.619673+0000","last_became_peered":"2023-08-25T03:34:03.619673+0000","last_unstale":"2023-08-25T03:34:05.348614+0000","last_undegraded":"2023-08-25T03:34:05.348614+0000","last_fullsized":"2023-08-25T03:34:05.348614+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:47:13.833805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349801+0000","last_change":"2023-08-25T03:34:03.621260+0000","last_active":"2023-08-25T03:34:05.349801+0000","last_peered":"2023-08-25T03:34:05.349801+0000","last_clean":"2023-08-25T03:34:05.349801+0000","last_became_active":"2023-08-25T03:34:03.620890+0000","last_became_peered":"2023-08-25T03:34:03.620890+0000","last_unstale":"2023-08-25T03:34:05.349801+0000","last_undegraded":"2023-08-25T03:34:05.349801+0000","last_fullsized":"2023-08-25T03:34:05.349801+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:52:26.311165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":11,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-08-25T03:34:05.349539+0000","last_change":"2023-08-25T03:34:03.615228+0000","last_active":"2023-08-25T03:34:05.349539+0000","last_peered":"2023-08-25T03:34:05.349539+0000","last_clean":"2023-08-25T03:34:05.349539+0000","last_became_active":"2023-08-25T03:34:03.613949+0000","last_became_peered":"2023-08-25T03:34:03.613949+0000","last_unstale":"2023-08-25T03:34:05.349539+0000","last_undegraded":"2023-08-25T03:34:05.349539+0000","last_fullsized":"2023-08-25T03:34:05.349539+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:34:02.577721+0000","last_clean_scrub_stamp":"2023-08-25T03:34:02.577721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:41:33.001585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542154,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":872448,"data_stored":685346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45400000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]}]},{"osd":6,"up_from":14,"seq":60129542154,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":249856,"data_stored":93150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":249856,"data_stored":93150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.6120000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.0950000000000006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4820000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.4400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3900000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.5229999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.5590000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":847872,"data_stored":684100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.19600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65400000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":249856,"data_stored":93150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27268,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344572,"statfs":{"total":96636764160,"available":96608841728,"internally_reserved":0,"allocated":253952,"data_stored":93240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]}]},{"osd":4,"up_from":14,"seq":60129542154,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":94818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66400000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68000000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78600000000000003}]}]},{"osd":5,"up_from":14,"seq":60129542153,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":258048,"data_stored":93802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-25T03:34:17.599 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-08-25T03:34:17.600 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-08-25T03:34:17.600 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-08-25T03:34:17.600 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-08-25T03:34:17.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.755+0000 7f946bef2700 1 Processor -- start 2023-08-25T03:34:17.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.755+0000 7f946bef2700 1 -- start start 2023-08-25T03:34:17.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.756+0000 7f946bef2700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94640ca3f0 0x7f94640cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:17.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.756+0000 7f946bef2700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f946406ae60 0x7f946406b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:17.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.756+0000 7f946bef2700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9464072420 con 0x7f94640ca7d0 2023-08-25T03:34:17.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.756+0000 7f946bef2700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f946406a9d0 con 0x7f946406ae60 2023-08-25T03:34:17.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.756+0000 7f946bef2700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9464072140 con 0x7f94640ca3f0 2023-08-25T03:34:17.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.756+0000 7f946a48f700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f946406ae60 0x7f946406b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:17.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.756+0000 7f946a48f700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f946406ae60 0x7f946406b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59754/0 (socket says 172.21.15.2:59754) 2023-08-25T03:34:17.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.756+0000 7f946a48f700 1 -- 172.21.15.2:0/2155406497 learned_addr learned my addr 172.21.15.2:0/2155406497 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:17.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.756+0000 7f9469c8e700 1 --2- 172.21.15.2:0/2155406497 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94640ca3f0 0x7f94640cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:17.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.756+0000 7f946a48f700 1 -- 172.21.15.2:0/2155406497 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94640ca3f0 msgr2=0x7f94640cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:17.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.756+0000 7f946a48f700 1 --2- 172.21.15.2:0/2155406497 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94640ca3f0 0x7f94640cd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:17.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.756+0000 7f946a48f700 1 -- 172.21.15.2:0/2155406497 >> v1:172.21.15.31:6789/0 conn(0x7f94640ca7d0 legacy=0x7f9464071680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:17.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.756+0000 7f946a48f700 1 -- 172.21.15.2:0/2155406497 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946406a380 con 0x7f946406ae60 2023-08-25T03:34:17.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.756+0000 7f9468c8c700 1 -- 172.21.15.2:0/2155406497 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3111852563 0 0) 0x7f9464072420 con 0x7f94640ca7d0 2023-08-25T03:34:17.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.757+0000 7f946a48f700 1 --2- 172.21.15.2:0/2155406497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f946406ae60 0x7f946406b2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f945c009e50 tx=0x7f945c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da43379f7ce822fa server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:17.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.757+0000 7f9468c8c700 1 -- 172.21.15.2:0/2155406497 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c00e070 con 0x7f946406ae60 2023-08-25T03:34:17.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.757+0000 7f9468c8c700 1 -- 172.21.15.2:0/2155406497 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945c00b910 con 0x7f946406ae60 2023-08-25T03:34:17.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.757+0000 7f9468c8c700 1 -- 172.21.15.2:0/2155406497 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945c00bcc0 con 0x7f946406ae60 2023-08-25T03:34:17.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.757+0000 7f946bef2700 1 -- 172.21.15.2:0/2155406497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f946406ae60 msgr2=0x7f946406b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:17.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.757+0000 7f946bef2700 1 --2- 172.21.15.2:0/2155406497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f946406ae60 0x7f946406b2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f945c009e50 tx=0x7f945c00b330 comp rx=0 tx=0).stop 2023-08-25T03:34:17.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.757+0000 7f946bef2700 1 -- 172.21.15.2:0/2155406497 shutdown_connections 2023-08-25T03:34:17.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.757+0000 7f946bef2700 1 --2- 172.21.15.2:0/2155406497 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f946406ae60 0x7f946406b2f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:17.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.757+0000 7f946bef2700 1 --2- 172.21.15.2:0/2155406497 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94640ca3f0 0x7f94640cd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:17.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.757+0000 7f946bef2700 1 -- 172.21.15.2:0/2155406497 >> 172.21.15.2:0/2155406497 conn(0x7f94640cc070 msgr2=0x7f94640cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:17.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.757+0000 7f946bef2700 1 -- 172.21.15.2:0/2155406497 shutdown_connections 2023-08-25T03:34:17.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.757+0000 7f946bef2700 1 -- 172.21.15.2:0/2155406497 wait complete. 2023-08-25T03:34:17.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.759+0000 7f946bef2700 1 Processor -- start 2023-08-25T03:34:17.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.759+0000 7f946bef2700 1 -- start start 2023-08-25T03:34:17.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.759+0000 7f946bef2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f946406ae60 0x7f94640cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:17.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.759+0000 7f946bef2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94640ca3f0 0x7f94640cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:17.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.759+0000 7f946bef2700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94640ca7d0 0x7f94640b2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:17.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.759+0000 7f946bef2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9464157480 con 0x7f94640ca3f0 2023-08-25T03:34:17.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.759+0000 7f946bef2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9464157760 con 0x7f946406ae60 2023-08-25T03:34:17.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.759+0000 7f946bef2700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9464157a40 con 0x7f94640ca7d0 2023-08-25T03:34:17.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.759+0000 7f946948d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94640ca3f0 0x7f94640cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:17.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.760+0000 7f946948d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94640ca3f0 0x7f94640cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59758/0 (socket says 172.21.15.2:59758) 2023-08-25T03:34:17.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.760+0000 7f946948d700 1 -- 172.21.15.2:0/2872722527 learned_addr learned my addr 172.21.15.2:0/2872722527 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:17.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.760+0000 7f9469c8e700 1 --2- 172.21.15.2:0/2872722527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f946406ae60 0x7f94640cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:17.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.760+0000 7f946a48f700 1 --2- 172.21.15.2:0/2872722527 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94640ca7d0 0x7f94640b2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:17.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.760+0000 7f946948d700 1 -- 172.21.15.2:0/2872722527 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94640ca7d0 msgr2=0x7f94640b2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:17.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.760+0000 7f946948d700 1 --2- 172.21.15.2:0/2872722527 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94640ca7d0 0x7f94640b2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:17.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.760+0000 7f946948d700 1 -- 172.21.15.2:0/2872722527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f946406ae60 msgr2=0x7f94640cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:17.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.760+0000 7f946948d700 1 --2- 172.21.15.2:0/2872722527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f946406ae60 0x7f94640cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:17.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.760+0000 7f946948d700 1 -- 172.21.15.2:0/2872722527 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945c009b30 con 0x7f94640ca3f0 2023-08-25T03:34:17.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.760+0000 7f946a48f700 1 --2- 172.21.15.2:0/2872722527 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94640ca7d0 0x7f94640b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:17.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.760+0000 7f946948d700 1 --2- 172.21.15.2:0/2872722527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94640ca3f0 0x7f94640cddf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f945400c550 tx=0x7f945400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:17.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.760+0000 7f945a7fc700 1 -- 172.21.15.2:0/2872722527 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94540054e0 con 0x7f94640ca3f0 2023-08-25T03:34:17.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.760+0000 7f945a7fc700 1 -- 172.21.15.2:0/2872722527 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9454003680 con 0x7f94640ca3f0 2023-08-25T03:34:17.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.760+0000 7f946bef2700 1 -- 172.21.15.2:0/2872722527 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94640c7400 con 0x7f94640ca3f0 2023-08-25T03:34:17.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.760+0000 7f945a7fc700 1 -- 172.21.15.2:0/2872722527 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9454003a30 con 0x7f94640ca3f0 2023-08-25T03:34:17.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.760+0000 7f946bef2700 1 -- 172.21.15.2:0/2872722527 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94640b3a60 con 0x7f94640ca3f0 2023-08-25T03:34:17.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.761+0000 7f946bef2700 1 -- 172.21.15.2:0/2872722527 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94640c7260 con 0x7f94640ca3f0 2023-08-25T03:34:17.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.762+0000 7f945a7fc700 1 -- 172.21.15.2:0/2872722527 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9454003b90 con 0x7f94640ca3f0 2023-08-25T03:34:17.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.762+0000 7f945a7fc700 1 --2- 172.21.15.2:0/2872722527 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f944806a0e0 0x7f944806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:17.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.762+0000 7f9469c8e700 1 --2- 172.21.15.2:0/2872722527 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f944806a0e0 0x7f944806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:17.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.762+0000 7f945a7fc700 1 -- 172.21.15.2:0/2872722527 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9454087970 con 0x7f94640ca3f0 2023-08-25T03:34:17.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.763+0000 7f9469c8e700 1 --2- 172.21.15.2:0/2872722527 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f944806a0e0 0x7f944806c5a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9460009b40 tx=0x7f9460009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:17.778 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:17.767+0000 7f945a7fc700 1 -- 172.21.15.2:0/2872722527 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f94640c7260 con 0x7f94640ca3f0 2023-08-25T03:34:18.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.013+0000 7f946bef2700 1 -- 172.21.15.2:0/2872722527 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f94640c7260 con 0x7f94640ca3f0 2023-08-25T03:34:18.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.013+0000 7f945a7fc700 1 -- 172.21.15.2:0/2872722527 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f94640c7260 con 0x7f94640ca3f0 2023-08-25T03:34:18.014 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:18.015 INFO:teuthology.orchestra.run.smithi002.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-08-25T03:34:18.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.015+0000 7f946bef2700 1 -- 172.21.15.2:0/2872722527 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f944806a0e0 msgr2=0x7f944806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:18.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.015+0000 7f946bef2700 1 --2- 172.21.15.2:0/2872722527 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f944806a0e0 0x7f944806c5a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9460009b40 tx=0x7f9460009210 comp rx=0 tx=0).stop 2023-08-25T03:34:18.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.015+0000 7f946bef2700 1 -- 172.21.15.2:0/2872722527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94640ca3f0 msgr2=0x7f94640cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:18.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.015+0000 7f946bef2700 1 --2- 172.21.15.2:0/2872722527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94640ca3f0 0x7f94640cddf0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f945400c550 tx=0x7f945400c860 comp rx=0 tx=0).stop 2023-08-25T03:34:18.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.016+0000 7f946bef2700 1 -- 172.21.15.2:0/2872722527 shutdown_connections 2023-08-25T03:34:18.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.016+0000 7f946bef2700 1 --2- 172.21.15.2:0/2872722527 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94640ca7d0 0x7f94640b2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.016+0000 7f946bef2700 1 --2- 172.21.15.2:0/2872722527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f946406ae60 0x7f94640cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.016+0000 7f946bef2700 1 --2- 172.21.15.2:0/2872722527 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f944806a0e0 0x7f944806c5a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.016+0000 7f946bef2700 1 --2- 172.21.15.2:0/2872722527 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94640ca3f0 0x7f94640cddf0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.016+0000 7f946bef2700 1 -- 172.21.15.2:0/2872722527 >> 172.21.15.2:0/2872722527 conn(0x7f94640cc070 msgr2=0x7f94640cc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:18.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.016+0000 7f946bef2700 1 -- 172.21.15.2:0/2872722527 shutdown_connections 2023-08-25T03:34:18.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.016+0000 7f946bef2700 1 -- 172.21.15.2:0/2872722527 wait complete. 2023-08-25T03:34:18.028 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-08-25T03:34:18.029 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:34:18.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.154+0000 7fe36dd14700 1 Processor -- start 2023-08-25T03:34:18.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.154+0000 7fe36dd14700 1 -- start start 2023-08-25T03:34:18.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.155+0000 7fe36dd14700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe36815a050 0x7fe36815db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:18.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.155+0000 7fe36dd14700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe3680b3780 0x7fe36806b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:18.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.155+0000 7fe36dd14700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe36806bb90 con 0x7fe36806b730 2023-08-25T03:34:18.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.155+0000 7fe36dd14700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe36815a430 con 0x7fe3680b3780 2023-08-25T03:34:18.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.155+0000 7fe36dd14700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36815e450 con 0x7fe36815a050 2023-08-25T03:34:18.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.155+0000 7fe366ffd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe3680b3780 0x7fe36806b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:18.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.155+0000 7fe367fff700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fe36806b730 0x7fe3680cdd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46434/0 (socket says 172.21.15.2:46434) 2023-08-25T03:34:18.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.155+0000 7fe366ffd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe3680b3780 0x7fe36806b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59764/0 (socket says 172.21.15.2:59764) 2023-08-25T03:34:18.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.155+0000 7fe367fff700 1 -- 172.21.15.2:0/192078134 learned_addr learned my addr 172.21.15.2:0/192078134 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:18.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.155+0000 7fe3677fe700 1 --2- 172.21.15.2:0/192078134 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe36815a050 0x7fe36815db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:18.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.155+0000 7fe366ffd700 1 -- 172.21.15.2:0/192078134 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe36815a050 msgr2=0x7fe36815db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:18.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.155+0000 7fe366ffd700 1 --2- 172.21.15.2:0/192078134 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe36815a050 0x7fe36815db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.155+0000 7fe366ffd700 1 -- 172.21.15.2:0/192078134 >> v1:172.21.15.2:6789/0 conn(0x7fe36806b730 legacy=0x7fe3680cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:18.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.155+0000 7fe366ffd700 1 -- 172.21.15.2:0/192078134 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36815e730 con 0x7fe3680b3780 2023-08-25T03:34:18.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.155+0000 7fe3667fc700 1 -- 172.21.15.2:0/192078134 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458807850 0 0) 0x7fe36806bb90 con 0x7fe36806b730 2023-08-25T03:34:18.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.155+0000 7fe3677fe700 1 --2- 172.21.15.2:0/192078134 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe36815a050 0x7fe36815db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:18.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.156+0000 7fe366ffd700 1 --2- 172.21.15.2:0/192078134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3680b3780 0x7fe36806b1f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009840 tx=0x7fe35c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b3d0943fb63cd6e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:18.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.156+0000 7fe3667fc700 1 -- 172.21.15.2:0/192078134 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c011830 con 0x7fe3680b3780 2023-08-25T03:34:18.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.156+0000 7fe3667fc700 1 -- 172.21.15.2:0/192078134 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35c011990 con 0x7fe3680b3780 2023-08-25T03:34:18.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.156+0000 7fe3667fc700 1 -- 172.21.15.2:0/192078134 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c011d20 con 0x7fe3680b3780 2023-08-25T03:34:18.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.156+0000 7fe36dd14700 1 -- 172.21.15.2:0/192078134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3680b3780 msgr2=0x7fe36806b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:18.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.156+0000 7fe36dd14700 1 --2- 172.21.15.2:0/192078134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3680b3780 0x7fe36806b1f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe35c009840 tx=0x7fe35c009b50 comp rx=0 tx=0).stop 2023-08-25T03:34:18.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.156+0000 7fe36dd14700 1 -- 172.21.15.2:0/192078134 shutdown_connections 2023-08-25T03:34:18.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.156+0000 7fe36dd14700 1 --2- 172.21.15.2:0/192078134 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe36815a050 0x7fe36815db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.156+0000 7fe36dd14700 1 --2- 172.21.15.2:0/192078134 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3680b3780 0x7fe36806b1f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.156+0000 7fe36dd14700 1 -- 172.21.15.2:0/192078134 >> 172.21.15.2:0/192078134 conn(0x7fe3680cfd10 msgr2=0x7fe36806ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:18.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.156+0000 7fe36dd14700 1 -- 172.21.15.2:0/192078134 shutdown_connections 2023-08-25T03:34:18.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.156+0000 7fe36dd14700 1 -- 172.21.15.2:0/192078134 wait complete. 2023-08-25T03:34:18.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.158+0000 7fe36dd14700 1 Processor -- start 2023-08-25T03:34:18.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.158+0000 7fe36dd14700 1 -- start start 2023-08-25T03:34:18.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.158+0000 7fe36dd14700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe36806b730 0x7fe3681530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:18.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.158+0000 7fe36dd14700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3680b3780 0x7fe3681678d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:18.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.158+0000 7fe36dd14700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe36815a050 0x7fe368163a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:18.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.158+0000 7fe36dd14700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe368166900 con 0x7fe3680b3780 2023-08-25T03:34:18.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.158+0000 7fe36dd14700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe368167440 con 0x7fe36806b730 2023-08-25T03:34:18.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.158+0000 7fe36dd14700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe368166b10 con 0x7fe36815a050 2023-08-25T03:34:18.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.159+0000 7fe366ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3680b3780 0x7fe3681678d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:18.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.159+0000 7fe366ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3680b3780 0x7fe3681678d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59772/0 (socket says 172.21.15.2:59772) 2023-08-25T03:34:18.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.159+0000 7fe366ffd700 1 -- 172.21.15.2:0/34428940 learned_addr learned my addr 172.21.15.2:0/34428940 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:18.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.159+0000 7fe3677fe700 1 --2- 172.21.15.2:0/34428940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe36806b730 0x7fe3681530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:18.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.159+0000 7fe367fff700 1 --2- 172.21.15.2:0/34428940 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe36815a050 0x7fe368163a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:18.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.159+0000 7fe366ffd700 1 -- 172.21.15.2:0/34428940 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe36815a050 msgr2=0x7fe368163a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:18.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.159+0000 7fe366ffd700 1 --2- 172.21.15.2:0/34428940 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe36815a050 0x7fe368163a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.159+0000 7fe366ffd700 1 -- 172.21.15.2:0/34428940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe36806b730 msgr2=0x7fe3681530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:18.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.159+0000 7fe366ffd700 1 --2- 172.21.15.2:0/34428940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe36806b730 0x7fe3681530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.159+0000 7fe366ffd700 1 -- 172.21.15.2:0/34428940 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c0094a0 con 0x7fe3680b3780 2023-08-25T03:34:18.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.159+0000 7fe366ffd700 1 --2- 172.21.15.2:0/34428940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3680b3780 0x7fe3681678d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe35c0090b0 tx=0x7fe35c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:18.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.159+0000 7fe34ffff700 1 -- 172.21.15.2:0/34428940 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c004510 con 0x7fe3680b3780 2023-08-25T03:34:18.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.159+0000 7fe34ffff700 1 -- 172.21.15.2:0/34428940 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35c01f070 con 0x7fe3680b3780 2023-08-25T03:34:18.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.159+0000 7fe36dd14700 1 -- 172.21.15.2:0/34428940 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe368166df0 con 0x7fe3680b3780 2023-08-25T03:34:18.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.159+0000 7fe34ffff700 1 -- 172.21.15.2:0/34428940 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c0038e0 con 0x7fe3680b3780 2023-08-25T03:34:18.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.159+0000 7fe36dd14700 1 -- 172.21.15.2:0/34428940 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe368167e10 con 0x7fe3680b3780 2023-08-25T03:34:18.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.160+0000 7fe36dd14700 1 -- 172.21.15.2:0/34428940 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3681675b0 con 0x7fe3680b3780 2023-08-25T03:34:18.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.161+0000 7fe34ffff700 1 -- 172.21.15.2:0/34428940 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe35c003a40 con 0x7fe3680b3780 2023-08-25T03:34:18.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.161+0000 7fe34ffff700 1 --2- 172.21.15.2:0/34428940 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe35406a0e0 0x7fe35406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:18.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.161+0000 7fe3677fe700 1 --2- 172.21.15.2:0/34428940 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe35406a0e0 0x7fe35406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:18.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.161+0000 7fe34ffff700 1 -- 172.21.15.2:0/34428940 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe35c021070 con 0x7fe3680b3780 2023-08-25T03:34:18.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.161+0000 7fe3677fe700 1 --2- 172.21.15.2:0/34428940 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe35406a0e0 0x7fe35406c5a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe368071430 tx=0x7fe358008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:18.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.165+0000 7fe34ffff700 1 -- 172.21.15.2:0/34428940 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe35c055490 con 0x7fe3680b3780 2023-08-25T03:34:18.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.366+0000 7fe36dd14700 1 -- 172.21.15.2:0/34428940 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3680746d0 con 0x7fe3680b3780 2023-08-25T03:34:18.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.368+0000 7fe34ffff700 1 -- 172.21.15.2:0/34428940 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18135 (secure 0 0 0) 0x7fe35c015090 con 0x7fe3680b3780 2023-08-25T03:34:18.369 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:18.369 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":24,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:34:04.604059+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:34:00.869118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T03:34:01.987016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:34:18.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.370+0000 7fe36dd14700 1 -- 172.21.15.2:0/34428940 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe35406a0e0 msgr2=0x7fe35406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:18.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.370+0000 7fe36dd14700 1 --2- 172.21.15.2:0/34428940 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe35406a0e0 0x7fe35406c5a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe368071430 tx=0x7fe358008040 comp rx=0 tx=0).stop 2023-08-25T03:34:18.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.370+0000 7fe36dd14700 1 -- 172.21.15.2:0/34428940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3680b3780 msgr2=0x7fe3681678d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:18.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.370+0000 7fe36dd14700 1 --2- 172.21.15.2:0/34428940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3680b3780 0x7fe3681678d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe35c0090b0 tx=0x7fe35c004750 comp rx=0 tx=0).stop 2023-08-25T03:34:18.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.370+0000 7fe36dd14700 1 -- 172.21.15.2:0/34428940 shutdown_connections 2023-08-25T03:34:18.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.370+0000 7fe36dd14700 1 --2- 172.21.15.2:0/34428940 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe36815a050 0x7fe368163a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.370+0000 7fe36dd14700 1 --2- 172.21.15.2:0/34428940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe36806b730 0x7fe3681530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.370+0000 7fe36dd14700 1 --2- 172.21.15.2:0/34428940 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe35406a0e0 0x7fe35406c5a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.370+0000 7fe36dd14700 1 --2- 172.21.15.2:0/34428940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3680b3780 0x7fe3681678d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.370+0000 7fe36dd14700 1 -- 172.21.15.2:0/34428940 >> 172.21.15.2:0/34428940 conn(0x7fe3680cfd10 msgr2=0x7fe368070690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:18.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.370+0000 7fe36dd14700 1 -- 172.21.15.2:0/34428940 shutdown_connections 2023-08-25T03:34:18.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.370+0000 7fe36dd14700 1 -- 172.21.15.2:0/34428940 wait complete. 2023-08-25T03:34:18.383 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T03:33:49.560643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T03:33:56.265184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T03:34:00.869118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T03:34:01.987016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T03:34:18.384 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T03:34:18.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.512+0000 7fe4b4f60700 1 Processor -- start 2023-08-25T03:34:18.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.512+0000 7fe4b4f60700 1 -- start start 2023-08-25T03:34:18.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.513+0000 7fe4b4f60700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe4b0066050 0x7fe4b0072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:18.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.513+0000 7fe4b4f60700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe4b006aa40 0x7fe4b015f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:18.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.513+0000 7fe4b4f60700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4b00b1e30 con 0x7fe4b006a660 2023-08-25T03:34:18.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.513+0000 7fe4b4f60700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4b00b12b0 con 0x7fe4b0066050 2023-08-25T03:34:18.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.513+0000 7fe4b4f60700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b00b1590 con 0x7fe4b006aa40 2023-08-25T03:34:18.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.513+0000 7fe4ae59c700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fe4b006a660 0x7fe4b015f200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46444/0 (socket says 172.21.15.2:46444) 2023-08-25T03:34:18.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.513+0000 7fe4ae59c700 1 -- 172.21.15.2:0/2801559721 learned_addr learned my addr 172.21.15.2:0/2801559721 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:18.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.513+0000 7fe4add9b700 1 --2- 172.21.15.2:0/2801559721 >> v2:172.21.15.31:3300/0 conn(0x7fe4b0066050 0x7fe4b0072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:18.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.513+0000 7fe4aed9d700 1 --2- 172.21.15.2:0/2801559721 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe4b006aa40 0x7fe4b015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:18.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.513+0000 7fe4ad59a700 1 -- 172.21.15.2:0/2801559721 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1832252411 0 0) 0x7fe4b00b1e30 con 0x7fe4b006a660 2023-08-25T03:34:18.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.513+0000 7fe4ad59a700 1 -- 172.21.15.2:0/2801559721 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4980035b0 con 0x7fe4b006a660 2023-08-25T03:34:18.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.514+0000 7fe4ad59a700 1 -- 172.21.15.2:0/2801559721 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2409289819 0 0) 0x7fe4980035b0 con 0x7fe4b006a660 2023-08-25T03:34:18.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.514+0000 7fe4ad59a700 1 -- 172.21.15.2:0/2801559721 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe4b006aa40 msgr2=0x7fe4b015f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:18.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.514+0000 7fe4ad59a700 1 --2- 172.21.15.2:0/2801559721 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe4b006aa40 0x7fe4b015f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.514+0000 7fe4ad59a700 1 -- 172.21.15.2:0/2801559721 >> v2:172.21.15.31:3300/0 conn(0x7fe4b0066050 msgr2=0x7fe4b0072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:18.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.514+0000 7fe4ad59a700 1 --2- 172.21.15.2:0/2801559721 >> v2:172.21.15.31:3300/0 conn(0x7fe4b0066050 0x7fe4b0072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.514+0000 7fe4ad59a700 1 -- 172.21.15.2:0/2801559721 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b00b1870 con 0x7fe4b006a660 2023-08-25T03:34:18.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.514+0000 7fe4ad59a700 1 -- 172.21.15.2:0/2801559721 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7fe4a40017f0 con 0x7fe4b006a660 2023-08-25T03:34:18.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.514+0000 7fe4ad59a700 1 -- 172.21.15.2:0/2801559721 >> v1:172.21.15.2:6789/0 conn(0x7fe4b006a660 legacy=0x7fe4b015f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:18.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.514+0000 7fe4ad59a700 1 --2- 172.21.15.2:0/2801559721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe498003a00 0x7fe498003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:18.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.514+0000 7fe4ad59a700 1 --2- 172.21.15.2:0/2801559721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe498004850 0x7fe498004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:18.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.514+0000 7fe4ad59a700 1 --2- 172.21.15.2:0/2801559721 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe4b006aa40 0x7fe498005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:18.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.514+0000 7fe4ad59a700 1 -- 172.21.15.2:0/2801559721 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b00b12b0 con 0x7fe498003a00 2023-08-25T03:34:18.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.514+0000 7fe4ad59a700 1 -- 172.21.15.2:0/2801559721 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b00b1590 con 0x7fe498004850 2023-08-25T03:34:18.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.514+0000 7fe4ad59a700 1 -- 172.21.15.2:0/2801559721 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49800ba40 con 0x7fe4b006aa40 2023-08-25T03:34:18.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.514+0000 7fe4ae59c700 1 --2- 172.21.15.2:0/2801559721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe498003a00 0x7fe498003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:18.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.514+0000 7fe4add9b700 1 --2- 172.21.15.2:0/2801559721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe498004850 0x7fe498004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:18.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.514+0000 7fe4aed9d700 1 --2- 172.21.15.2:0/2801559721 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe4b006aa40 0x7fe498005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:18.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.514+0000 7fe4ae59c700 1 -- 172.21.15.2:0/2801559721 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe4b006aa40 msgr2=0x7fe498005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:18.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.514+0000 7fe4ae59c700 1 --2- 172.21.15.2:0/2801559721 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe4b006aa40 0x7fe498005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.514+0000 7fe4ae59c700 1 -- 172.21.15.2:0/2801559721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe498004850 msgr2=0x7fe498004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:18.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.514+0000 7fe4ae59c700 1 --2- 172.21.15.2:0/2801559721 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe498004850 0x7fe498004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.514+0000 7fe4ae59c700 1 -- 172.21.15.2:0/2801559721 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4980035b0 con 0x7fe498003a00 2023-08-25T03:34:18.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.514+0000 7fe4ae59c700 1 --2- 172.21.15.2:0/2801559721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe498003a00 0x7fe498003e20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe4a400c320 tx=0x7fe498005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8f3111ef8092e93 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:18.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.514+0000 7fe4ad59a700 1 -- 172.21.15.2:0/2801559721 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a400ce00 con 0x7fe498003a00 2023-08-25T03:34:18.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.514+0000 7fe4ad59a700 1 -- 172.21.15.2:0/2801559721 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a4007bf0 con 0x7fe498003a00 2023-08-25T03:34:18.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.514+0000 7fe4ad59a700 1 -- 172.21.15.2:0/2801559721 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4007310 con 0x7fe498003a00 2023-08-25T03:34:18.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.515+0000 7fe4b4f60700 1 -- 172.21.15.2:0/2801559721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe498003a00 msgr2=0x7fe498003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:18.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.515+0000 7fe4b4f60700 1 --2- 172.21.15.2:0/2801559721 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe498003a00 0x7fe498003e20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fe4a400c320 tx=0x7fe498005880 comp rx=0 tx=0).stop 2023-08-25T03:34:18.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.515+0000 7fe4aed9d700 1 -- 172.21.15.2:0/2801559721 reap_dead start 2023-08-25T03:34:18.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.515+0000 7fe4b4f60700 1 -- 172.21.15.2:0/2801559721 shutdown_connections 2023-08-25T03:34:18.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.515+0000 7fe4b4f60700 1 -- 172.21.15.2:0/2801559721 >> 172.21.15.2:0/2801559721 conn(0x7fe4b00cf500 msgr2=0x7fe4b00cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:18.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.515+0000 7fe4b4f60700 1 -- 172.21.15.2:0/2801559721 shutdown_connections 2023-08-25T03:34:18.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.515+0000 7fe4b4f60700 1 -- 172.21.15.2:0/2801559721 wait complete. 2023-08-25T03:34:18.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.516+0000 7fe4b4f60700 1 Processor -- start 2023-08-25T03:34:18.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.516+0000 7fe4b4f60700 1 -- start start 2023-08-25T03:34:18.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.517+0000 7fe4b4f60700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe498003a00 0x7fe4b00ca5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:18.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.517+0000 7fe4b4f60700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe4b00ce9e0 0x7fe4b00c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:18.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.517+0000 7fe4b4f60700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4b01f61d0 0x7fe4b01552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:18.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.517+0000 7fe4b4f60700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b00c3d00 con 0x7fe4b01f61d0 2023-08-25T03:34:18.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.517+0000 7fe4b4f60700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b00cab00 con 0x7fe498003a00 2023-08-25T03:34:18.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.517+0000 7fe4b4f60700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b00cef50 con 0x7fe4b00ce9e0 2023-08-25T03:34:18.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.517+0000 7fe4aed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4b01f61d0 0x7fe4b01552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:18.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.517+0000 7fe4aed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4b01f61d0 0x7fe4b01552a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59792/0 (socket says 172.21.15.2:59792) 2023-08-25T03:34:18.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.517+0000 7fe4aed9d700 1 -- 172.21.15.2:0/2170824659 learned_addr learned my addr 172.21.15.2:0/2170824659 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:18.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.517+0000 7fe4ae59c700 1 --2- 172.21.15.2:0/2170824659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe498003a00 0x7fe4b00ca5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:18.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.517+0000 7fe4add9b700 1 --2- 172.21.15.2:0/2170824659 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe4b00ce9e0 0x7fe4b00c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:18.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.518+0000 7fe4aed9d700 1 -- 172.21.15.2:0/2170824659 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe4b00ce9e0 msgr2=0x7fe4b00c6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:18.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.518+0000 7fe4aed9d700 1 --2- 172.21.15.2:0/2170824659 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe4b00ce9e0 0x7fe4b00c6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.518+0000 7fe4aed9d700 1 -- 172.21.15.2:0/2170824659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe498003a00 msgr2=0x7fe4b00ca5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:18.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.518+0000 7fe4aed9d700 1 --2- 172.21.15.2:0/2170824659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe498003a00 0x7fe4b00ca5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.518+0000 7fe4aed9d700 1 -- 172.21.15.2:0/2170824659 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b00c3f10 con 0x7fe4b01f61d0 2023-08-25T03:34:18.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.518+0000 7fe4aed9d700 1 --2- 172.21.15.2:0/2170824659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4b01f61d0 0x7fe4b01552a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe4b006e400 tx=0x7fe4a8011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:18.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.518+0000 7fe49effd700 1 -- 172.21.15.2:0/2170824659 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a800e040 con 0x7fe4b01f61d0 2023-08-25T03:34:18.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.518+0000 7fe49effd700 1 -- 172.21.15.2:0/2170824659 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a800f040 con 0x7fe4b01f61d0 2023-08-25T03:34:18.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.518+0000 7fe4b4f60700 1 -- 172.21.15.2:0/2170824659 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4b00c41f0 con 0x7fe4b01f61d0 2023-08-25T03:34:18.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.518+0000 7fe49effd700 1 -- 172.21.15.2:0/2170824659 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a800fe30 con 0x7fe4b01f61d0 2023-08-25T03:34:18.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.518+0000 7fe4b4f60700 1 -- 172.21.15.2:0/2170824659 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4b00c7500 con 0x7fe4b01f61d0 2023-08-25T03:34:18.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.519+0000 7fe4b4f60700 1 -- 172.21.15.2:0/2170824659 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4b00746d0 con 0x7fe4b01f61d0 2023-08-25T03:34:18.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.519+0000 7fe49effd700 1 -- 172.21.15.2:0/2170824659 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe4a8022460 con 0x7fe4b01f61d0 2023-08-25T03:34:18.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.520+0000 7fe49effd700 1 --2- 172.21.15.2:0/2170824659 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe49006a0e0 0x7fe49006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:18.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.520+0000 7fe49effd700 1 -- 172.21.15.2:0/2170824659 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe4a8088a70 con 0x7fe4b01f61d0 2023-08-25T03:34:18.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.520+0000 7fe4ae59c700 1 --2- 172.21.15.2:0/2170824659 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe49006a0e0 0x7fe49006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:18.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.520+0000 7fe4ae59c700 1 --2- 172.21.15.2:0/2170824659 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe49006a0e0 0x7fe49006c5a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4000ef0 tx=0x7fe4a400a000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:18.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.524+0000 7fe49effd700 1 -- 172.21.15.2:0/2170824659 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe4a8054160 con 0x7fe4b01f61d0 2023-08-25T03:34:18.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.726+0000 7fe4b4f60700 1 -- 172.21.15.2:0/2170824659 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe4b015fd50 con 0x7fe4b01f61d0 2023-08-25T03:34:18.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.727+0000 7fe49effd700 1 -- 172.21.15.2:0/2170824659 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe4a8057780 con 0x7fe4b01f61d0 2023-08-25T03:34:18.728 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-08-25T03:34:18.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.729+0000 7fe4b4f60700 1 -- 172.21.15.2:0/2170824659 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe49006a0e0 msgr2=0x7fe49006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:18.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.729+0000 7fe4b4f60700 1 --2- 172.21.15.2:0/2170824659 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe49006a0e0 0x7fe49006c5a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4000ef0 tx=0x7fe4a400a000 comp rx=0 tx=0).stop 2023-08-25T03:34:18.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.729+0000 7fe4b4f60700 1 -- 172.21.15.2:0/2170824659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4b01f61d0 msgr2=0x7fe4b01552a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:18.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.729+0000 7fe4b4f60700 1 --2- 172.21.15.2:0/2170824659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4b01f61d0 0x7fe4b01552a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe4b006e400 tx=0x7fe4a8011f60 comp rx=0 tx=0).stop 2023-08-25T03:34:18.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.729+0000 7fe4b4f60700 1 -- 172.21.15.2:0/2170824659 shutdown_connections 2023-08-25T03:34:18.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.729+0000 7fe4b4f60700 1 --2- 172.21.15.2:0/2170824659 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe49006a0e0 0x7fe49006c5a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.729+0000 7fe4b4f60700 1 --2- 172.21.15.2:0/2170824659 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe4b01f61d0 0x7fe4b01552a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.729+0000 7fe4b4f60700 1 --2- 172.21.15.2:0/2170824659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe498003a00 0x7fe4b00ca5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.729+0000 7fe4b4f60700 1 --2- 172.21.15.2:0/2170824659 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe4b00ce9e0 0x7fe4b00c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.729+0000 7fe4b4f60700 1 -- 172.21.15.2:0/2170824659 >> 172.21.15.2:0/2170824659 conn(0x7fe4b00cf500 msgr2=0x7fe4b015d640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:18.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.729+0000 7fe4b4f60700 1 -- 172.21.15.2:0/2170824659 shutdown_connections 2023-08-25T03:34:18.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.729+0000 7fe4b4f60700 1 -- 172.21.15.2:0/2170824659 wait complete. 2023-08-25T03:34:18.741 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T03:34:18.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.868+0000 7f60596dc700 1 Processor -- start 2023-08-25T03:34:18.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.868+0000 7f60596dc700 1 -- start start 2023-08-25T03:34:18.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.869+0000 7f60596dc700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6054066050 0x7f6054072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:18.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.869+0000 7f60596dc700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60540b1e30 con 0x7f605406aa40 2023-08-25T03:34:18.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.869+0000 7f60596dc700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60540b12b0 con 0x7f605406a660 2023-08-25T03:34:18.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.869+0000 7f60596dc700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60540b1590 con 0x7f6054066050 2023-08-25T03:34:18.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.869+0000 7f60537fe700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f605406aa40 0x7f605415aaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46466/0 (socket says 172.21.15.2:46466) 2023-08-25T03:34:18.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.869+0000 7f60537fe700 1 -- 172.21.15.2:0/4032120803 learned_addr learned my addr 172.21.15.2:0/4032120803 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:18.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.870+0000 7f60527fc700 1 --2- 172.21.15.2:0/4032120803 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6054066050 0x7f6054072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:18.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.870+0000 7f6051ffb700 1 -- 172.21.15.2:0/4032120803 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2825076608 0 0) 0x7f60540b1e30 con 0x7f605406aa40 2023-08-25T03:34:18.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.870+0000 7f6051ffb700 1 -- 172.21.15.2:0/4032120803 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f603c0035b0 con 0x7f605406aa40 2023-08-25T03:34:18.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.870+0000 7f6051ffb700 1 -- 172.21.15.2:0/4032120803 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1498554318 0 0) 0x7f603c0035b0 con 0x7f605406aa40 2023-08-25T03:34:18.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.870+0000 7f6051ffb700 1 -- 172.21.15.2:0/4032120803 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6054066050 msgr2=0x7f6054072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:18.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.870+0000 7f6051ffb700 1 --2- 172.21.15.2:0/4032120803 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6054066050 0x7f6054072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.870+0000 7f6051ffb700 1 -- 172.21.15.2:0/4032120803 >> v1:172.21.15.31:6789/0 conn(0x7f605406a660 legacy=0x7f605415f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:18.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.870+0000 7f6051ffb700 1 -- 172.21.15.2:0/4032120803 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60540b1870 con 0x7f605406aa40 2023-08-25T03:34:18.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.870+0000 7f6051ffb700 1 -- 172.21.15.2:0/4032120803 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f604c0019f0 con 0x7f605406aa40 2023-08-25T03:34:18.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.870+0000 7f6051ffb700 1 -- 172.21.15.2:0/4032120803 >> v1:172.21.15.2:6789/0 conn(0x7f605406aa40 legacy=0x7f605415aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:18.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.870+0000 7f6051ffb700 1 --2- 172.21.15.2:0/4032120803 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6054066050 0x7f603c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:18.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.870+0000 7f6051ffb700 1 --2- 172.21.15.2:0/4032120803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f603c003fb0 0x7f603c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:18.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.870+0000 7f6051ffb700 1 --2- 172.21.15.2:0/4032120803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f603c004d90 0x7f603c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:18.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.870+0000 7f6051ffb700 1 -- 172.21.15.2:0/4032120803 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60540b1590 con 0x7f603c004d90 2023-08-25T03:34:18.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.870+0000 7f6051ffb700 1 -- 172.21.15.2:0/4032120803 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c00bba0 con 0x7f603c003fb0 2023-08-25T03:34:18.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.870+0000 7f6051ffb700 1 -- 172.21.15.2:0/4032120803 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f603c00bd60 con 0x7f6054066050 2023-08-25T03:34:18.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.870+0000 7f6052ffd700 1 --2- 172.21.15.2:0/4032120803 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6054066050 0x7f603c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:18.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.870+0000 7f60527fc700 1 --2- 172.21.15.2:0/4032120803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f603c003fb0 0x7f603c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:18.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.870+0000 7f60537fe700 1 --2- 172.21.15.2:0/4032120803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f603c004d90 0x7f603c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:18.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.870+0000 7f60537fe700 1 -- 172.21.15.2:0/4032120803 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6054066050 msgr2=0x7f603c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:18.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.870+0000 7f60537fe700 1 --2- 172.21.15.2:0/4032120803 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6054066050 0x7f603c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.870+0000 7f60537fe700 1 -- 172.21.15.2:0/4032120803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f603c003fb0 msgr2=0x7f603c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:18.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.870+0000 7f60537fe700 1 --2- 172.21.15.2:0/4032120803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f603c003fb0 0x7f603c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.871+0000 7f60537fe700 1 -- 172.21.15.2:0/4032120803 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603c0035b0 con 0x7f603c004d90 2023-08-25T03:34:18.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.871+0000 7f6052ffd700 1 --2- 172.21.15.2:0/4032120803 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6054066050 0x7f603c003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:34:18.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.871+0000 7f60537fe700 1 --2- 172.21.15.2:0/4032120803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f603c004d90 0x7f603c005210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f604c00a830 tx=0x7f603c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc055e5398db779 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:18.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.871+0000 7f6051ffb700 1 -- 172.21.15.2:0/4032120803 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604c00a2f0 con 0x7f603c004d90 2023-08-25T03:34:18.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.871+0000 7f6051ffb700 1 -- 172.21.15.2:0/4032120803 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f604c012600 con 0x7f603c004d90 2023-08-25T03:34:18.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.871+0000 7f6051ffb700 1 -- 172.21.15.2:0/4032120803 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604c0129b0 con 0x7f603c004d90 2023-08-25T03:34:18.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.871+0000 7f60596dc700 1 -- 172.21.15.2:0/4032120803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f603c004d90 msgr2=0x7f603c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:18.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.871+0000 7f60596dc700 1 --2- 172.21.15.2:0/4032120803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f603c004d90 0x7f603c005210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f604c00a830 tx=0x7f603c005880 comp rx=0 tx=0).stop 2023-08-25T03:34:18.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.871+0000 7f60537fe700 1 -- 172.21.15.2:0/4032120803 reap_dead start 2023-08-25T03:34:18.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.871+0000 7f60596dc700 1 -- 172.21.15.2:0/4032120803 shutdown_connections 2023-08-25T03:34:18.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.871+0000 7f60596dc700 1 -- 172.21.15.2:0/4032120803 >> 172.21.15.2:0/4032120803 conn(0x7f60540cf500 msgr2=0x7f60540cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:18.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.871+0000 7f60596dc700 1 -- 172.21.15.2:0/4032120803 shutdown_connections 2023-08-25T03:34:18.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.871+0000 7f60596dc700 1 -- 172.21.15.2:0/4032120803 wait complete. 2023-08-25T03:34:18.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.873+0000 7f60596dc700 1 Processor -- start 2023-08-25T03:34:18.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.873+0000 7f60596dc700 1 -- start start 2023-08-25T03:34:18.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.873+0000 7f60596dc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f605415b0b0 0x7f60540acd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:18.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.873+0000 7f60596dc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f605415b490 0x7f60540ad240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:18.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.873+0000 7f60596dc700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f60540c80d0 0x7f60540c8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:18.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.873+0000 7f60596dc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60540c8aa0 con 0x7f605415b0b0 2023-08-25T03:34:18.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.873+0000 7f60596dc700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60540c8cd0 con 0x7f605415b490 2023-08-25T03:34:18.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.873+0000 7f60596dc700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f605406e190 con 0x7f60540c80d0 2023-08-25T03:34:18.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.874+0000 7f6052ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f605415b0b0 0x7f60540acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:18.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.874+0000 7f6052ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f605415b0b0 0x7f60540acd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59808/0 (socket says 172.21.15.2:59808) 2023-08-25T03:34:18.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.874+0000 7f6052ffd700 1 -- 172.21.15.2:0/934732034 learned_addr learned my addr 172.21.15.2:0/934732034 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:18.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.874+0000 7f60527fc700 1 --2- 172.21.15.2:0/934732034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f605415b490 0x7f60540ad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:18.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.874+0000 7f60537fe700 1 --2- 172.21.15.2:0/934732034 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f60540c80d0 0x7f60540c8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:18.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.874+0000 7f6052ffd700 1 -- 172.21.15.2:0/934732034 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f60540c80d0 msgr2=0x7f60540c8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:18.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.874+0000 7f6052ffd700 1 --2- 172.21.15.2:0/934732034 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f60540c80d0 0x7f60540c8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.874+0000 7f6052ffd700 1 -- 172.21.15.2:0/934732034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f605415b490 msgr2=0x7f60540ad240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:18.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.874+0000 7f6052ffd700 1 --2- 172.21.15.2:0/934732034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f605415b490 0x7f60540ad240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:18.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.874+0000 7f6052ffd700 1 -- 172.21.15.2:0/934732034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605406c300 con 0x7f605415b0b0 2023-08-25T03:34:18.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.874+0000 7f60527fc700 1 --2- 172.21.15.2:0/934732034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f605415b490 0x7f60540ad240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:18.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.874+0000 7f6052ffd700 1 --2- 172.21.15.2:0/934732034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f605415b0b0 0x7f60540acd00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f604400f990 tx=0x7f604400fd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:18.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.874+0000 7f603b7fe700 1 -- 172.21.15.2:0/934732034 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60440052e0 con 0x7f605415b0b0 2023-08-25T03:34:18.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.874+0000 7f603b7fe700 1 -- 172.21.15.2:0/934732034 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6044005440 con 0x7f605415b0b0 2023-08-25T03:34:18.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.874+0000 7f603b7fe700 1 -- 172.21.15.2:0/934732034 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60440055b0 con 0x7f605415b0b0 2023-08-25T03:34:18.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.874+0000 7f60596dc700 1 -- 172.21.15.2:0/934732034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f605406c5e0 con 0x7f605415b0b0 2023-08-25T03:34:18.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.874+0000 7f60596dc700 1 -- 172.21.15.2:0/934732034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f605406cba0 con 0x7f605415b0b0 2023-08-25T03:34:18.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.875+0000 7f60596dc700 1 -- 172.21.15.2:0/934732034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f605406cd60 con 0x7f605415b0b0 2023-08-25T03:34:18.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.876+0000 7f603b7fe700 1 -- 172.21.15.2:0/934732034 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6044003910 con 0x7f605415b0b0 2023-08-25T03:34:18.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.876+0000 7f603b7fe700 1 --2- 172.21.15.2:0/934732034 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f604006a0e0 0x7f604006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:18.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.876+0000 7f603b7fe700 1 -- 172.21.15.2:0/934732034 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f60440885a0 con 0x7f605415b0b0 2023-08-25T03:34:18.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.876+0000 7f60527fc700 1 --2- 172.21.15.2:0/934732034 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f604006a0e0 0x7f604006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:18.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.877+0000 7f60527fc700 1 --2- 172.21.15.2:0/934732034 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f604006a0e0 0x7f604006c5a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f605415b8d0 tx=0x7f6048008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:18.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:18.881+0000 7f603b7fe700 1 -- 172.21.15.2:0/934732034 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6044053c90 con 0x7f605415b0b0 2023-08-25T03:34:19.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.078+0000 7f60596dc700 1 -- 172.21.15.2:0/934732034 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f60540746d0 con 0x7f605415b0b0 2023-08-25T03:34:19.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.078+0000 7f603b7fe700 1 -- 172.21.15.2:0/934732034 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f60440572b0 con 0x7f605415b0b0 2023-08-25T03:34:19.080 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 8 2023-08-25T03:34:19.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.080+0000 7f60596dc700 1 -- 172.21.15.2:0/934732034 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f604006a0e0 msgr2=0x7f604006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:19.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.080+0000 7f60596dc700 1 --2- 172.21.15.2:0/934732034 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f604006a0e0 0x7f604006c5a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f605415b8d0 tx=0x7f6048008040 comp rx=0 tx=0).stop 2023-08-25T03:34:19.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.080+0000 7f60596dc700 1 -- 172.21.15.2:0/934732034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f605415b0b0 msgr2=0x7f60540acd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:19.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.080+0000 7f60596dc700 1 --2- 172.21.15.2:0/934732034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f605415b0b0 0x7f60540acd00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f604400f990 tx=0x7f604400fd50 comp rx=0 tx=0).stop 2023-08-25T03:34:19.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.080+0000 7f60596dc700 1 -- 172.21.15.2:0/934732034 shutdown_connections 2023-08-25T03:34:19.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.080+0000 7f60596dc700 1 --2- 172.21.15.2:0/934732034 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f604006a0e0 0x7f604006c5a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.080+0000 7f60596dc700 1 --2- 172.21.15.2:0/934732034 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f605415b0b0 0x7f60540acd00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.080+0000 7f60596dc700 1 --2- 172.21.15.2:0/934732034 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f60540c80d0 0x7f60540c8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.080+0000 7f60596dc700 1 --2- 172.21.15.2:0/934732034 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f605415b490 0x7f60540ad240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.080+0000 7f60596dc700 1 -- 172.21.15.2:0/934732034 >> 172.21.15.2:0/934732034 conn(0x7f60540cf500 msgr2=0x7f60540ac180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:19.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.081+0000 7f60596dc700 1 -- 172.21.15.2:0/934732034 shutdown_connections 2023-08-25T03:34:19.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.081+0000 7f60596dc700 1 -- 172.21.15.2:0/934732034 wait complete. 2023-08-25T03:34:19.093 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T03:34:19.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.220+0000 7f3bafe44700 1 Processor -- start 2023-08-25T03:34:19.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.220+0000 7f3bafe44700 1 -- start start 2023-08-25T03:34:19.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.221+0000 7f3bafe44700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f3ba81607f0 0x7f3ba8070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:19.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.221+0000 7f3bafe44700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3ba806a1e0 0x7f3ba806a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:19.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.221+0000 7f3bafe44700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba8160bd0 con 0x7f3ba80af180 2023-08-25T03:34:19.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.221+0000 7f3bafe44700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba806abd0 con 0x7f3ba81607f0 2023-08-25T03:34:19.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.221+0000 7f3bafe44700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba80713a0 con 0x7f3ba806a1e0 2023-08-25T03:34:19.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.221+0000 7f3badbe0700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f3ba81607f0 0x7f3ba8070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:19.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.221+0000 7f3badbe0700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f3ba81607f0 0x7f3ba8070e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59818/0 (socket says 172.21.15.2:59818) 2023-08-25T03:34:19.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.221+0000 7f3badbe0700 1 -- 172.21.15.2:0/2378615146 learned_addr learned my addr 172.21.15.2:0/2378615146 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:19.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.221+0000 7f3bad3df700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f3ba80af180 0x7f3ba80ac8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46482/0 (socket says 172.21.15.2:46482) 2023-08-25T03:34:19.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.222+0000 7f3bae3e1700 1 --2- 172.21.15.2:0/2378615146 >> v2:172.21.15.31:3300/0 conn(0x7f3ba806a1e0 0x7f3ba806a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:19.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.222+0000 7f3badbe0700 1 -- 172.21.15.2:0/2378615146 >> v2:172.21.15.31:3300/0 conn(0x7f3ba806a1e0 msgr2=0x7f3ba806a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:19.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.222+0000 7f3badbe0700 1 --2- 172.21.15.2:0/2378615146 >> v2:172.21.15.31:3300/0 conn(0x7f3ba806a1e0 0x7f3ba806a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.222+0000 7f3badbe0700 1 -- 172.21.15.2:0/2378615146 >> v1:172.21.15.2:6789/0 conn(0x7f3ba80af180 legacy=0x7f3ba80ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:19.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.222+0000 7f3badbe0700 1 -- 172.21.15.2:0/2378615146 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba80cdd00 con 0x7f3ba81607f0 2023-08-25T03:34:19.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.222+0000 7f3bacbde700 1 -- 172.21.15.2:0/2378615146 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1795246088 0 0) 0x7f3ba8160bd0 con 0x7f3ba80af180 2023-08-25T03:34:19.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.222+0000 7f3badbe0700 1 --2- 172.21.15.2:0/2378615146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba81607f0 0x7f3ba8070e60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3ba80cdc70 tx=0x7f3b98009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89069a7dfd2e0c8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:19.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.222+0000 7f3bacbde700 1 -- 172.21.15.2:0/2378615146 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b98011830 con 0x7f3ba81607f0 2023-08-25T03:34:19.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.222+0000 7f3bacbde700 1 -- 172.21.15.2:0/2378615146 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b98011990 con 0x7f3ba81607f0 2023-08-25T03:34:19.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.222+0000 7f3bacbde700 1 -- 172.21.15.2:0/2378615146 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b98011d20 con 0x7f3ba81607f0 2023-08-25T03:34:19.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.223+0000 7f3bafe44700 1 -- 172.21.15.2:0/2378615146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba81607f0 msgr2=0x7f3ba8070e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:19.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.223+0000 7f3bafe44700 1 --2- 172.21.15.2:0/2378615146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba81607f0 0x7f3ba8070e60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3ba80cdc70 tx=0x7f3b98009bd0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.223+0000 7f3bafe44700 1 -- 172.21.15.2:0/2378615146 shutdown_connections 2023-08-25T03:34:19.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.223+0000 7f3bafe44700 1 --2- 172.21.15.2:0/2378615146 >> v2:172.21.15.31:3300/0 conn(0x7f3ba806a1e0 0x7f3ba806a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.223+0000 7f3bafe44700 1 --2- 172.21.15.2:0/2378615146 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba81607f0 0x7f3ba8070e60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.223+0000 7f3bafe44700 1 -- 172.21.15.2:0/2378615146 >> 172.21.15.2:0/2378615146 conn(0x7f3ba80cfd90 msgr2=0x7f3ba80d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:19.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.223+0000 7f3bafe44700 1 -- 172.21.15.2:0/2378615146 shutdown_connections 2023-08-25T03:34:19.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.223+0000 7f3bafe44700 1 -- 172.21.15.2:0/2378615146 wait complete. 2023-08-25T03:34:19.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.224+0000 7f3bafe44700 1 Processor -- start 2023-08-25T03:34:19.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.224+0000 7f3bafe44700 1 -- start start 2023-08-25T03:34:19.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.225+0000 7f3bafe44700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba806a1e0 0x7f3ba8159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:19.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.225+0000 7f3bafe44700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ba80af180 0x7f3ba8158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:19.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.225+0000 7f3bafe44700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3ba81607f0 0x7f3ba81f5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:19.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.225+0000 7f3bafe44700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba81f69b0 con 0x7f3ba806a1e0 2023-08-25T03:34:19.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.225+0000 7f3bafe44700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba81f6c90 con 0x7f3ba80af180 2023-08-25T03:34:19.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.225+0000 7f3bafe44700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba81593f0 con 0x7f3ba81607f0 2023-08-25T03:34:19.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.225+0000 7f3badbe0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba806a1e0 0x7f3ba8159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:19.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.225+0000 7f3badbe0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba806a1e0 0x7f3ba8159ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59822/0 (socket says 172.21.15.2:59822) 2023-08-25T03:34:19.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.225+0000 7f3badbe0700 1 -- 172.21.15.2:0/462273306 learned_addr learned my addr 172.21.15.2:0/462273306 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:19.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.225+0000 7f3bad3df700 1 --2- 172.21.15.2:0/462273306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ba80af180 0x7f3ba8158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:19.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.225+0000 7f3bae3e1700 1 --2- 172.21.15.2:0/462273306 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3ba81607f0 0x7f3ba81f5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:19.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.225+0000 7f3badbe0700 1 -- 172.21.15.2:0/462273306 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3ba81607f0 msgr2=0x7f3ba81f5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:19.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.225+0000 7f3badbe0700 1 --2- 172.21.15.2:0/462273306 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3ba81607f0 0x7f3ba81f5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.225+0000 7f3badbe0700 1 -- 172.21.15.2:0/462273306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ba80af180 msgr2=0x7f3ba8158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:19.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.225+0000 7f3badbe0700 1 --2- 172.21.15.2:0/462273306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ba80af180 0x7f3ba8158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.225+0000 7f3badbe0700 1 -- 172.21.15.2:0/462273306 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b98009550 con 0x7f3ba806a1e0 2023-08-25T03:34:19.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.225+0000 7f3bae3e1700 1 --2- 172.21.15.2:0/462273306 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3ba81607f0 0x7f3ba81f5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:19.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.225+0000 7f3badbe0700 1 --2- 172.21.15.2:0/462273306 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba806a1e0 0x7f3ba8159ed0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009c10 tx=0x7f3b98004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:19.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.225+0000 7f3b9e7fc700 1 -- 172.21.15.2:0/462273306 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b980036a0 con 0x7f3ba806a1e0 2023-08-25T03:34:19.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.226+0000 7f3b9e7fc700 1 -- 172.21.15.2:0/462273306 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b98003800 con 0x7f3ba806a1e0 2023-08-25T03:34:19.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.226+0000 7f3bafe44700 1 -- 172.21.15.2:0/462273306 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba81596d0 con 0x7f3ba806a1e0 2023-08-25T03:34:19.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.226+0000 7f3b9e7fc700 1 -- 172.21.15.2:0/462273306 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b98003b70 con 0x7f3ba806a1e0 2023-08-25T03:34:19.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.226+0000 7f3bafe44700 1 -- 172.21.15.2:0/462273306 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba815bc90 con 0x7f3ba806a1e0 2023-08-25T03:34:19.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.227+0000 7f3bafe44700 1 -- 172.21.15.2:0/462273306 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ba8159bf0 con 0x7f3ba806a1e0 2023-08-25T03:34:19.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.227+0000 7f3b9e7fc700 1 -- 172.21.15.2:0/462273306 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3b98003cd0 con 0x7f3ba806a1e0 2023-08-25T03:34:19.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.227+0000 7f3b9e7fc700 1 --2- 172.21.15.2:0/462273306 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3b8c06a0e0 0x7f3b8c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:19.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.228+0000 7f3b9e7fc700 1 -- 172.21.15.2:0/462273306 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3b98021030 con 0x7f3ba806a1e0 2023-08-25T03:34:19.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.228+0000 7f3bad3df700 1 --2- 172.21.15.2:0/462273306 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3b8c06a0e0 0x7f3b8c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:19.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.228+0000 7f3bad3df700 1 --2- 172.21.15.2:0/462273306 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3b8c06a0e0 0x7f3b8c06c5a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3ba815b730 tx=0x7f3ba4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:19.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.232+0000 7f3b9e7fc700 1 -- 172.21.15.2:0/462273306 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3ba8159bf0 con 0x7f3ba806a1e0 2023-08-25T03:34:19.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.432+0000 7f3bafe44700 1 -- 172.21.15.2:0/462273306 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3ba80746d0 con 0x7f3ba806a1e0 2023-08-25T03:34:19.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.432+0000 7f3b9e7fc700 1 -- 172.21.15.2:0/462273306 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3b98089e60 con 0x7f3ba806a1e0 2023-08-25T03:34:19.433 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-08-25T03:34:19.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.434+0000 7f3bafe44700 1 -- 172.21.15.2:0/462273306 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3b8c06a0e0 msgr2=0x7f3b8c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:19.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.434+0000 7f3bafe44700 1 --2- 172.21.15.2:0/462273306 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3b8c06a0e0 0x7f3b8c06c5a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3ba815b730 tx=0x7f3ba4009040 comp rx=0 tx=0).stop 2023-08-25T03:34:19.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.434+0000 7f3bafe44700 1 -- 172.21.15.2:0/462273306 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba806a1e0 msgr2=0x7f3ba8159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:19.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.434+0000 7f3bafe44700 1 --2- 172.21.15.2:0/462273306 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba806a1e0 0x7f3ba8159ed0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009c10 tx=0x7f3b98004750 comp rx=0 tx=0).stop 2023-08-25T03:34:19.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.434+0000 7f3bafe44700 1 -- 172.21.15.2:0/462273306 shutdown_connections 2023-08-25T03:34:19.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.434+0000 7f3bafe44700 1 --2- 172.21.15.2:0/462273306 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3b8c06a0e0 0x7f3b8c06c5a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.434+0000 7f3bafe44700 1 --2- 172.21.15.2:0/462273306 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3ba806a1e0 0x7f3ba8159ed0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.434+0000 7f3bafe44700 1 --2- 172.21.15.2:0/462273306 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3ba81607f0 0x7f3ba81f5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.434+0000 7f3bafe44700 1 --2- 172.21.15.2:0/462273306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3ba80af180 0x7f3ba8158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.434+0000 7f3bafe44700 1 -- 172.21.15.2:0/462273306 >> 172.21.15.2:0/462273306 conn(0x7f3ba80cfd90 msgr2=0x7f3ba80d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:19.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.434+0000 7f3bafe44700 1 -- 172.21.15.2:0/462273306 shutdown_connections 2023-08-25T03:34:19.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.434+0000 7f3bafe44700 1 -- 172.21.15.2:0/462273306 wait complete. 2023-08-25T03:34:19.448 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T03:34:19.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.575+0000 7f6c50288700 1 Processor -- start 2023-08-25T03:34:19.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.575+0000 7f6c50288700 1 -- start start 2023-08-25T03:34:19.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.576+0000 7f6c50288700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6c480cacf0 0x7f6c480cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:19.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.576+0000 7f6c50288700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c4806eaf0 con 0x7f6c480cedf0 2023-08-25T03:34:19.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.576+0000 7f6c50288700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c4815e2c0 con 0x7f6c48159f10 2023-08-25T03:34:19.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.576+0000 7f6c50288700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c4806e850 con 0x7f6c480cacf0 2023-08-25T03:34:19.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.576+0000 7f6c4d823700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f6c480cedf0 0x7f6c4806ef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46496/0 (socket says 172.21.15.2:46496) 2023-08-25T03:34:19.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.576+0000 7f6c4d823700 1 -- 172.21.15.2:0/2791572677 learned_addr learned my addr 172.21.15.2:0/2791572677 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:19.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.576+0000 7f6c4e825700 1 --2- 172.21.15.2:0/2791572677 >> v2:172.21.15.31:3300/0 conn(0x7f6c480cacf0 0x7f6c480cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:19.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.576+0000 7f6c4d022700 1 -- 172.21.15.2:0/2791572677 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2278987716 0 0) 0x7f6c4806eaf0 con 0x7f6c480cedf0 2023-08-25T03:34:19.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.576+0000 7f6c4d022700 1 -- 172.21.15.2:0/2791572677 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c340035b0 con 0x7f6c480cedf0 2023-08-25T03:34:19.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.576+0000 7f6c4d022700 1 -- 172.21.15.2:0/2791572677 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2553595371 0 0) 0x7f6c4815e2c0 con 0x7f6c48159f10 2023-08-25T03:34:19.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.576+0000 7f6c4d022700 1 -- 172.21.15.2:0/2791572677 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c4806eaf0 con 0x7f6c48159f10 2023-08-25T03:34:19.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.576+0000 7f6c4d022700 1 -- 172.21.15.2:0/2791572677 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1645459786 0 0) 0x7f6c340035b0 con 0x7f6c480cedf0 2023-08-25T03:34:19.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.576+0000 7f6c4d022700 1 -- 172.21.15.2:0/2791572677 >> v2:172.21.15.31:3300/0 conn(0x7f6c480cacf0 msgr2=0x7f6c480cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:19.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.576+0000 7f6c4d022700 1 --2- 172.21.15.2:0/2791572677 >> v2:172.21.15.31:3300/0 conn(0x7f6c480cacf0 0x7f6c480cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.576+0000 7f6c4d022700 1 -- 172.21.15.2:0/2791572677 >> v1:172.21.15.31:6789/0 conn(0x7f6c48159f10 legacy=0x7f6c4815dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:19.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.576+0000 7f6c4d022700 1 -- 172.21.15.2:0/2791572677 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4815e5a0 con 0x7f6c480cedf0 2023-08-25T03:34:19.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.576+0000 7f6c4d022700 1 -- 172.21.15.2:0/2791572677 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f6c38001670 con 0x7f6c480cedf0 2023-08-25T03:34:19.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.576+0000 7f6c4d022700 1 -- 172.21.15.2:0/2791572677 >> v1:172.21.15.2:6789/0 conn(0x7f6c480cedf0 legacy=0x7f6c4806ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:19.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.576+0000 7f6c4d022700 1 --2- 172.21.15.2:0/2791572677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c34003b90 0x7f6c34003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:19.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.577+0000 7f6c4d022700 1 --2- 172.21.15.2:0/2791572677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c34004850 0x7f6c34004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:19.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.577+0000 7f6c4d022700 1 --2- 172.21.15.2:0/2791572677 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c34005210 0x7f6c3400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:19.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.577+0000 7f6c4d022700 1 -- 172.21.15.2:0/2791572677 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4806e850 con 0x7f6c34004850 2023-08-25T03:34:19.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.577+0000 7f6c4d022700 1 -- 172.21.15.2:0/2791572677 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3400c090 con 0x7f6c34003b90 2023-08-25T03:34:19.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.577+0000 7f6c4d022700 1 -- 172.21.15.2:0/2791572677 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3400c250 con 0x7f6c34005210 2023-08-25T03:34:19.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.577+0000 7f6c4e825700 1 --2- 172.21.15.2:0/2791572677 >> v2:172.21.15.31:3300/0 conn(0x7f6c480cacf0 0x7f6c480cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:34:19.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.577+0000 7f6c4d823700 1 --2- 172.21.15.2:0/2791572677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c34004850 0x7f6c34004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:19.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.577+0000 7f6c4e024700 1 --2- 172.21.15.2:0/2791572677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c34003b90 0x7f6c34003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:19.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.577+0000 7f6c4e825700 1 --2- 172.21.15.2:0/2791572677 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c34005210 0x7f6c3400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:19.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.577+0000 7f6c4d823700 1 -- 172.21.15.2:0/2791572677 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c34005210 msgr2=0x7f6c3400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:19.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.577+0000 7f6c4d823700 1 --2- 172.21.15.2:0/2791572677 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c34005210 0x7f6c3400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.577+0000 7f6c4d823700 1 -- 172.21.15.2:0/2791572677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c34003b90 msgr2=0x7f6c34003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:19.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.577+0000 7f6c4d823700 1 --2- 172.21.15.2:0/2791572677 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c34003b90 0x7f6c34003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.577+0000 7f6c4d823700 1 -- 172.21.15.2:0/2791572677 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c340035b0 con 0x7f6c34004850 2023-08-25T03:34:19.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.577+0000 7f6c4e825700 1 -- 172.21.15.2:0/2791572677 reap_dead start 2023-08-25T03:34:19.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.577+0000 7f6c4d823700 1 --2- 172.21.15.2:0/2791572677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c34004850 0x7f6c34004cd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800c210 tx=0x7f6c34005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c146f5a72e3cdb4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:19.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.577+0000 7f6c4d022700 1 -- 172.21.15.2:0/2791572677 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3800cd50 con 0x7f6c34004850 2023-08-25T03:34:19.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.577+0000 7f6c4d022700 1 -- 172.21.15.2:0/2791572677 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c38005960 con 0x7f6c34004850 2023-08-25T03:34:19.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.577+0000 7f6c4d022700 1 -- 172.21.15.2:0/2791572677 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c38005d10 con 0x7f6c34004850 2023-08-25T03:34:19.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.578+0000 7f6c50288700 1 -- 172.21.15.2:0/2791572677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c34004850 msgr2=0x7f6c34004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:19.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.578+0000 7f6c50288700 1 --2- 172.21.15.2:0/2791572677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c34004850 0x7f6c34004cd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f6c3800c210 tx=0x7f6c34005880 comp rx=0 tx=0).stop 2023-08-25T03:34:19.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.578+0000 7f6c50288700 1 -- 172.21.15.2:0/2791572677 shutdown_connections 2023-08-25T03:34:19.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.578+0000 7f6c50288700 1 --2- 172.21.15.2:0/2791572677 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c34004850 0x7f6c34004cd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.578+0000 7f6c50288700 1 -- 172.21.15.2:0/2791572677 >> 172.21.15.2:0/2791572677 conn(0x7f6c480cfd90 msgr2=0x7f6c480d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:19.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.578+0000 7f6c50288700 1 -- 172.21.15.2:0/2791572677 shutdown_connections 2023-08-25T03:34:19.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.578+0000 7f6c50288700 1 -- 172.21.15.2:0/2791572677 wait complete. 2023-08-25T03:34:19.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.579+0000 7f6c50288700 1 Processor -- start 2023-08-25T03:34:19.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.580+0000 7f6c50288700 1 -- start start 2023-08-25T03:34:19.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.580+0000 7f6c50288700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c34004850 0x7f6c48071060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:19.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.580+0000 7f6c50288700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c481548d0 0x7f6c4806a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:19.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.580+0000 7f6c50288700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c48154d50 0x7f6c48153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:19.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.580+0000 7f6c50288700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c480acaf0 con 0x7f6c34004850 2023-08-25T03:34:19.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.581+0000 7f6c50288700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c480715a0 con 0x7f6c481548d0 2023-08-25T03:34:19.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.581+0000 7f6c50288700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c481f3890 con 0x7f6c48154d50 2023-08-25T03:34:19.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.581+0000 7f6c4e024700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c34004850 0x7f6c48071060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:19.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.581+0000 7f6c4e024700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c34004850 0x7f6c48071060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59842/0 (socket says 172.21.15.2:59842) 2023-08-25T03:34:19.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.581+0000 7f6c4e024700 1 -- 172.21.15.2:0/3176951240 learned_addr learned my addr 172.21.15.2:0/3176951240 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:19.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.581+0000 7f6c4e024700 1 -- 172.21.15.2:0/3176951240 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c48154d50 msgr2=0x7f6c48153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:19.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.581+0000 7f6c4e024700 1 --2- 172.21.15.2:0/3176951240 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c48154d50 0x7f6c48153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.581+0000 7f6c4e024700 1 -- 172.21.15.2:0/3176951240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c481548d0 msgr2=0x7f6c4806a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:19.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.581+0000 7f6c4d823700 1 --2- 172.21.15.2:0/3176951240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c481548d0 0x7f6c4806a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:19.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.581+0000 7f6c4e024700 1 --2- 172.21.15.2:0/3176951240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c481548d0 0x7f6c4806a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.581+0000 7f6c4e024700 1 -- 172.21.15.2:0/3176951240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c481f2ff0 con 0x7f6c34004850 2023-08-25T03:34:19.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.581+0000 7f6c4e024700 1 --2- 172.21.15.2:0/3176951240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c34004850 0x7f6c48071060 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400f7a0 tx=0x7f6c4400fb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:19.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.581+0000 7f6c3effd700 1 -- 172.21.15.2:0/3176951240 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c440052e0 con 0x7f6c34004850 2023-08-25T03:34:19.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.581+0000 7f6c3effd700 1 -- 172.21.15.2:0/3176951240 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c44005440 con 0x7f6c34004850 2023-08-25T03:34:19.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.581+0000 7f6c50288700 1 -- 172.21.15.2:0/3176951240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c481f32d0 con 0x7f6c34004850 2023-08-25T03:34:19.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.581+0000 7f6c3effd700 1 -- 172.21.15.2:0/3176951240 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c440038c0 con 0x7f6c34004850 2023-08-25T03:34:19.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.581+0000 7f6c50288700 1 -- 172.21.15.2:0/3176951240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c4806a990 con 0x7f6c34004850 2023-08-25T03:34:19.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.582+0000 7f6c50288700 1 -- 172.21.15.2:0/3176951240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c4806ab50 con 0x7f6c34004850 2023-08-25T03:34:19.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.583+0000 7f6c3effd700 1 -- 172.21.15.2:0/3176951240 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6c44003a20 con 0x7f6c34004850 2023-08-25T03:34:19.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.583+0000 7f6c3effd700 1 --2- 172.21.15.2:0/3176951240 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6c2c06a0e0 0x7f6c2c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:19.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.583+0000 7f6c3effd700 1 -- 172.21.15.2:0/3176951240 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6c44088310 con 0x7f6c34004850 2023-08-25T03:34:19.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.583+0000 7f6c4d823700 1 --2- 172.21.15.2:0/3176951240 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6c2c06a0e0 0x7f6c2c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:19.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.583+0000 7f6c4d823700 1 --2- 172.21.15.2:0/3176951240 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6c2c06a0e0 0x7f6c2c06c5a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6c38015040 tx=0x7f6c38007f90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:19.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.587+0000 7f6c3effd700 1 -- 172.21.15.2:0/3176951240 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6c44053a00 con 0x7f6c34004850 2023-08-25T03:34:19.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.792+0000 7f6c50288700 1 -- 172.21.15.2:0/3176951240 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6c480746d0 con 0x7f6c34004850 2023-08-25T03:34:19.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.793+0000 7f6c3effd700 1 -- 172.21.15.2:0/3176951240 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6c44057020 con 0x7f6c34004850 2023-08-25T03:34:19.794 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-08-25T03:34:19.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.795+0000 7f6c50288700 1 -- 172.21.15.2:0/3176951240 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6c2c06a0e0 msgr2=0x7f6c2c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:19.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.795+0000 7f6c50288700 1 --2- 172.21.15.2:0/3176951240 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6c2c06a0e0 0x7f6c2c06c5a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f6c38015040 tx=0x7f6c38007f90 comp rx=0 tx=0).stop 2023-08-25T03:34:19.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.795+0000 7f6c50288700 1 -- 172.21.15.2:0/3176951240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c34004850 msgr2=0x7f6c48071060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:19.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.795+0000 7f6c50288700 1 --2- 172.21.15.2:0/3176951240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c34004850 0x7f6c48071060 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6c4400f7a0 tx=0x7f6c4400fb60 comp rx=0 tx=0).stop 2023-08-25T03:34:19.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.795+0000 7f6c50288700 1 -- 172.21.15.2:0/3176951240 shutdown_connections 2023-08-25T03:34:19.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.795+0000 7f6c50288700 1 --2- 172.21.15.2:0/3176951240 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6c2c06a0e0 0x7f6c2c06c5a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.795+0000 7f6c50288700 1 --2- 172.21.15.2:0/3176951240 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6c34004850 0x7f6c48071060 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.795+0000 7f6c50288700 1 --2- 172.21.15.2:0/3176951240 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c48154d50 0x7f6c48153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.795+0000 7f6c50288700 1 --2- 172.21.15.2:0/3176951240 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c481548d0 0x7f6c4806a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.795+0000 7f6c50288700 1 -- 172.21.15.2:0/3176951240 >> 172.21.15.2:0/3176951240 conn(0x7f6c480cfd90 msgr2=0x7f6c480af800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:19.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.795+0000 7f6c50288700 1 -- 172.21.15.2:0/3176951240 shutdown_connections 2023-08-25T03:34:19.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.795+0000 7f6c50288700 1 -- 172.21.15.2:0/3176951240 wait complete. 2023-08-25T03:34:19.808 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T03:34:19.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.932+0000 7f2d96ad3700 1 Processor -- start 2023-08-25T03:34:19.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.932+0000 7f2d96ad3700 1 -- start start 2023-08-25T03:34:19.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.933+0000 7f2d96ad3700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2d9006b1f0 0x7f2d9006b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:19.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.933+0000 7f2d96ad3700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2d900b1c40 0x7f2d900ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:19.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.933+0000 7f2d96ad3700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d900b3bb0 con 0x7f2d900b4480 2023-08-25T03:34:19.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.933+0000 7f2d96ad3700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d9006bb10 con 0x7f2d9006b1f0 2023-08-25T03:34:19.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.933+0000 7f2d96ad3700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d900c4760 con 0x7f2d900b1c40 2023-08-25T03:34:19.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.933+0000 7f2d9486f700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2d9006b1f0 0x7f2d9006b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:19.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.933+0000 7f2d9486f700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2d9006b1f0 0x7f2d9006b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59850/0 (socket says 172.21.15.2:59850) 2023-08-25T03:34:19.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.933+0000 7f2d9486f700 1 -- 172.21.15.2:0/2018237102 learned_addr learned my addr 172.21.15.2:0/2018237102 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:19.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.933+0000 7f2d9486f700 1 -- 172.21.15.2:0/2018237102 >> v2:172.21.15.31:3300/0 conn(0x7f2d900b1c40 msgr2=0x7f2d900ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:19.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.933+0000 7f2d8ffff700 1 --2- 172.21.15.2:0/2018237102 >> v2:172.21.15.31:3300/0 conn(0x7f2d900b1c40 0x7f2d900ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:19.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.933+0000 7f2d9486f700 1 --2- 172.21.15.2:0/2018237102 >> v2:172.21.15.31:3300/0 conn(0x7f2d900b1c40 0x7f2d900ce260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.933+0000 7f2d9486f700 1 -- 172.21.15.2:0/2018237102 >> v1:172.21.15.31:6789/0 conn(0x7f2d900b4480 legacy=0x7f2d900ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:19.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.933+0000 7f2d9486f700 1 -- 172.21.15.2:0/2018237102 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d900b3560 con 0x7f2d9006b1f0 2023-08-25T03:34:19.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.933+0000 7f2d9486f700 1 --2- 172.21.15.2:0/2018237102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d9006b1f0 0x7f2d9006b5d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2d9006f240 tx=0x7f2d84009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab2f397aa634a1b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:19.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.933+0000 7f2d8f7fe700 1 -- 172.21.15.2:0/2018237102 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84004230 con 0x7f2d9006b1f0 2023-08-25T03:34:19.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.933+0000 7f2d8f7fe700 1 -- 172.21.15.2:0/2018237102 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d84004390 con 0x7f2d9006b1f0 2023-08-25T03:34:19.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.933+0000 7f2d8f7fe700 1 -- 172.21.15.2:0/2018237102 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d840036a0 con 0x7f2d9006b1f0 2023-08-25T03:34:19.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.934+0000 7f2d96ad3700 1 -- 172.21.15.2:0/2018237102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d9006b1f0 msgr2=0x7f2d9006b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:19.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.934+0000 7f2d96ad3700 1 --2- 172.21.15.2:0/2018237102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d9006b1f0 0x7f2d9006b5d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2d9006f240 tx=0x7f2d84009bd0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.934+0000 7f2d96ad3700 1 -- 172.21.15.2:0/2018237102 shutdown_connections 2023-08-25T03:34:19.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.934+0000 7f2d96ad3700 1 --2- 172.21.15.2:0/2018237102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d9006b1f0 0x7f2d9006b5d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.934+0000 7f2d96ad3700 1 --2- 172.21.15.2:0/2018237102 >> v2:172.21.15.31:3300/0 conn(0x7f2d900b1c40 0x7f2d900ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.934+0000 7f2d96ad3700 1 -- 172.21.15.2:0/2018237102 >> 172.21.15.2:0/2018237102 conn(0x7f2d900cfd10 msgr2=0x7f2d9006ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:19.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.934+0000 7f2d96ad3700 1 -- 172.21.15.2:0/2018237102 shutdown_connections 2023-08-25T03:34:19.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.934+0000 7f2d96ad3700 1 -- 172.21.15.2:0/2018237102 wait complete. 2023-08-25T03:34:19.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.935+0000 7f2d96ad3700 1 Processor -- start 2023-08-25T03:34:19.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.935+0000 7f2d96ad3700 1 -- start start 2023-08-25T03:34:19.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.936+0000 7f2d96ad3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d9006b1f0 0x7f2d901578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:19.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.936+0000 7f2d96ad3700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2d900b1c40 0x7f2d901f2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:19.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.936+0000 7f2d96ad3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d900b4480 0x7f2d90071810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:19.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.936+0000 7f2d96ad3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d901f3870 con 0x7f2d9006b1f0 2023-08-25T03:34:19.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.936+0000 7f2d96ad3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d901f2da0 con 0x7f2d900b4480 2023-08-25T03:34:19.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.936+0000 7f2d96ad3700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d90151e40 con 0x7f2d900b1c40 2023-08-25T03:34:19.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.936+0000 7f2d9486f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d9006b1f0 0x7f2d901578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:19.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.936+0000 7f2d9486f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d9006b1f0 0x7f2d901578b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59854/0 (socket says 172.21.15.2:59854) 2023-08-25T03:34:19.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.936+0000 7f2d9486f700 1 -- 172.21.15.2:0/2597913867 learned_addr learned my addr 172.21.15.2:0/2597913867 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:19.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.936+0000 7f2d95070700 1 --2- 172.21.15.2:0/2597913867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d900b4480 0x7f2d90071810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:19.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.936+0000 7f2d8ffff700 1 --2- 172.21.15.2:0/2597913867 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2d900b1c40 0x7f2d901f2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:19.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.937+0000 7f2d9486f700 1 -- 172.21.15.2:0/2597913867 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2d900b1c40 msgr2=0x7f2d901f2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:19.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.937+0000 7f2d9486f700 1 --2- 172.21.15.2:0/2597913867 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2d900b1c40 0x7f2d901f2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.937+0000 7f2d9486f700 1 -- 172.21.15.2:0/2597913867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d900b4480 msgr2=0x7f2d90071810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:19.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.937+0000 7f2d9486f700 1 --2- 172.21.15.2:0/2597913867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d900b4480 0x7f2d90071810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:19.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.937+0000 7f2d9486f700 1 -- 172.21.15.2:0/2597913867 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d84009550 con 0x7f2d9006b1f0 2023-08-25T03:34:19.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.937+0000 7f2d9486f700 1 --2- 172.21.15.2:0/2597913867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d9006b1f0 0x7f2d901578b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2d90072940 tx=0x7f2d84004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:19.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.937+0000 7f2d8d7fa700 1 -- 172.21.15.2:0/2597913867 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84003c90 con 0x7f2d9006b1f0 2023-08-25T03:34:19.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.937+0000 7f2d8d7fa700 1 -- 172.21.15.2:0/2597913867 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8401f070 con 0x7f2d9006b1f0 2023-08-25T03:34:19.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.937+0000 7f2d96ad3700 1 -- 172.21.15.2:0/2597913867 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d901f2fd0 con 0x7f2d9006b1f0 2023-08-25T03:34:19.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.937+0000 7f2d8d7fa700 1 -- 172.21.15.2:0/2597913867 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d84011870 con 0x7f2d9006b1f0 2023-08-25T03:34:19.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.937+0000 7f2d96ad3700 1 -- 172.21.15.2:0/2597913867 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d901f3590 con 0x7f2d9006b1f0 2023-08-25T03:34:19.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.938+0000 7f2d96ad3700 1 -- 172.21.15.2:0/2597913867 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d90151c10 con 0x7f2d9006b1f0 2023-08-25T03:34:19.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.938+0000 7f2d8d7fa700 1 -- 172.21.15.2:0/2597913867 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2d84021030 con 0x7f2d9006b1f0 2023-08-25T03:34:19.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.939+0000 7f2d8d7fa700 1 --2- 172.21.15.2:0/2597913867 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2d7406a0e0 0x7f2d7406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:19.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.939+0000 7f2d8ffff700 1 --2- 172.21.15.2:0/2597913867 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2d7406a0e0 0x7f2d7406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:19.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.939+0000 7f2d8d7fa700 1 -- 172.21.15.2:0/2597913867 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2d84089970 con 0x7f2d9006b1f0 2023-08-25T03:34:19.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.939+0000 7f2d8ffff700 1 --2- 172.21.15.2:0/2597913867 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2d7406a0e0 0x7f2d7406c5a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2d90158c00 tx=0x7f2d7c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:19.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:19.944+0000 7f2d8d7fa700 1 -- 172.21.15.2:0/2597913867 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2d90151c10 con 0x7f2d9006b1f0 2023-08-25T03:34:20.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.188+0000 7f2d96ad3700 1 -- 172.21.15.2:0/2597913867 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d90151c10 con 0x7f2d9006b1f0 2023-08-25T03:34:20.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.189+0000 7f2d8d7fa700 1 -- 172.21.15.2:0/2597913867 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4857 (secure 0 0 0) 0x7f2d90151c10 con 0x7f2d9006b1f0 2023-08-25T03:34:20.190 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:20.190 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2074312983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2074312983},{"type":"v1","addr":"172.21.15.2:6837","nonce":2074312983}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4059180922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4059180922},{"type":"v1","addr":"172.21.15.31:6835","nonce":4059180922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/922384230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":922384230},{"type":"v1","addr":"172.21.15.31:6834","nonce":922384230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T03:34:03.009109+0000","modified":"2023-08-25T03:34:06.613200+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.2:6835/193990892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":193990892},{"type":"v1","addr":"172.21.15.2:6835","nonce":193990892}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T03:34:20.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.190+0000 7f2d96ad3700 1 -- 172.21.15.2:0/2597913867 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2d7406a0e0 msgr2=0x7f2d7406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:20.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.190+0000 7f2d96ad3700 1 --2- 172.21.15.2:0/2597913867 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2d7406a0e0 0x7f2d7406c5a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2d90158c00 tx=0x7f2d7c008040 comp rx=0 tx=0).stop 2023-08-25T03:34:20.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.190+0000 7f2d96ad3700 1 -- 172.21.15.2:0/2597913867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d9006b1f0 msgr2=0x7f2d901578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:20.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.191+0000 7f2d96ad3700 1 --2- 172.21.15.2:0/2597913867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d9006b1f0 0x7f2d901578b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2d90072940 tx=0x7f2d84004750 comp rx=0 tx=0).stop 2023-08-25T03:34:20.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.191+0000 7f2d96ad3700 1 -- 172.21.15.2:0/2597913867 shutdown_connections 2023-08-25T03:34:20.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.191+0000 7f2d96ad3700 1 --2- 172.21.15.2:0/2597913867 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2d7406a0e0 0x7f2d7406c5a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:20.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.191+0000 7f2d96ad3700 1 --2- 172.21.15.2:0/2597913867 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2d9006b1f0 0x7f2d901578b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:20.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.191+0000 7f2d96ad3700 1 --2- 172.21.15.2:0/2597913867 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2d900b4480 0x7f2d90071810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:20.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.191+0000 7f2d96ad3700 1 --2- 172.21.15.2:0/2597913867 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2d900b1c40 0x7f2d901f2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:20.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.191+0000 7f2d96ad3700 1 -- 172.21.15.2:0/2597913867 >> 172.21.15.2:0/2597913867 conn(0x7f2d900cfd10 msgr2=0x7f2d90066150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:20.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.191+0000 7f2d96ad3700 1 -- 172.21.15.2:0/2597913867 shutdown_connections 2023-08-25T03:34:20.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.191+0000 7f2d96ad3700 1 -- 172.21.15.2:0/2597913867 wait complete. 2023-08-25T03:34:20.195 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 8 2023-08-25T03:34:20.204 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:34:20.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.336+0000 7ff76348f700 1 Processor -- start 2023-08-25T03:34:20.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.336+0000 7ff76348f700 1 -- start start 2023-08-25T03:34:20.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.337+0000 7ff76348f700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff75c0b19a0 0x7ff75c15e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:20.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.337+0000 7ff76348f700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff75c0caea0 0x7ff75c0cb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:20.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.337+0000 7ff76348f700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff75c071b40 con 0x7ff75c0b1d80 2023-08-25T03:34:20.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.337+0000 7ff76348f700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75c0720b0 con 0x7ff75c0caea0 2023-08-25T03:34:20.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.337+0000 7ff76348f700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff75c071dd0 con 0x7ff75c0b19a0 2023-08-25T03:34:20.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.337+0000 7ff761a2c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff75c0caea0 0x7ff75c0cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:20.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.337+0000 7ff761a2c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff75c0caea0 0x7ff75c0cb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59866/0 (socket says 172.21.15.2:59866) 2023-08-25T03:34:20.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.337+0000 7ff761a2c700 1 -- 172.21.15.2:0/120266389 learned_addr learned my addr 172.21.15.2:0/120266389 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:20.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.337+0000 7ff760a2a700 1 --1- 172.21.15.2:0/120266389 >> v1:172.21.15.2:6789/0 conn(0x7ff75c0b1d80 0x7ff75c071430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46528/0 (socket says 172.21.15.2:46528) 2023-08-25T03:34:20.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.337+0000 7ff76122b700 1 --2- 172.21.15.2:0/120266389 >> v2:172.21.15.31:3300/0 conn(0x7ff75c0b19a0 0x7ff75c15e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:20.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.337+0000 7ff761a2c700 1 -- 172.21.15.2:0/120266389 >> v2:172.21.15.31:3300/0 conn(0x7ff75c0b19a0 msgr2=0x7ff75c15e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:20.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.337+0000 7ff761a2c700 1 --2- 172.21.15.2:0/120266389 >> v2:172.21.15.31:3300/0 conn(0x7ff75c0b19a0 0x7ff75c15e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:20.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.337+0000 7ff761a2c700 1 -- 172.21.15.2:0/120266389 >> v1:172.21.15.2:6789/0 conn(0x7ff75c0b1d80 legacy=0x7ff75c071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:20.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.337+0000 7ff753fff700 1 -- 172.21.15.2:0/120266389 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2788054623 0 0) 0x7ff75c071b40 con 0x7ff75c0b1d80 2023-08-25T03:34:20.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.337+0000 7ff761a2c700 1 -- 172.21.15.2:0/120266389 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff75c15ddb0 con 0x7ff75c0caea0 2023-08-25T03:34:20.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.337+0000 7ff76122b700 1 --2- 172.21.15.2:0/120266389 >> v2:172.21.15.31:3300/0 conn(0x7ff75c0b19a0 0x7ff75c15e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:20.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.337+0000 7ff761a2c700 1 --2- 172.21.15.2:0/120266389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff75c0caea0 0x7ff75c0cb330 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff75c06c390 tx=0x7ff75400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=552c015a236acffa server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:20.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.337+0000 7ff753fff700 1 -- 172.21.15.2:0/120266389 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75400e070 con 0x7ff75c0caea0 2023-08-25T03:34:20.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.337+0000 7ff753fff700 1 -- 172.21.15.2:0/120266389 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff75400b9c0 con 0x7ff75c0caea0 2023-08-25T03:34:20.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.337+0000 7ff753fff700 1 -- 172.21.15.2:0/120266389 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff75400bd70 con 0x7ff75c0caea0 2023-08-25T03:34:20.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.338+0000 7ff76348f700 1 -- 172.21.15.2:0/120266389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff75c0caea0 msgr2=0x7ff75c0cb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:20.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.338+0000 7ff76348f700 1 --2- 172.21.15.2:0/120266389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff75c0caea0 0x7ff75c0cb330 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff75c06c390 tx=0x7ff75400b330 comp rx=0 tx=0).stop 2023-08-25T03:34:20.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.338+0000 7ff76348f700 1 -- 172.21.15.2:0/120266389 shutdown_connections 2023-08-25T03:34:20.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.338+0000 7ff76348f700 1 --2- 172.21.15.2:0/120266389 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff75c0caea0 0x7ff75c0cb330 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:20.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.338+0000 7ff76348f700 1 --2- 172.21.15.2:0/120266389 >> v2:172.21.15.31:3300/0 conn(0x7ff75c0b19a0 0x7ff75c15e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:20.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.338+0000 7ff76348f700 1 -- 172.21.15.2:0/120266389 >> 172.21.15.2:0/120266389 conn(0x7ff75c0cfea0 msgr2=0x7ff75c0d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:20.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.338+0000 7ff76348f700 1 -- 172.21.15.2:0/120266389 shutdown_connections 2023-08-25T03:34:20.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.338+0000 7ff76348f700 1 -- 172.21.15.2:0/120266389 wait complete. 2023-08-25T03:34:20.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.339+0000 7ff76348f700 1 Processor -- start 2023-08-25T03:34:20.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.339+0000 7ff76348f700 1 -- start start 2023-08-25T03:34:20.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.340+0000 7ff76348f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff75c0b19a0 0x7ff75c144fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:20.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.340+0000 7ff76348f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff75c0b1d80 0x7ff75c0c26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:20.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.340+0000 7ff76348f700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff75c0caea0 0x7ff75c143640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:20.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.340+0000 7ff76348f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c0c2c00 con 0x7ff75c0b1d80 2023-08-25T03:34:20.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.340+0000 7ff76348f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c143f40 con 0x7ff75c0b19a0 2023-08-25T03:34:20.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.340+0000 7ff76348f700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75c144220 con 0x7ff75c0caea0 2023-08-25T03:34:20.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.340+0000 7ff760a2a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff75c0b1d80 0x7ff75c0c26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:20.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.340+0000 7ff760a2a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff75c0b1d80 0x7ff75c0c26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59870/0 (socket says 172.21.15.2:59870) 2023-08-25T03:34:20.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.340+0000 7ff760a2a700 1 -- 172.21.15.2:0/2925572763 learned_addr learned my addr 172.21.15.2:0/2925572763 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:20.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.340+0000 7ff76122b700 1 --2- 172.21.15.2:0/2925572763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff75c0b19a0 0x7ff75c144fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:20.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.340+0000 7ff761a2c700 1 --2- 172.21.15.2:0/2925572763 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff75c0caea0 0x7ff75c143640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:20.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.340+0000 7ff760a2a700 1 -- 172.21.15.2:0/2925572763 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff75c0caea0 msgr2=0x7ff75c143640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:20.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.340+0000 7ff760a2a700 1 --2- 172.21.15.2:0/2925572763 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff75c0caea0 0x7ff75c143640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:20.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.340+0000 7ff760a2a700 1 -- 172.21.15.2:0/2925572763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff75c0b19a0 msgr2=0x7ff75c144fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:20.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.340+0000 7ff760a2a700 1 --2- 172.21.15.2:0/2925572763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff75c0b19a0 0x7ff75c144fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:20.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.340+0000 7ff760a2a700 1 -- 172.21.15.2:0/2925572763 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff754009b30 con 0x7ff75c0b1d80 2023-08-25T03:34:20.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.340+0000 7ff760a2a700 1 --2- 172.21.15.2:0/2925572763 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff75c0b1d80 0x7ff75c0c26c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0011e0 tx=0x7ff74c00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:20.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.341+0000 7ff751ffb700 1 -- 172.21.15.2:0/2925572763 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c004c80 con 0x7ff75c0b1d80 2023-08-25T03:34:20.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.341+0000 7ff751ffb700 1 -- 172.21.15.2:0/2925572763 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74c004e00 con 0x7ff75c0b1d80 2023-08-25T03:34:20.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.341+0000 7ff76348f700 1 -- 172.21.15.2:0/2925572763 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff75c144500 con 0x7ff75c0b1d80 2023-08-25T03:34:20.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.341+0000 7ff751ffb700 1 -- 172.21.15.2:0/2925572763 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74c007310 con 0x7ff75c0b1d80 2023-08-25T03:34:20.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.341+0000 7ff76348f700 1 -- 172.21.15.2:0/2925572763 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75c144ac0 con 0x7ff75c0b1d80 2023-08-25T03:34:20.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.342+0000 7ff76348f700 1 -- 172.21.15.2:0/2925572763 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff75c066f70 con 0x7ff75c0b1d80 2023-08-25T03:34:20.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.342+0000 7ff751ffb700 1 -- 172.21.15.2:0/2925572763 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff74c007470 con 0x7ff75c0b1d80 2023-08-25T03:34:20.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.342+0000 7ff751ffb700 1 --2- 172.21.15.2:0/2925572763 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff74006a0e0 0x7ff74006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:20.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.342+0000 7ff751ffb700 1 -- 172.21.15.2:0/2925572763 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff74c0882a0 con 0x7ff75c0b1d80 2023-08-25T03:34:20.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.343+0000 7ff76122b700 1 --2- 172.21.15.2:0/2925572763 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff74006a0e0 0x7ff74006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:20.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.343+0000 7ff76122b700 1 --2- 172.21.15.2:0/2925572763 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff74006a0e0 0x7ff74006c5a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff75800c900 tx=0x7ff758008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:20.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.347+0000 7ff751ffb700 1 -- 172.21.15.2:0/2925572763 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff74c053990 con 0x7ff75c0b1d80 2023-08-25T03:34:20.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.548+0000 7ff76348f700 1 -- 172.21.15.2:0/2925572763 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff75c144ce0 con 0x7ff75c0b1d80 2023-08-25T03:34:20.550 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.550+0000 7ff751ffb700 1 -- 172.21.15.2:0/2925572763 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18135 (secure 0 0 0) 0x7ff74c056fb0 con 0x7ff75c0b1d80 2023-08-25T03:34:20.551 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:20.551 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":24,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:34:04.604059+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:34:00.869118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T03:34:01.987016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:34:20.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.552+0000 7ff76348f700 1 -- 172.21.15.2:0/2925572763 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff74006a0e0 msgr2=0x7ff74006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:20.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.552+0000 7ff76348f700 1 --2- 172.21.15.2:0/2925572763 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff74006a0e0 0x7ff74006c5a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff75800c900 tx=0x7ff758008040 comp rx=0 tx=0).stop 2023-08-25T03:34:20.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.552+0000 7ff76348f700 1 -- 172.21.15.2:0/2925572763 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff75c0b1d80 msgr2=0x7ff75c0c26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:20.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.552+0000 7ff76348f700 1 --2- 172.21.15.2:0/2925572763 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff75c0b1d80 0x7ff75c0c26c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0011e0 tx=0x7ff74c00cae0 comp rx=0 tx=0).stop 2023-08-25T03:34:20.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.552+0000 7ff76348f700 1 -- 172.21.15.2:0/2925572763 shutdown_connections 2023-08-25T03:34:20.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.552+0000 7ff76348f700 1 --2- 172.21.15.2:0/2925572763 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff75c0caea0 0x7ff75c143640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:20.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.552+0000 7ff76348f700 1 --2- 172.21.15.2:0/2925572763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff75c0b19a0 0x7ff75c144fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:20.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.552+0000 7ff76348f700 1 --2- 172.21.15.2:0/2925572763 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff74006a0e0 0x7ff74006c5a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:20.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.552+0000 7ff76348f700 1 --2- 172.21.15.2:0/2925572763 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff75c0b1d80 0x7ff75c0c26c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:20.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.552+0000 7ff76348f700 1 -- 172.21.15.2:0/2925572763 >> 172.21.15.2:0/2925572763 conn(0x7ff75c0cfea0 msgr2=0x7ff75c06a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:20.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.552+0000 7ff76348f700 1 -- 172.21.15.2:0/2925572763 shutdown_connections 2023-08-25T03:34:20.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.552+0000 7ff76348f700 1 -- 172.21.15.2:0/2925572763 wait complete. 2023-08-25T03:34:20.566 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T03:33:49.560643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T03:33:56.265184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T03:34:00.869118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T03:34:01.987016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T03:34:20.566 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T03:34:20.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.693+0000 7ff979229700 1 Processor -- start 2023-08-25T03:34:20.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.693+0000 7ff979229700 1 -- start start 2023-08-25T03:34:20.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.694+0000 7ff979229700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff97406ab00 0x7ff974160f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:20.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.694+0000 7ff979229700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff974072370 0x7ff9741614b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:20.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.694+0000 7ff979229700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9740b0f80 con 0x7ff9740ae6b0 2023-08-25T03:34:20.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.694+0000 7ff979229700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9740b1b00 con 0x7ff97406ab00 2023-08-25T03:34:20.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.694+0000 7ff979229700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff97415e990 con 0x7ff974072370 2023-08-25T03:34:20.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.694+0000 7ff96a59c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff97406ab00 0x7ff974160f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:20.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.694+0000 7ff972d9d700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7ff9740ae6b0 0x7ff97406a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46542/0 (socket says 172.21.15.2:46542) 2023-08-25T03:34:20.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.694+0000 7ff972d9d700 1 -- 172.21.15.2:0/4154722417 learned_addr learned my addr 172.21.15.2:0/4154722417 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:20.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.694+0000 7ff97359e700 1 --2- 172.21.15.2:0/4154722417 >> v2:172.21.15.31:3300/0 conn(0x7ff974072370 0x7ff9741614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:20.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.694+0000 7ff97259c700 1 -- 172.21.15.2:0/4154722417 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1287864794 0 0) 0x7ff9740b0f80 con 0x7ff9740ae6b0 2023-08-25T03:34:20.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.694+0000 7ff97259c700 1 -- 172.21.15.2:0/4154722417 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9600035b0 con 0x7ff9740ae6b0 2023-08-25T03:34:20.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.694+0000 7ff96a59c700 1 -- 172.21.15.2:0/4154722417 >> v2:172.21.15.31:3300/0 conn(0x7ff974072370 msgr2=0x7ff9741614b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:20.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.694+0000 7ff96a59c700 1 --2- 172.21.15.2:0/4154722417 >> v2:172.21.15.31:3300/0 conn(0x7ff974072370 0x7ff9741614b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:20.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.694+0000 7ff96a59c700 1 -- 172.21.15.2:0/4154722417 >> v1:172.21.15.2:6789/0 conn(0x7ff9740ae6b0 legacy=0x7ff97406a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:20.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.694+0000 7ff96a59c700 1 -- 172.21.15.2:0/4154722417 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9740b11d0 con 0x7ff97406ab00 2023-08-25T03:34:20.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.694+0000 7ff97359e700 1 --2- 172.21.15.2:0/4154722417 >> v2:172.21.15.31:3300/0 conn(0x7ff974072370 0x7ff9741614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:20.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.695+0000 7ff96a59c700 1 --2- 172.21.15.2:0/4154722417 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff97406ab00 0x7ff974160f70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff974071370 tx=0x7ff95c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa2c3199ce634af server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:20.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.695+0000 7ff97259c700 1 -- 172.21.15.2:0/4154722417 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c011830 con 0x7ff97406ab00 2023-08-25T03:34:20.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.695+0000 7ff97259c700 1 -- 172.21.15.2:0/4154722417 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff95c011990 con 0x7ff97406ab00 2023-08-25T03:34:20.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.695+0000 7ff97259c700 1 -- 172.21.15.2:0/4154722417 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff95c011d20 con 0x7ff97406ab00 2023-08-25T03:34:20.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.695+0000 7ff979229700 1 -- 172.21.15.2:0/4154722417 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff97406ab00 msgr2=0x7ff974160f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:20.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.695+0000 7ff979229700 1 --2- 172.21.15.2:0/4154722417 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff97406ab00 0x7ff974160f70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff974071370 tx=0x7ff95c009b20 comp rx=0 tx=0).stop 2023-08-25T03:34:20.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.695+0000 7ff979229700 1 -- 172.21.15.2:0/4154722417 shutdown_connections 2023-08-25T03:34:20.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.695+0000 7ff979229700 1 --2- 172.21.15.2:0/4154722417 >> v2:172.21.15.31:3300/0 conn(0x7ff974072370 0x7ff9741614b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:20.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.695+0000 7ff979229700 1 --2- 172.21.15.2:0/4154722417 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff97406ab00 0x7ff974160f70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:20.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.695+0000 7ff979229700 1 -- 172.21.15.2:0/4154722417 >> 172.21.15.2:0/4154722417 conn(0x7ff9740cf500 msgr2=0x7ff9740cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:20.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.695+0000 7ff979229700 1 -- 172.21.15.2:0/4154722417 shutdown_connections 2023-08-25T03:34:20.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.695+0000 7ff979229700 1 -- 172.21.15.2:0/4154722417 wait complete. 2023-08-25T03:34:20.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.697+0000 7ff979229700 1 Processor -- start 2023-08-25T03:34:20.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.697+0000 7ff979229700 1 -- start start 2023-08-25T03:34:20.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.697+0000 7ff979229700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff97406ab00 0x7ff9740cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:20.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.697+0000 7ff979229700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff974072370 0x7ff974156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:20.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.697+0000 7ff979229700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9740ae6b0 0x7ff974154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:20.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.697+0000 7ff979229700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff974156d90 con 0x7ff97406ab00 2023-08-25T03:34:20.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.697+0000 7ff979229700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9740ce0e0 con 0x7ff9740ae6b0 2023-08-25T03:34:20.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.697+0000 7ff979229700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9740ce3c0 con 0x7ff974072370 2023-08-25T03:34:20.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.698+0000 7ff972d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff97406ab00 0x7ff9740cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:20.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.698+0000 7ff972d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff97406ab00 0x7ff9740cda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59882/0 (socket says 172.21.15.2:59882) 2023-08-25T03:34:20.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.698+0000 7ff972d9d700 1 -- 172.21.15.2:0/2708827784 learned_addr learned my addr 172.21.15.2:0/2708827784 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:20.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.698+0000 7ff96a59c700 1 --2- 172.21.15.2:0/2708827784 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff974072370 0x7ff974156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:20.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.698+0000 7ff97359e700 1 --2- 172.21.15.2:0/2708827784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9740ae6b0 0x7ff974154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:20.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.698+0000 7ff972d9d700 1 -- 172.21.15.2:0/2708827784 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff974072370 msgr2=0x7ff974156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:20.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.698+0000 7ff972d9d700 1 --2- 172.21.15.2:0/2708827784 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff974072370 0x7ff974156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:20.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.698+0000 7ff972d9d700 1 -- 172.21.15.2:0/2708827784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9740ae6b0 msgr2=0x7ff974154300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:20.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.698+0000 7ff972d9d700 1 --2- 172.21.15.2:0/2708827784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9740ae6b0 0x7ff974154300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:20.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.698+0000 7ff972d9d700 1 -- 172.21.15.2:0/2708827784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95c0094a0 con 0x7ff97406ab00 2023-08-25T03:34:20.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.698+0000 7ff96a59c700 1 --2- 172.21.15.2:0/2708827784 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff974072370 0x7ff974156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:20.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.698+0000 7ff97359e700 1 --2- 172.21.15.2:0/2708827784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9740ae6b0 0x7ff974154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:20.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.698+0000 7ff972d9d700 1 --2- 172.21.15.2:0/2708827784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff97406ab00 0x7ff9740cda50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff964001260 tx=0x7ff96400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:20.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.698+0000 7ff96bfff700 1 -- 172.21.15.2:0/2708827784 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9640074e0 con 0x7ff97406ab00 2023-08-25T03:34:20.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.698+0000 7ff96bfff700 1 -- 172.21.15.2:0/2708827784 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff964007640 con 0x7ff97406ab00 2023-08-25T03:34:20.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.698+0000 7ff979229700 1 -- 172.21.15.2:0/2708827784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9740ce6a0 con 0x7ff97406ab00 2023-08-25T03:34:20.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.698+0000 7ff96bfff700 1 -- 172.21.15.2:0/2708827784 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff964005a50 con 0x7ff97406ab00 2023-08-25T03:34:20.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.698+0000 7ff979229700 1 -- 172.21.15.2:0/2708827784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9740cec60 con 0x7ff97406ab00 2023-08-25T03:34:20.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.699+0000 7ff979229700 1 -- 172.21.15.2:0/2708827784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9740746d0 con 0x7ff97406ab00 2023-08-25T03:34:20.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.700+0000 7ff96bfff700 1 -- 172.21.15.2:0/2708827784 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff964005bb0 con 0x7ff97406ab00 2023-08-25T03:34:20.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.700+0000 7ff96bfff700 1 --2- 172.21.15.2:0/2708827784 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff95406a0e0 0x7ff95406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:20.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.700+0000 7ff96bfff700 1 -- 172.21.15.2:0/2708827784 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff964087ae0 con 0x7ff97406ab00 2023-08-25T03:34:20.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.700+0000 7ff96a59c700 1 --2- 172.21.15.2:0/2708827784 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff95406a0e0 0x7ff95406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:20.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.701+0000 7ff96a59c700 1 --2- 172.21.15.2:0/2708827784 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff95406a0e0 0x7ff95406c5a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff974157b00 tx=0x7ff95c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:20.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.705+0000 7ff96bfff700 1 -- 172.21.15.2:0/2708827784 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff9640531d0 con 0x7ff97406ab00 2023-08-25T03:34:20.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.900+0000 7ff979229700 1 -- 172.21.15.2:0/2708827784 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff97406ce10 con 0x7ff97406ab00 2023-08-25T03:34:20.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.901+0000 7ff96bfff700 1 -- 172.21.15.2:0/2708827784 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff9640170a0 con 0x7ff97406ab00 2023-08-25T03:34:20.902 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-08-25T03:34:20.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.903+0000 7ff979229700 1 -- 172.21.15.2:0/2708827784 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff95406a0e0 msgr2=0x7ff95406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:20.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.903+0000 7ff979229700 1 --2- 172.21.15.2:0/2708827784 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff95406a0e0 0x7ff95406c5a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff974157b00 tx=0x7ff95c008030 comp rx=0 tx=0).stop 2023-08-25T03:34:20.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.903+0000 7ff979229700 1 -- 172.21.15.2:0/2708827784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff97406ab00 msgr2=0x7ff9740cda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:20.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.903+0000 7ff979229700 1 --2- 172.21.15.2:0/2708827784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff97406ab00 0x7ff9740cda50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff964001260 tx=0x7ff96400c860 comp rx=0 tx=0).stop 2023-08-25T03:34:20.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.903+0000 7ff979229700 1 -- 172.21.15.2:0/2708827784 shutdown_connections 2023-08-25T03:34:20.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.903+0000 7ff979229700 1 --2- 172.21.15.2:0/2708827784 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff95406a0e0 0x7ff95406c5a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:20.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.903+0000 7ff979229700 1 --2- 172.21.15.2:0/2708827784 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff97406ab00 0x7ff9740cda50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:20.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.903+0000 7ff979229700 1 --2- 172.21.15.2:0/2708827784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9740ae6b0 0x7ff974154300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:20.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.903+0000 7ff979229700 1 --2- 172.21.15.2:0/2708827784 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff974072370 0x7ff974156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:20.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.903+0000 7ff979229700 1 -- 172.21.15.2:0/2708827784 >> 172.21.15.2:0/2708827784 conn(0x7ff9740cf500 msgr2=0x7ff974160360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:20.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.903+0000 7ff979229700 1 -- 172.21.15.2:0/2708827784 shutdown_connections 2023-08-25T03:34:20.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:20.903+0000 7ff979229700 1 -- 172.21.15.2:0/2708827784 wait complete. 2023-08-25T03:34:20.916 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T03:34:21.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.045+0000 7f5721907700 1 Processor -- start 2023-08-25T03:34:21.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.045+0000 7f5721907700 1 -- start start 2023-08-25T03:34:21.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.046+0000 7f5721907700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f571c0cfb80 0x7f571c0cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:21.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.046+0000 7f5721907700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f571c0d0000 con 0x7f571c15b170 2023-08-25T03:34:21.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.046+0000 7f5721907700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f571c0cf6f0 con 0x7f571c0cdd30 2023-08-25T03:34:21.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.046+0000 7f5721907700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571c0ce1c0 con 0x7f571c0cfb80 2023-08-25T03:34:21.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.046+0000 7f571affd700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f571c15b170 0x7f571c0cc640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46556/0 (socket says 172.21.15.2:46556) 2023-08-25T03:34:21.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.046+0000 7f571affd700 1 -- 172.21.15.2:0/95209601 learned_addr learned my addr 172.21.15.2:0/95209601 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:21.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.046+0000 7f571a7fc700 1 --2- 172.21.15.2:0/95209601 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f571c0cfb80 0x7f571c0cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:21.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.046+0000 7f5719ffb700 1 -- 172.21.15.2:0/95209601 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1345145732 0 0) 0x7f5710001290 con 0x7f571c15b170 2023-08-25T03:34:21.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.046+0000 7f5719ffb700 1 -- 172.21.15.2:0/95209601 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57040035b0 con 0x7f571c15b170 2023-08-25T03:34:21.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f5719ffb700 1 -- 172.21.15.2:0/95209601 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1624333739 0 0) 0x7f571c0cf6f0 con 0x7f571c0cdd30 2023-08-25T03:34:21.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f5719ffb700 1 -- 172.21.15.2:0/95209601 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5710001290 con 0x7f571c0cdd30 2023-08-25T03:34:21.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f5719ffb700 1 -- 172.21.15.2:0/95209601 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2870551290 0 0) 0x7f57040035b0 con 0x7f571c15b170 2023-08-25T03:34:21.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f5719ffb700 1 -- 172.21.15.2:0/95209601 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f571c0cfb80 msgr2=0x7f571c0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:21.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f5719ffb700 1 --2- 172.21.15.2:0/95209601 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f571c0cfb80 0x7f571c0cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f5719ffb700 1 -- 172.21.15.2:0/95209601 >> v1:172.21.15.31:6789/0 conn(0x7f571c0cdd30 legacy=0x7f571c06d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f5719ffb700 1 -- 172.21.15.2:0/95209601 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571c0d1d50 con 0x7f571c15b170 2023-08-25T03:34:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f5719ffb700 1 -- 172.21.15.2:0/95209601 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f5710001980 con 0x7f571c15b170 2023-08-25T03:34:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f5719ffb700 1 -- 172.21.15.2:0/95209601 >> v1:172.21.15.2:6789/0 conn(0x7f571c15b170 legacy=0x7f571c0cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f5719ffb700 1 --2- 172.21.15.2:0/95209601 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f571c0cfb80 0x7f5704003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f5719ffb700 1 --2- 172.21.15.2:0/95209601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5704004170 0x7f5704004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f5719ffb700 1 --2- 172.21.15.2:0/95209601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5704004d90 0x7f5704005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f5719ffb700 1 -- 172.21.15.2:0/95209601 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c0ce1c0 con 0x7f5704004d90 2023-08-25T03:34:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f5719ffb700 1 -- 172.21.15.2:0/95209601 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570400bb50 con 0x7f5704004170 2023-08-25T03:34:21.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f5719ffb700 1 -- 172.21.15.2:0/95209601 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570400bd10 con 0x7f571c0cfb80 2023-08-25T03:34:21.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f571b7fe700 1 --2- 172.21.15.2:0/95209601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5704004d90 0x7f5704005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:21.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f571affd700 1 --2- 172.21.15.2:0/95209601 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f571c0cfb80 0x7f5704003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:21.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f571a7fc700 1 --2- 172.21.15.2:0/95209601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5704004170 0x7f5704004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:21.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f571b7fe700 1 -- 172.21.15.2:0/95209601 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f571c0cfb80 msgr2=0x7f5704003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:21.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f571b7fe700 1 --2- 172.21.15.2:0/95209601 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f571c0cfb80 0x7f5704003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f571b7fe700 1 -- 172.21.15.2:0/95209601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5704004170 msgr2=0x7f5704004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f571b7fe700 1 --2- 172.21.15.2:0/95209601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5704004170 0x7f5704004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f571b7fe700 1 -- 172.21.15.2:0/95209601 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57040035b0 con 0x7f5704004d90 2023-08-25T03:34:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f571b7fe700 1 --2- 172.21.15.2:0/95209601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5704004d90 0x7f5704005210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f571400c2f0 tx=0x7f5704005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6bd2db1811b738 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f5719ffb700 1 -- 172.21.15.2:0/95209601 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5714001f30 con 0x7f5704004d90 2023-08-25T03:34:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f5719ffb700 1 -- 172.21.15.2:0/95209601 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571400ce90 con 0x7f5704004d90 2023-08-25T03:34:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.047+0000 7f5719ffb700 1 -- 172.21.15.2:0/95209601 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5714003c40 con 0x7f5704004d90 2023-08-25T03:34:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.048+0000 7f5721907700 1 -- 172.21.15.2:0/95209601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5704004d90 msgr2=0x7f5704005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.048+0000 7f5721907700 1 --2- 172.21.15.2:0/95209601 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5704004d90 0x7f5704005210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f571400c2f0 tx=0x7f5704005880 comp rx=0 tx=0).stop 2023-08-25T03:34:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.048+0000 7f571b7fe700 1 -- 172.21.15.2:0/95209601 reap_dead start 2023-08-25T03:34:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.048+0000 7f5721907700 1 -- 172.21.15.2:0/95209601 shutdown_connections 2023-08-25T03:34:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.048+0000 7f5721907700 1 -- 172.21.15.2:0/95209601 >> 172.21.15.2:0/95209601 conn(0x7f571c09a200 msgr2=0x7f571c0d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.048+0000 7f5721907700 1 -- 172.21.15.2:0/95209601 shutdown_connections 2023-08-25T03:34:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.048+0000 7f5721907700 1 -- 172.21.15.2:0/95209601 wait complete. 2023-08-25T03:34:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.049+0000 7f5721907700 1 Processor -- start 2023-08-25T03:34:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.049+0000 7f5721907700 1 -- start start 2023-08-25T03:34:21.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.050+0000 7f5721907700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5704004d90 0x7f571c15c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:21.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.050+0000 7f5721907700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f571c06c0f0 0x7f571c15ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:21.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.050+0000 7f5721907700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f571c15cfb0 0x7f571c0b1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:21.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.050+0000 7f5721907700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c15c0a0 con 0x7f571c15cfb0 2023-08-25T03:34:21.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.050+0000 7f5721907700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f571c1f4000 con 0x7f571c06c0f0 2023-08-25T03:34:21.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.050+0000 7f5721907700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f571c0b3260 con 0x7f5704004d90 2023-08-25T03:34:21.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.050+0000 7f571b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f571c15cfb0 0x7f571c0b1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:21.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.050+0000 7f571b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f571c15cfb0 0x7f571c0b1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59906/0 (socket says 172.21.15.2:59906) 2023-08-25T03:34:21.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.050+0000 7f571b7fe700 1 -- 172.21.15.2:0/1031553935 learned_addr learned my addr 172.21.15.2:0/1031553935 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:21.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.050+0000 7f571affd700 1 --2- 172.21.15.2:0/1031553935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5704004d90 0x7f571c15c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:21.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.050+0000 7f571a7fc700 1 --2- 172.21.15.2:0/1031553935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f571c06c0f0 0x7f571c15ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:21.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.051+0000 7f571b7fe700 1 -- 172.21.15.2:0/1031553935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5704004d90 msgr2=0x7f571c15c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.051+0000 7f571b7fe700 1 --2- 172.21.15.2:0/1031553935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5704004d90 0x7f571c15c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.051+0000 7f571b7fe700 1 -- 172.21.15.2:0/1031553935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f571c06c0f0 msgr2=0x7f571c15ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.051+0000 7f571b7fe700 1 --2- 172.21.15.2:0/1031553935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f571c06c0f0 0x7f571c15ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.051+0000 7f571b7fe700 1 -- 172.21.15.2:0/1031553935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f571c15b770 con 0x7f571c15cfb0 2023-08-25T03:34:21.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.051+0000 7f571b7fe700 1 --2- 172.21.15.2:0/1031553935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f571c15cfb0 0x7f571c0b1920 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5714004000 tx=0x7f5714001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.051+0000 7f5703fff700 1 -- 172.21.15.2:0/1031553935 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571400cc90 con 0x7f571c15cfb0 2023-08-25T03:34:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.051+0000 7f5703fff700 1 -- 172.21.15.2:0/1031553935 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57140050d0 con 0x7f571c15cfb0 2023-08-25T03:34:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.051+0000 7f5703fff700 1 -- 172.21.15.2:0/1031553935 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5714005480 con 0x7f571c15cfb0 2023-08-25T03:34:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.051+0000 7f5721907700 1 -- 172.21.15.2:0/1031553935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f571c15ba50 con 0x7f571c15cfb0 2023-08-25T03:34:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.051+0000 7f5721907700 1 -- 172.21.15.2:0/1031553935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f571c15d440 con 0x7f571c15cfb0 2023-08-25T03:34:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.052+0000 7f5721907700 1 -- 172.21.15.2:0/1031553935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f571c15d600 con 0x7f571c15cfb0 2023-08-25T03:34:21.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.052+0000 7f5703fff700 1 -- 172.21.15.2:0/1031553935 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5714019070 con 0x7f571c15cfb0 2023-08-25T03:34:21.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.053+0000 7f5703fff700 1 --2- 172.21.15.2:0/1031553935 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f570806a110 0x7f570806c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:21.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.053+0000 7f5703fff700 1 -- 172.21.15.2:0/1031553935 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f57140894a0 con 0x7f571c15cfb0 2023-08-25T03:34:21.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.053+0000 7f571affd700 1 --2- 172.21.15.2:0/1031553935 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f570806a110 0x7f570806c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:21.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.053+0000 7f571affd700 1 --2- 172.21.15.2:0/1031553935 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f570806a110 0x7f570806c5d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5710000c00 tx=0x7f5710009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:21.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.057+0000 7f5703fff700 1 -- 172.21.15.2:0/1031553935 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5714054c20 con 0x7f571c15cfb0 2023-08-25T03:34:21.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.258+0000 7f5721907700 1 -- 172.21.15.2:0/1031553935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f571c0746d0 con 0x7f571c15cfb0 2023-08-25T03:34:21.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.259+0000 7f5703fff700 1 -- 172.21.15.2:0/1031553935 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f57140011c0 con 0x7f571c15cfb0 2023-08-25T03:34:21.260 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 8 2023-08-25T03:34:21.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.261+0000 7f5721907700 1 -- 172.21.15.2:0/1031553935 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f570806a110 msgr2=0x7f570806c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:21.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.261+0000 7f5721907700 1 --2- 172.21.15.2:0/1031553935 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f570806a110 0x7f570806c5d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f5710000c00 tx=0x7f5710009040 comp rx=0 tx=0).stop 2023-08-25T03:34:21.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.261+0000 7f5721907700 1 -- 172.21.15.2:0/1031553935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f571c15cfb0 msgr2=0x7f571c0b1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:21.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.261+0000 7f5721907700 1 --2- 172.21.15.2:0/1031553935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f571c15cfb0 0x7f571c0b1920 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5714004000 tx=0x7f5714001650 comp rx=0 tx=0).stop 2023-08-25T03:34:21.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.261+0000 7f5721907700 1 -- 172.21.15.2:0/1031553935 shutdown_connections 2023-08-25T03:34:21.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.261+0000 7f5721907700 1 --2- 172.21.15.2:0/1031553935 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f570806a110 0x7f570806c5d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.261+0000 7f5721907700 1 --2- 172.21.15.2:0/1031553935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f571c15cfb0 0x7f571c0b1920 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.261+0000 7f5721907700 1 --2- 172.21.15.2:0/1031553935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5704004d90 0x7f571c15c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.261+0000 7f5721907700 1 --2- 172.21.15.2:0/1031553935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f571c06c0f0 0x7f571c15ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.261+0000 7f5721907700 1 -- 172.21.15.2:0/1031553935 >> 172.21.15.2:0/1031553935 conn(0x7f571c09a200 msgr2=0x7f571c06e100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:21.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.261+0000 7f5721907700 1 -- 172.21.15.2:0/1031553935 shutdown_connections 2023-08-25T03:34:21.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.261+0000 7f5721907700 1 -- 172.21.15.2:0/1031553935 wait complete. 2023-08-25T03:34:21.275 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T03:34:21.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.404+0000 7ff33f891700 1 Processor -- start 2023-08-25T03:34:21.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.404+0000 7ff33f891700 1 -- start start 2023-08-25T03:34:21.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.405+0000 7ff33f891700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff3381607f0 0x7ff338070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:21.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.405+0000 7ff33f891700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff33806a1e0 0x7ff33806a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:21.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.405+0000 7ff33f891700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff338160bd0 con 0x7ff3380af180 2023-08-25T03:34:21.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.405+0000 7ff33f891700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff33806abd0 con 0x7ff3381607f0 2023-08-25T03:34:21.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.405+0000 7ff33f891700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3380713a0 con 0x7ff33806a1e0 2023-08-25T03:34:21.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.405+0000 7ff33d62d700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff3381607f0 0x7ff338070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:21.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.405+0000 7ff33d62d700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff3381607f0 0x7ff338070e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59910/0 (socket says 172.21.15.2:59910) 2023-08-25T03:34:21.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.405+0000 7ff33d62d700 1 -- 172.21.15.2:0/1998953348 learned_addr learned my addr 172.21.15.2:0/1998953348 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:21.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.405+0000 7ff33de2e700 1 --2- 172.21.15.2:0/1998953348 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff33806a1e0 0x7ff33806a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:21.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.405+0000 7ff33d62d700 1 -- 172.21.15.2:0/1998953348 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff33806a1e0 msgr2=0x7ff33806a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:21.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.405+0000 7ff33d62d700 1 --2- 172.21.15.2:0/1998953348 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff33806a1e0 0x7ff33806a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.405+0000 7ff33d62d700 1 -- 172.21.15.2:0/1998953348 >> v1:172.21.15.31:6789/0 conn(0x7ff3380af180 legacy=0x7ff3380ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:21.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.405+0000 7ff33d62d700 1 -- 172.21.15.2:0/1998953348 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3380cdd00 con 0x7ff3381607f0 2023-08-25T03:34:21.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.405+0000 7ff33d62d700 1 --2- 172.21.15.2:0/1998953348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3381607f0 0x7ff338070e60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff3340098f0 tx=0x7ff334009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69eaaca4342f884b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:21.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.405+0000 7ff32ffff700 1 -- 172.21.15.2:0/1998953348 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33400b8a0 con 0x7ff3381607f0 2023-08-25T03:34:21.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.406+0000 7ff32ffff700 1 -- 172.21.15.2:0/1998953348 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33400ba00 con 0x7ff3381607f0 2023-08-25T03:34:21.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.406+0000 7ff32ffff700 1 -- 172.21.15.2:0/1998953348 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33400bd90 con 0x7ff3381607f0 2023-08-25T03:34:21.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.406+0000 7ff33f891700 1 -- 172.21.15.2:0/1998953348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3381607f0 msgr2=0x7ff338070e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:21.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.406+0000 7ff33f891700 1 --2- 172.21.15.2:0/1998953348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3381607f0 0x7ff338070e60 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff3340098f0 tx=0x7ff334009c00 comp rx=0 tx=0).stop 2023-08-25T03:34:21.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.406+0000 7ff33f891700 1 -- 172.21.15.2:0/1998953348 shutdown_connections 2023-08-25T03:34:21.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.406+0000 7ff33f891700 1 --2- 172.21.15.2:0/1998953348 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff33806a1e0 0x7ff33806a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.406+0000 7ff33f891700 1 --2- 172.21.15.2:0/1998953348 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3381607f0 0x7ff338070e60 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.406+0000 7ff33f891700 1 -- 172.21.15.2:0/1998953348 >> 172.21.15.2:0/1998953348 conn(0x7ff3380cfd90 msgr2=0x7ff3380d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:21.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.406+0000 7ff33f891700 1 -- 172.21.15.2:0/1998953348 shutdown_connections 2023-08-25T03:34:21.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.406+0000 7ff33f891700 1 -- 172.21.15.2:0/1998953348 wait complete. 2023-08-25T03:34:21.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.407+0000 7ff33f891700 1 Processor -- start 2023-08-25T03:34:21.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.407+0000 7ff33f891700 1 -- start start 2023-08-25T03:34:21.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.408+0000 7ff33f891700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff33806a1e0 0x7ff3380ad3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:21.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.408+0000 7ff33f891700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff3380af180 0x7ff3380c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:21.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.408+0000 7ff33f891700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3381607f0 0x7ff3380c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:21.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.408+0000 7ff33f891700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3380c7de0 con 0x7ff33806a1e0 2023-08-25T03:34:21.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.408+0000 7ff33f891700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3380c78e0 con 0x7ff3381607f0 2023-08-25T03:34:21.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.408+0000 7ff33f891700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3380c7b00 con 0x7ff3380af180 2023-08-25T03:34:21.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.408+0000 7ff33d62d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff33806a1e0 0x7ff3380ad3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:21.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.408+0000 7ff33d62d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff33806a1e0 0x7ff3380ad3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59916/0 (socket says 172.21.15.2:59916) 2023-08-25T03:34:21.418 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.408+0000 7ff33d62d700 1 -- 172.21.15.2:0/74621719 learned_addr learned my addr 172.21.15.2:0/74621719 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:21.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.408+0000 7ff33de2e700 1 --2- 172.21.15.2:0/74621719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3381607f0 0x7ff3380c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:21.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.408+0000 7ff33ce2c700 1 --2- 172.21.15.2:0/74621719 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff3380af180 0x7ff3380c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:21.419 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.408+0000 7ff33d62d700 1 -- 172.21.15.2:0/74621719 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff3380af180 msgr2=0x7ff3380c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:21.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.408+0000 7ff33d62d700 1 --2- 172.21.15.2:0/74621719 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff3380af180 0x7ff3380c6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.408+0000 7ff33d62d700 1 -- 172.21.15.2:0/74621719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3381607f0 msgr2=0x7ff3380c73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:21.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.408+0000 7ff33d62d700 1 --2- 172.21.15.2:0/74621719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3381607f0 0x7ff3380c73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.420 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.408+0000 7ff33d62d700 1 -- 172.21.15.2:0/74621719 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff334009550 con 0x7ff33806a1e0 2023-08-25T03:34:21.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.409+0000 7ff33de2e700 1 --2- 172.21.15.2:0/74621719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3381607f0 0x7ff3380c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:21.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.409+0000 7ff33d62d700 1 --2- 172.21.15.2:0/74621719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff33806a1e0 0x7ff3380ad3e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff334014040 tx=0x7ff334004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:21.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.409+0000 7ff32dffb700 1 -- 172.21.15.2:0/74621719 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3340036a0 con 0x7ff33806a1e0 2023-08-25T03:34:21.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.409+0000 7ff32dffb700 1 -- 172.21.15.2:0/74621719 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff33401e070 con 0x7ff33806a1e0 2023-08-25T03:34:21.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.409+0000 7ff33f891700 1 -- 172.21.15.2:0/74621719 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff338146cf0 con 0x7ff33806a1e0 2023-08-25T03:34:21.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.409+0000 7ff32dffb700 1 -- 172.21.15.2:0/74621719 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff334003a60 con 0x7ff33806a1e0 2023-08-25T03:34:21.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.409+0000 7ff33f891700 1 -- 172.21.15.2:0/74621719 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3381472b0 con 0x7ff33806a1e0 2023-08-25T03:34:21.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.410+0000 7ff33f891700 1 -- 172.21.15.2:0/74621719 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff338146b50 con 0x7ff33806a1e0 2023-08-25T03:34:21.422 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.410+0000 7ff32dffb700 1 -- 172.21.15.2:0/74621719 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff334003bc0 con 0x7ff33806a1e0 2023-08-25T03:34:21.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.411+0000 7ff32dffb700 1 --2- 172.21.15.2:0/74621719 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff31c06a0e0 0x7ff31c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:21.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.411+0000 7ff32dffb700 1 -- 172.21.15.2:0/74621719 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff334020070 con 0x7ff33806a1e0 2023-08-25T03:34:21.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.411+0000 7ff33ce2c700 1 --2- 172.21.15.2:0/74621719 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff31c06a0e0 0x7ff31c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:21.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.411+0000 7ff33ce2c700 1 --2- 172.21.15.2:0/74621719 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff31c06a0e0 0x7ff31c06c5a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff3380c6e10 tx=0x7ff328009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:21.424 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.415+0000 7ff32dffb700 1 -- 172.21.15.2:0/74621719 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff338146b50 con 0x7ff33806a1e0 2023-08-25T03:34:21.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.614+0000 7ff33f891700 1 -- 172.21.15.2:0/74621719 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff338146b50 con 0x7ff33806a1e0 2023-08-25T03:34:21.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.615+0000 7ff32dffb700 1 -- 172.21.15.2:0/74621719 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff338146b50 con 0x7ff33806a1e0 2023-08-25T03:34:21.617 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-08-25T03:34:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.617+0000 7ff33f891700 1 -- 172.21.15.2:0/74621719 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff31c06a0e0 msgr2=0x7ff31c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.617+0000 7ff33f891700 1 --2- 172.21.15.2:0/74621719 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff31c06a0e0 0x7ff31c06c5a0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff3380c6e10 tx=0x7ff328009040 comp rx=0 tx=0).stop 2023-08-25T03:34:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.617+0000 7ff33f891700 1 -- 172.21.15.2:0/74621719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff33806a1e0 msgr2=0x7ff3380ad3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:21.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.617+0000 7ff33f891700 1 --2- 172.21.15.2:0/74621719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff33806a1e0 0x7ff3380ad3e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff334014040 tx=0x7ff334004750 comp rx=0 tx=0).stop 2023-08-25T03:34:21.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.617+0000 7ff33f891700 1 -- 172.21.15.2:0/74621719 shutdown_connections 2023-08-25T03:34:21.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.617+0000 7ff33f891700 1 --2- 172.21.15.2:0/74621719 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff31c06a0e0 0x7ff31c06c5a0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.617+0000 7ff33f891700 1 --2- 172.21.15.2:0/74621719 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff33806a1e0 0x7ff3380ad3e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.617+0000 7ff33f891700 1 --2- 172.21.15.2:0/74621719 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3381607f0 0x7ff3380c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.617+0000 7ff33f891700 1 --2- 172.21.15.2:0/74621719 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff3380af180 0x7ff3380c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.617+0000 7ff33f891700 1 -- 172.21.15.2:0/74621719 >> 172.21.15.2:0/74621719 conn(0x7ff3380cfd90 msgr2=0x7ff3380ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:21.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.617+0000 7ff33f891700 1 -- 172.21.15.2:0/74621719 shutdown_connections 2023-08-25T03:34:21.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.617+0000 7ff33f891700 1 -- 172.21.15.2:0/74621719 wait complete. 2023-08-25T03:34:21.630 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T03:34:21.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.761+0000 7f93bd926700 1 Processor -- start 2023-08-25T03:34:21.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.761+0000 7f93bd926700 1 -- start start 2023-08-25T03:34:21.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.763+0000 7f93bd926700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f93b8159f10 0x7f93b815dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:21.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.763+0000 7f93bd926700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f93b80cedf0 0x7f93b806ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:21.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.763+0000 7f93bd926700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b806eaf0 con 0x7f93b806f4c0 2023-08-25T03:34:21.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.763+0000 7f93bd926700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b806e850 con 0x7f93b8159f10 2023-08-25T03:34:21.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.763+0000 7f93bd926700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b815e020 con 0x7f93b80cedf0 2023-08-25T03:34:21.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.763+0000 7f93b6ffd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f93b8159f10 0x7f93b815dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:21.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.763+0000 7f93b6ffd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f93b8159f10 0x7f93b815dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59924/0 (socket says 172.21.15.2:59924) 2023-08-25T03:34:21.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.763+0000 7f93b6ffd700 1 -- 172.21.15.2:0/2269257347 learned_addr learned my addr 172.21.15.2:0/2269257347 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:21.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.763+0000 7f93b67fc700 1 --2- 172.21.15.2:0/2269257347 >> v2:172.21.15.31:3300/0 conn(0x7f93b80cedf0 0x7f93b806ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:21.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.763+0000 7f93b6ffd700 1 -- 172.21.15.2:0/2269257347 >> v2:172.21.15.31:3300/0 conn(0x7f93b80cedf0 msgr2=0x7f93b806ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:21.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.763+0000 7f93b6ffd700 1 --2- 172.21.15.2:0/2269257347 >> v2:172.21.15.31:3300/0 conn(0x7f93b80cedf0 0x7f93b806ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.763+0000 7f93b6ffd700 1 -- 172.21.15.2:0/2269257347 >> v1:172.21.15.31:6789/0 conn(0x7f93b806f4c0 legacy=0x7f93b80cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:21.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.763+0000 7f93b6ffd700 1 -- 172.21.15.2:0/2269257347 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b815e2c0 con 0x7f93b8159f10 2023-08-25T03:34:21.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.763+0000 7f93b6ffd700 1 --2- 172.21.15.2:0/2269257347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93b8159f10 0x7f93b815dae0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f93b8072300 tx=0x7f93b0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67683151355c66ba server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:21.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.763+0000 7f93b5ffb700 1 -- 172.21.15.2:0/2269257347 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0004230 con 0x7f93b8159f10 2023-08-25T03:34:21.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.763+0000 7f93b5ffb700 1 -- 172.21.15.2:0/2269257347 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b0004390 con 0x7f93b8159f10 2023-08-25T03:34:21.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.763+0000 7f93b5ffb700 1 -- 172.21.15.2:0/2269257347 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b00036a0 con 0x7f93b8159f10 2023-08-25T03:34:21.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.764+0000 7f93bd926700 1 -- 172.21.15.2:0/2269257347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93b8159f10 msgr2=0x7f93b815dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:21.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.764+0000 7f93bd926700 1 --2- 172.21.15.2:0/2269257347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93b8159f10 0x7f93b815dae0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f93b8072300 tx=0x7f93b0009bd0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.764+0000 7f93bd926700 1 -- 172.21.15.2:0/2269257347 shutdown_connections 2023-08-25T03:34:21.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.764+0000 7f93bd926700 1 --2- 172.21.15.2:0/2269257347 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93b8159f10 0x7f93b815dae0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.764+0000 7f93bd926700 1 --2- 172.21.15.2:0/2269257347 >> v2:172.21.15.31:3300/0 conn(0x7f93b80cedf0 0x7f93b806ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.764+0000 7f93bd926700 1 -- 172.21.15.2:0/2269257347 >> 172.21.15.2:0/2269257347 conn(0x7f93b80cfd90 msgr2=0x7f93b80d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:21.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.764+0000 7f93bd926700 1 -- 172.21.15.2:0/2269257347 shutdown_connections 2023-08-25T03:34:21.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.764+0000 7f93bd926700 1 -- 172.21.15.2:0/2269257347 wait complete. 2023-08-25T03:34:21.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.766+0000 7f93bd926700 1 Processor -- start 2023-08-25T03:34:21.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.766+0000 7f93bd926700 1 -- start start 2023-08-25T03:34:21.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.767+0000 7f93bd926700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93b806f4c0 0x7f93b8154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:21.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.767+0000 7f93bd926700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93b80cedf0 0x7f93b80ad2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:21.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.767+0000 7f93bd926700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93b8159f10 0x7f93b80b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:21.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.767+0000 7f93bd926700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b815b910 con 0x7f93b8159f10 2023-08-25T03:34:21.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.767+0000 7f93bd926700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b815b610 con 0x7f93b80cedf0 2023-08-25T03:34:21.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.767+0000 7f93bd926700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b8154c90 con 0x7f93b806f4c0 2023-08-25T03:34:21.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.767+0000 7f93b77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93b8159f10 0x7f93b80b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:21.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.767+0000 7f93b77fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93b8159f10 0x7f93b80b48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59934/0 (socket says 172.21.15.2:59934) 2023-08-25T03:34:21.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.767+0000 7f93b77fe700 1 -- 172.21.15.2:0/2769715723 learned_addr learned my addr 172.21.15.2:0/2769715723 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:21.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.767+0000 7f93b67fc700 1 --2- 172.21.15.2:0/2769715723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93b80cedf0 0x7f93b80ad2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:21.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.767+0000 7f93b6ffd700 1 --2- 172.21.15.2:0/2769715723 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93b806f4c0 0x7f93b8154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:21.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.767+0000 7f93b77fe700 1 -- 172.21.15.2:0/2769715723 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93b806f4c0 msgr2=0x7f93b8154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:21.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.767+0000 7f93b77fe700 1 --2- 172.21.15.2:0/2769715723 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93b806f4c0 0x7f93b8154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.767+0000 7f93b77fe700 1 -- 172.21.15.2:0/2769715723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93b80cedf0 msgr2=0x7f93b80ad2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:21.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.767+0000 7f93b77fe700 1 --2- 172.21.15.2:0/2769715723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93b80cedf0 0x7f93b80ad2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.767+0000 7f93b77fe700 1 -- 172.21.15.2:0/2769715723 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b0009550 con 0x7f93b8159f10 2023-08-25T03:34:21.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.767+0000 7f93b6ffd700 1 --2- 172.21.15.2:0/2769715723 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93b806f4c0 0x7f93b8154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:21.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.767+0000 7f93b77fe700 1 --2- 172.21.15.2:0/2769715723 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93b8159f10 0x7f93b80b48e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f93a800b9a0 tx=0x7f93a800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:21.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.767+0000 7f93affff700 1 -- 172.21.15.2:0/2769715723 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a800c9b0 con 0x7f93b8159f10 2023-08-25T03:34:21.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.767+0000 7f93affff700 1 -- 172.21.15.2:0/2769715723 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a800cb10 con 0x7f93b8159f10 2023-08-25T03:34:21.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.767+0000 7f93bd926700 1 -- 172.21.15.2:0/2769715723 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b80adc60 con 0x7f93b8159f10 2023-08-25T03:34:21.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.767+0000 7f93affff700 1 -- 172.21.15.2:0/2769715723 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8014440 con 0x7f93b8159f10 2023-08-25T03:34:21.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.767+0000 7f93bd926700 1 -- 172.21.15.2:0/2769715723 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b80ae220 con 0x7f93b8159f10 2023-08-25T03:34:21.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.768+0000 7f93bd926700 1 -- 172.21.15.2:0/2769715723 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b815bac0 con 0x7f93b8159f10 2023-08-25T03:34:21.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.769+0000 7f93affff700 1 -- 172.21.15.2:0/2769715723 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f93a801d3b0 con 0x7f93b8159f10 2023-08-25T03:34:21.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.769+0000 7f93affff700 1 --2- 172.21.15.2:0/2769715723 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f939806a0e0 0x7f939806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:21.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.770+0000 7f93affff700 1 -- 172.21.15.2:0/2769715723 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f93a8088190 con 0x7f93b8159f10 2023-08-25T03:34:21.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.770+0000 7f93b6ffd700 1 --2- 172.21.15.2:0/2769715723 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f939806a0e0 0x7f939806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:21.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.770+0000 7f93b6ffd700 1 --2- 172.21.15.2:0/2769715723 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f939806a0e0 0x7f939806c5a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009c10 tx=0x7f93b0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:21.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.774+0000 7f93affff700 1 -- 172.21.15.2:0/2769715723 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f93b815bac0 con 0x7f93b8159f10 2023-08-25T03:34:21.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.972+0000 7f93bd926700 1 -- 172.21.15.2:0/2769715723 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f93b815bac0 con 0x7f93b8159f10 2023-08-25T03:34:21.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.972+0000 7f93affff700 1 -- 172.21.15.2:0/2769715723 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f93b815bac0 con 0x7f93b8159f10 2023-08-25T03:34:21.974 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-08-25T03:34:21.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.974+0000 7f93bd926700 1 -- 172.21.15.2:0/2769715723 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f939806a0e0 msgr2=0x7f939806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:21.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.974+0000 7f93bd926700 1 --2- 172.21.15.2:0/2769715723 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f939806a0e0 0x7f939806c5a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f93b0009c10 tx=0x7f93b0008030 comp rx=0 tx=0).stop 2023-08-25T03:34:21.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.974+0000 7f93bd926700 1 -- 172.21.15.2:0/2769715723 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93b8159f10 msgr2=0x7f93b80b48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:21.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.974+0000 7f93bd926700 1 --2- 172.21.15.2:0/2769715723 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93b8159f10 0x7f93b80b48e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f93a800b9a0 tx=0x7f93a800bd60 comp rx=0 tx=0).stop 2023-08-25T03:34:21.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.974+0000 7f93bd926700 1 -- 172.21.15.2:0/2769715723 shutdown_connections 2023-08-25T03:34:21.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.974+0000 7f93bd926700 1 --2- 172.21.15.2:0/2769715723 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f939806a0e0 0x7f939806c5a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.974+0000 7f93bd926700 1 --2- 172.21.15.2:0/2769715723 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93b8159f10 0x7f93b80b48e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.974+0000 7f93bd926700 1 --2- 172.21.15.2:0/2769715723 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93b806f4c0 0x7f93b8154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.974+0000 7f93bd926700 1 --2- 172.21.15.2:0/2769715723 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93b80cedf0 0x7f93b80ad2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:21.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.974+0000 7f93bd926700 1 -- 172.21.15.2:0/2769715723 >> 172.21.15.2:0/2769715723 conn(0x7f93b80cfd90 msgr2=0x7f93b80cacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:21.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.974+0000 7f93bd926700 1 -- 172.21.15.2:0/2769715723 shutdown_connections 2023-08-25T03:34:21.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:21.974+0000 7f93bd926700 1 -- 172.21.15.2:0/2769715723 wait complete. 2023-08-25T03:34:21.991 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T03:34:22.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.115+0000 7f8f0ec0a700 1 Processor -- start 2023-08-25T03:34:22.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.116+0000 7f8f0ec0a700 1 -- start start 2023-08-25T03:34:22.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.116+0000 7f8f0ec0a700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8f08071db0 0x7f8f0806b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:22.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.116+0000 7f8f0ec0a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8f0815a3d0 0x7f8f0806b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:22.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.116+0000 7f8f0ec0a700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f080cf0a0 con 0x7f8f08159ff0 2023-08-25T03:34:22.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.116+0000 7f8f0ec0a700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f080ce580 con 0x7f8f08071db0 2023-08-25T03:34:22.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.116+0000 7f8f0ec0a700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f080ce770 con 0x7f8f0815a3d0 2023-08-25T03:34:22.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.116+0000 7f8f07fff700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8f08071db0 0x7f8f0806b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:22.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.116+0000 7f8f07fff700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f8f08071db0 0x7f8f0806b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59940/0 (socket says 172.21.15.2:59940) 2023-08-25T03:34:22.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.116+0000 7f8f0c9a6700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f8f08159ff0 0x7f8f080cde70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46604/0 (socket says 172.21.15.2:46604) 2023-08-25T03:34:22.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.116+0000 7f8f07fff700 1 -- 172.21.15.2:0/200122143 learned_addr learned my addr 172.21.15.2:0/200122143 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:22.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.117+0000 7f8f0d1a7700 1 --2- 172.21.15.2:0/200122143 >> v2:172.21.15.31:3300/0 conn(0x7f8f0815a3d0 0x7f8f0806b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:22.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.117+0000 7f8f07fff700 1 -- 172.21.15.2:0/200122143 >> v2:172.21.15.31:3300/0 conn(0x7f8f0815a3d0 msgr2=0x7f8f0806b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:22.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.117+0000 7f8f07fff700 1 --2- 172.21.15.2:0/200122143 >> v2:172.21.15.31:3300/0 conn(0x7f8f0815a3d0 0x7f8f0806b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:22.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.117+0000 7f8f07fff700 1 -- 172.21.15.2:0/200122143 >> v1:172.21.15.2:6789/0 conn(0x7f8f08159ff0 legacy=0x7f8f080cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:22.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.117+0000 7f8f077fe700 1 -- 172.21.15.2:0/200122143 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 374749607 0 0) 0x7f8f080cf0a0 con 0x7f8f08159ff0 2023-08-25T03:34:22.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.117+0000 7f8f07fff700 1 -- 172.21.15.2:0/200122143 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f080cea50 con 0x7f8f08071db0 2023-08-25T03:34:22.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.117+0000 7f8f07fff700 1 --2- 172.21.15.2:0/200122143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f08071db0 0x7f8f0806b2f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8f08070900 tx=0x7f8ef400be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8397084854bb90 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:22.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.117+0000 7f8f077fe700 1 -- 172.21.15.2:0/200122143 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef400dc70 con 0x7f8f08071db0 2023-08-25T03:34:22.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.117+0000 7f8f077fe700 1 -- 172.21.15.2:0/200122143 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef400ddd0 con 0x7f8f08071db0 2023-08-25T03:34:22.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.117+0000 7f8f077fe700 1 -- 172.21.15.2:0/200122143 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4010690 con 0x7f8f08071db0 2023-08-25T03:34:22.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.118+0000 7f8f0ec0a700 1 -- 172.21.15.2:0/200122143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f08071db0 msgr2=0x7f8f0806b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:22.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.118+0000 7f8f0ec0a700 1 --2- 172.21.15.2:0/200122143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f08071db0 0x7f8f0806b2f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8f08070900 tx=0x7f8ef400be70 comp rx=0 tx=0).stop 2023-08-25T03:34:22.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.118+0000 7f8f0ec0a700 1 -- 172.21.15.2:0/200122143 shutdown_connections 2023-08-25T03:34:22.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.118+0000 7f8f0ec0a700 1 --2- 172.21.15.2:0/200122143 >> v2:172.21.15.31:3300/0 conn(0x7f8f0815a3d0 0x7f8f0806b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:22.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.118+0000 7f8f0ec0a700 1 --2- 172.21.15.2:0/200122143 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f08071db0 0x7f8f0806b2f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:22.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.118+0000 7f8f0ec0a700 1 -- 172.21.15.2:0/200122143 >> 172.21.15.2:0/200122143 conn(0x7f8f080b1bf0 msgr2=0x7f8f080b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:22.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.118+0000 7f8f0ec0a700 1 -- 172.21.15.2:0/200122143 shutdown_connections 2023-08-25T03:34:22.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.118+0000 7f8f0ec0a700 1 -- 172.21.15.2:0/200122143 wait complete. 2023-08-25T03:34:22.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.119+0000 7f8f0ec0a700 1 Processor -- start 2023-08-25T03:34:22.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.119+0000 7f8f0ec0a700 1 -- start start 2023-08-25T03:34:22.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.120+0000 7f8f0ec0a700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8f08071db0 0x7f8f0806ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:22.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.120+0000 7f8f0ec0a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f08159ff0 0x7f8f080c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:22.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.120+0000 7f8f0ec0a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f0815a3d0 0x7f8f080c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:22.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.120+0000 7f8f0ec0a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0815d340 con 0x7f8f08159ff0 2023-08-25T03:34:22.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.120+0000 7f8f0ec0a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0815c730 con 0x7f8f0815a3d0 2023-08-25T03:34:22.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.120+0000 7f8f0ec0a700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0815ca10 con 0x7f8f08071db0 2023-08-25T03:34:22.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.120+0000 7f8f07fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f08159ff0 0x7f8f080c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:22.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.120+0000 7f8f07fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f08159ff0 0x7f8f080c9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59946/0 (socket says 172.21.15.2:59946) 2023-08-25T03:34:22.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.120+0000 7f8f07fff700 1 -- 172.21.15.2:0/3927027909 learned_addr learned my addr 172.21.15.2:0/3927027909 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:22.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.120+0000 7f8f0d1a7700 1 --2- 172.21.15.2:0/3927027909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f0815a3d0 0x7f8f080c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:22.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.120+0000 7f8f07fff700 1 -- 172.21.15.2:0/3927027909 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8f08071db0 msgr2=0x7f8f0806ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:22.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.120+0000 7f8f0c9a6700 1 --2- 172.21.15.2:0/3927027909 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8f08071db0 0x7f8f0806ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:22.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.120+0000 7f8f07fff700 1 --2- 172.21.15.2:0/3927027909 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8f08071db0 0x7f8f0806ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:22.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.120+0000 7f8f07fff700 1 -- 172.21.15.2:0/3927027909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f0815a3d0 msgr2=0x7f8f080c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:22.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.120+0000 7f8f07fff700 1 --2- 172.21.15.2:0/3927027909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f0815a3d0 0x7f8f080c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:22.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.120+0000 7f8f07fff700 1 -- 172.21.15.2:0/3927027909 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef400b7f0 con 0x7f8f08159ff0 2023-08-25T03:34:22.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.120+0000 7f8f0d1a7700 1 --2- 172.21.15.2:0/3927027909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f0815a3d0 0x7f8f080c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:22.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.120+0000 7f8f0c9a6700 1 --2- 172.21.15.2:0/3927027909 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8f08071db0 0x7f8f0806ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:22.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.121+0000 7f8f07fff700 1 --2- 172.21.15.2:0/3927027909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f08159ff0 0x7f8f080c9220 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8ef400beb0 tx=0x7f8ef4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:22.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.121+0000 7f8f057fa700 1 -- 172.21.15.2:0/3927027909 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef401d070 con 0x7f8f08159ff0 2023-08-25T03:34:22.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.121+0000 7f8f057fa700 1 -- 172.21.15.2:0/3927027909 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ef401e040 con 0x7f8f08159ff0 2023-08-25T03:34:22.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.121+0000 7f8f0ec0a700 1 -- 172.21.15.2:0/3927027909 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0815ccf0 con 0x7f8f08159ff0 2023-08-25T03:34:22.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.121+0000 7f8f057fa700 1 -- 172.21.15.2:0/3927027909 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4010c70 con 0x7f8f08159ff0 2023-08-25T03:34:22.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.121+0000 7f8f0ec0a700 1 -- 172.21.15.2:0/3927027909 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0806b150 con 0x7f8f08159ff0 2023-08-25T03:34:22.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.122+0000 7f8f0ec0a700 1 -- 172.21.15.2:0/3927027909 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f08066f70 con 0x7f8f08159ff0 2023-08-25T03:34:22.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.122+0000 7f8f057fa700 1 -- 172.21.15.2:0/3927027909 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8ef40036a0 con 0x7f8f08159ff0 2023-08-25T03:34:22.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.123+0000 7f8f057fa700 1 --2- 172.21.15.2:0/3927027909 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8eec06a0e0 0x7f8eec06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:22.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.123+0000 7f8f057fa700 1 -- 172.21.15.2:0/3927027909 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f8ef408c7a0 con 0x7f8f08159ff0 2023-08-25T03:34:22.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.123+0000 7f8f0c9a6700 1 --2- 172.21.15.2:0/3927027909 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8eec06a0e0 0x7f8eec06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:22.132 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.123+0000 7f8f0c9a6700 1 --2- 172.21.15.2:0/3927027909 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8eec06a0e0 0x7f8eec06c5a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8efc000f60 tx=0x7f8efc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:22.133 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.127+0000 7f8f057fa700 1 -- 172.21.15.2:0/3927027909 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8ef4057e80 con 0x7f8f08159ff0 2023-08-25T03:34:22.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.372+0000 7f8f0ec0a700 1 -- 172.21.15.2:0/3927027909 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f0806da40 con 0x7f8f08159ff0 2023-08-25T03:34:22.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.372+0000 7f8f057fa700 1 -- 172.21.15.2:0/3927027909 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4857 (secure 0 0 0) 0x7f8ef405b4a0 con 0x7f8f08159ff0 2023-08-25T03:34:22.374 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:22.374 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2074312983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2074312983},{"type":"v1","addr":"172.21.15.2:6837","nonce":2074312983}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4059180922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4059180922},{"type":"v1","addr":"172.21.15.31:6835","nonce":4059180922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/922384230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":922384230},{"type":"v1","addr":"172.21.15.31:6834","nonce":922384230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T03:34:03.009109+0000","modified":"2023-08-25T03:34:06.613200+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.2:6835/193990892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":193990892},{"type":"v1","addr":"172.21.15.2:6835","nonce":193990892}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T03:34:22.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.374+0000 7f8f0ec0a700 1 -- 172.21.15.2:0/3927027909 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8eec06a0e0 msgr2=0x7f8eec06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:22.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.374+0000 7f8f0ec0a700 1 --2- 172.21.15.2:0/3927027909 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8eec06a0e0 0x7f8eec06c5a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8efc000f60 tx=0x7f8efc009040 comp rx=0 tx=0).stop 2023-08-25T03:34:22.376 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.374+0000 7f8f0ec0a700 1 -- 172.21.15.2:0/3927027909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f08159ff0 msgr2=0x7f8f080c9220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:22.376 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.374+0000 7f8f0ec0a700 1 --2- 172.21.15.2:0/3927027909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f08159ff0 0x7f8f080c9220 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8ef400beb0 tx=0x7f8ef4004750 comp rx=0 tx=0).stop 2023-08-25T03:34:22.376 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.374+0000 7f8f0ec0a700 1 -- 172.21.15.2:0/3927027909 shutdown_connections 2023-08-25T03:34:22.376 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.374+0000 7f8f0ec0a700 1 --2- 172.21.15.2:0/3927027909 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f0815a3d0 0x7f8f080c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:22.376 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.374+0000 7f8f0ec0a700 1 --2- 172.21.15.2:0/3927027909 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8f08071db0 0x7f8f0806ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:22.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.374+0000 7f8f0ec0a700 1 --2- 172.21.15.2:0/3927027909 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8eec06a0e0 0x7f8eec06c5a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:22.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.374+0000 7f8f0ec0a700 1 --2- 172.21.15.2:0/3927027909 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8f08159ff0 0x7f8f080c9220 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:22.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.374+0000 7f8f0ec0a700 1 -- 172.21.15.2:0/3927027909 >> 172.21.15.2:0/3927027909 conn(0x7f8f080b1bf0 msgr2=0x7f8f08159670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:22.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.374+0000 7f8f0ec0a700 1 -- 172.21.15.2:0/3927027909 shutdown_connections 2023-08-25T03:34:22.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.374+0000 7f8f0ec0a700 1 -- 172.21.15.2:0/3927027909 wait complete. 2023-08-25T03:34:22.378 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 8 2023-08-25T03:34:22.388 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:34:22.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.514+0000 7f1bf1b56700 1 Processor -- start 2023-08-25T03:34:22.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.514+0000 7f1bf1b56700 1 -- start start 2023-08-25T03:34:22.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.515+0000 7f1bf1b56700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1bec15a050 0x7f1bec15db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:22.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.515+0000 7f1bf1b56700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1bec06b900 0x7f1bec15ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:22.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.515+0000 7f1bf1b56700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bec15a430 con 0x7f1bec0b3780 2023-08-25T03:34:22.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.515+0000 7f1bf1b56700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bec15e450 con 0x7f1bec15a050 2023-08-25T03:34:22.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.515+0000 7f1bf1b56700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec15e730 con 0x7f1bec06b900 2023-08-25T03:34:22.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.516+0000 7f1beb7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1bec15a050 0x7f1bec15db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:22.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.516+0000 7f1bebfff700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1bec06b900 0x7f1bec15ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:22.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.516+0000 7f1bebfff700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1bec06b900 0x7f1bec15ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.2:60200/0 (socket says 172.21.15.2:60200) 2023-08-25T03:34:22.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.516+0000 7f1beb7fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1bec15a050 0x7f1bec15db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.2:51242/0 (socket says 172.21.15.2:51242) 2023-08-25T03:34:22.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.516+0000 7f1bebfff700 1 -- 172.21.15.2:0/843928764 learned_addr learned my addr 172.21.15.2:0/843928764 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:22.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.516+0000 7f1beaffd700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f1bec0b3780 0x7f1bec06b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.2:54644/0 (socket says 172.21.15.2:54644) 2023-08-25T03:34:22.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.516+0000 7f1beb7fe700 1 -- 172.21.15.2:0/843928764 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1bec06b900 msgr2=0x7f1bec15ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:22.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.516+0000 7f1beb7fe700 1 --2- 172.21.15.2:0/843928764 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1bec06b900 0x7f1bec15ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:22.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.516+0000 7f1beb7fe700 1 -- 172.21.15.2:0/843928764 >> v1:172.21.15.31:6789/0 conn(0x7f1bec0b3780 legacy=0x7f1bec06b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:22.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.516+0000 7f1bea7fc700 1 -- 172.21.15.2:0/843928764 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4053143292 0 0) 0x7f1bdc001110 con 0x7f1bec0b3780 2023-08-25T03:34:22.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.516+0000 7f1beb7fe700 1 -- 172.21.15.2:0/843928764 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bec15ea10 con 0x7f1bec15a050 2023-08-25T03:34:22.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.516+0000 7f1bebfff700 1 --2- 172.21.15.2:0/843928764 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1bec06b900 0x7f1bec15ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:34:22.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.516+0000 7f1beb7fe700 1 --2- 172.21.15.2:0/843928764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1bec15a050 0x7f1bec15db50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4009840 tx=0x7f1bd4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0fa286d2b033fd3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:22.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.516+0000 7f1bea7fc700 1 -- 172.21.15.2:0/843928764 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4011830 con 0x7f1bec15a050 2023-08-25T03:34:22.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.516+0000 7f1bea7fc700 1 -- 172.21.15.2:0/843928764 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bd4011990 con 0x7f1bec15a050 2023-08-25T03:34:22.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.516+0000 7f1bea7fc700 1 -- 172.21.15.2:0/843928764 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4011d20 con 0x7f1bec15a050 2023-08-25T03:34:22.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.517+0000 7f1bf1b56700 1 -- 172.21.15.2:0/843928764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1bec15a050 msgr2=0x7f1bec15db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:22.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.517+0000 7f1bf1b56700 1 --2- 172.21.15.2:0/843928764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1bec15a050 0x7f1bec15db50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4009840 tx=0x7f1bd4009b50 comp rx=0 tx=0).stop 2023-08-25T03:34:22.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.517+0000 7f1bf1b56700 1 -- 172.21.15.2:0/843928764 shutdown_connections 2023-08-25T03:34:22.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.517+0000 7f1bf1b56700 1 --2- 172.21.15.2:0/843928764 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1bec06b900 0x7f1bec15ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:22.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.517+0000 7f1bf1b56700 1 --2- 172.21.15.2:0/843928764 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1bec15a050 0x7f1bec15db50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:22.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.517+0000 7f1bf1b56700 1 -- 172.21.15.2:0/843928764 >> 172.21.15.2:0/843928764 conn(0x7f1bec0cfd10 msgr2=0x7f1bec06ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:22.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.517+0000 7f1bf1b56700 1 -- 172.21.15.2:0/843928764 shutdown_connections 2023-08-25T03:34:22.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.517+0000 7f1bf1b56700 1 -- 172.21.15.2:0/843928764 wait complete. 2023-08-25T03:34:22.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.518+0000 7f1bf1b56700 1 Processor -- start 2023-08-25T03:34:22.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.518+0000 7f1bf1b56700 1 -- start start 2023-08-25T03:34:22.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.519+0000 7f1bf1b56700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1bec06b900 0x7f1bec14c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:22.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.519+0000 7f1bf1b56700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1bec0b3780 0x7f1bec14ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:22.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.519+0000 7f1bf1b56700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1bec15a050 0x7f1bec154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:22.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.519+0000 7f1bf1b56700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec14b580 con 0x7f1bec15a050 2023-08-25T03:34:22.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.519+0000 7f1bf1b56700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec14c7d0 con 0x7f1bec0b3780 2023-08-25T03:34:22.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.519+0000 7f1bf1b56700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec14cab0 con 0x7f1bec06b900 2023-08-25T03:34:22.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.519+0000 7f1bebfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1bec15a050 0x7f1bec154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:22.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.519+0000 7f1bebfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1bec15a050 0x7f1bec154f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59962/0 (socket says 172.21.15.2:59962) 2023-08-25T03:34:22.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.519+0000 7f1bebfff700 1 -- 172.21.15.2:0/1143487670 learned_addr learned my addr 172.21.15.2:0/1143487670 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:22.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.519+0000 7f1beaffd700 1 --2- 172.21.15.2:0/1143487670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1bec0b3780 0x7f1bec14ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:22.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.519+0000 7f1beb7fe700 1 --2- 172.21.15.2:0/1143487670 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1bec06b900 0x7f1bec14c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:22.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.519+0000 7f1bebfff700 1 -- 172.21.15.2:0/1143487670 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1bec06b900 msgr2=0x7f1bec14c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:22.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.519+0000 7f1bebfff700 1 --2- 172.21.15.2:0/1143487670 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1bec06b900 0x7f1bec14c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:22.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.519+0000 7f1bebfff700 1 -- 172.21.15.2:0/1143487670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1bec0b3780 msgr2=0x7f1bec14ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:22.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.519+0000 7f1bebfff700 1 --2- 172.21.15.2:0/1143487670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1bec0b3780 0x7f1bec14ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:22.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.519+0000 7f1bebfff700 1 -- 172.21.15.2:0/1143487670 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be4009b30 con 0x7f1bec15a050 2023-08-25T03:34:22.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.520+0000 7f1bebfff700 1 --2- 172.21.15.2:0/1143487670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1bec15a050 0x7f1bec154f50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1be400efc0 tx=0x7f1be4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:22.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.520+0000 7f1be3fff700 1 -- 172.21.15.2:0/1143487670 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be4018070 con 0x7f1bec15a050 2023-08-25T03:34:22.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.520+0000 7f1bf1b56700 1 -- 172.21.15.2:0/1143487670 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bd40094a0 con 0x7f1bec15a050 2023-08-25T03:34:22.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.520+0000 7f1be3fff700 1 -- 172.21.15.2:0/1143487670 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be400f040 con 0x7f1bec15a050 2023-08-25T03:34:22.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.520+0000 7f1be3fff700 1 -- 172.21.15.2:0/1143487670 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be4003c40 con 0x7f1bec15a050 2023-08-25T03:34:22.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.520+0000 7f1bf1b56700 1 -- 172.21.15.2:0/1143487670 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bec14d070 con 0x7f1bec15a050 2023-08-25T03:34:22.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.521+0000 7f1bf1b56700 1 -- 172.21.15.2:0/1143487670 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bec0746d0 con 0x7f1bec15a050 2023-08-25T03:34:22.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.521+0000 7f1be3fff700 1 -- 172.21.15.2:0/1143487670 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1be400c9e0 con 0x7f1bec15a050 2023-08-25T03:34:22.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.522+0000 7f1be3fff700 1 --2- 172.21.15.2:0/1143487670 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1bcc06a0e0 0x7f1bcc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:22.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.522+0000 7f1be3fff700 1 -- 172.21.15.2:0/1143487670 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1be4088870 con 0x7f1bec15a050 2023-08-25T03:34:22.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.522+0000 7f1beb7fe700 1 --2- 172.21.15.2:0/1143487670 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1bcc06a0e0 0x7f1bcc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:22.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.522+0000 7f1beb7fe700 1 --2- 172.21.15.2:0/1143487670 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1bcc06a0e0 0x7f1bcc06c5a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4000c00 tx=0x7f1bd401c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:22.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.526+0000 7f1be3fff700 1 -- 172.21.15.2:0/1143487670 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1be4053f70 con 0x7f1bec15a050 2023-08-25T03:34:22.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.728+0000 7f1bf1b56700 1 -- 172.21.15.2:0/1143487670 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1bec14bfd0 con 0x7f1bec15a050 2023-08-25T03:34:22.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.729+0000 7f1be3fff700 1 -- 172.21.15.2:0/1143487670 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18135 (secure 0 0 0) 0x7f1be4057590 con 0x7f1bec15a050 2023-08-25T03:34:22.730 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:22.731 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":24,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:34:04.604059+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:34:00.869118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T03:34:01.987016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:34:22.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.732+0000 7f1bf1b56700 1 -- 172.21.15.2:0/1143487670 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1bcc06a0e0 msgr2=0x7f1bcc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:22.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.732+0000 7f1bf1b56700 1 --2- 172.21.15.2:0/1143487670 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1bcc06a0e0 0x7f1bcc06c5a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4000c00 tx=0x7f1bd401c040 comp rx=0 tx=0).stop 2023-08-25T03:34:22.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.732+0000 7f1bf1b56700 1 -- 172.21.15.2:0/1143487670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1bec15a050 msgr2=0x7f1bec154f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:22.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.732+0000 7f1bf1b56700 1 --2- 172.21.15.2:0/1143487670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1bec15a050 0x7f1bec154f50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1be400efc0 tx=0x7f1be4009f90 comp rx=0 tx=0).stop 2023-08-25T03:34:22.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.732+0000 7f1bf1b56700 1 -- 172.21.15.2:0/1143487670 shutdown_connections 2023-08-25T03:34:22.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.732+0000 7f1bf1b56700 1 --2- 172.21.15.2:0/1143487670 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1bcc06a0e0 0x7f1bcc06c5a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:22.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.732+0000 7f1bf1b56700 1 --2- 172.21.15.2:0/1143487670 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1bec15a050 0x7f1bec154f50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:22.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.732+0000 7f1bf1b56700 1 --2- 172.21.15.2:0/1143487670 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1bec06b900 0x7f1bec14c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:22.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.732+0000 7f1bf1b56700 1 --2- 172.21.15.2:0/1143487670 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1bec0b3780 0x7f1bec14ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:22.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.732+0000 7f1bf1b56700 1 -- 172.21.15.2:0/1143487670 >> 172.21.15.2:0/1143487670 conn(0x7f1bec0cfd10 msgr2=0x7f1bec06a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:22.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.732+0000 7f1bf1b56700 1 -- 172.21.15.2:0/1143487670 shutdown_connections 2023-08-25T03:34:22.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.732+0000 7f1bf1b56700 1 -- 172.21.15.2:0/1143487670 wait complete. 2023-08-25T03:34:22.745 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T03:34:22.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.869+0000 7efe7b96e700 1 Processor -- start 2023-08-25T03:34:22.870 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.869+0000 7efe7b96e700 1 -- start start 2023-08-25T03:34:22.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.869+0000 7efe7b96e700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7efe740b19a0 0x7efe7415e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:22.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.870+0000 7efe7b96e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7efe740caea0 0x7efe740cb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:22.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.870+0000 7efe7b96e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe74071b40 con 0x7efe740b1d80 2023-08-25T03:34:22.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.870+0000 7efe7b96e700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe740720b0 con 0x7efe740b19a0 2023-08-25T03:34:22.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.870+0000 7efe7b96e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe74071dd0 con 0x7efe740caea0 2023-08-25T03:34:22.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.870+0000 7efe7970a700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7efe740b19a0 0x7efe7415e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:22.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.870+0000 7efe7970a700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7efe740b19a0 0x7efe7415e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59966/0 (socket says 172.21.15.2:59966) 2023-08-25T03:34:22.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.870+0000 7efe7970a700 1 -- 172.21.15.2:0/2349650582 learned_addr learned my addr 172.21.15.2:0/2349650582 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:22.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.870+0000 7efe79f0b700 1 --2- 172.21.15.2:0/2349650582 >> v2:172.21.15.31:3300/0 conn(0x7efe740caea0 0x7efe740cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:22.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.870+0000 7efe7970a700 1 -- 172.21.15.2:0/2349650582 >> v2:172.21.15.31:3300/0 conn(0x7efe740caea0 msgr2=0x7efe740cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:22.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.870+0000 7efe7970a700 1 --2- 172.21.15.2:0/2349650582 >> v2:172.21.15.31:3300/0 conn(0x7efe740caea0 0x7efe740cb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:22.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.870+0000 7efe7970a700 1 -- 172.21.15.2:0/2349650582 >> v1:172.21.15.31:6789/0 conn(0x7efe740b1d80 legacy=0x7efe74071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:22.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.870+0000 7efe7970a700 1 -- 172.21.15.2:0/2349650582 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7415ddb0 con 0x7efe740b19a0 2023-08-25T03:34:22.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.870+0000 7efe79f0b700 1 --2- 172.21.15.2:0/2349650582 >> v2:172.21.15.31:3300/0 conn(0x7efe740caea0 0x7efe740cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:22.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.870+0000 7efe7970a700 1 --2- 172.21.15.2:0/2349650582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe740b19a0 0x7efe7415e890 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efe740cc8b0 tx=0x7efe68009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475a0d73b4eb4dac server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:22.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.870+0000 7efe67fff700 1 -- 172.21.15.2:0/2349650582 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6800b870 con 0x7efe740b19a0 2023-08-25T03:34:22.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.870+0000 7efe67fff700 1 -- 172.21.15.2:0/2349650582 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6800b9d0 con 0x7efe740b19a0 2023-08-25T03:34:22.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.870+0000 7efe67fff700 1 -- 172.21.15.2:0/2349650582 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6800bd60 con 0x7efe740b19a0 2023-08-25T03:34:22.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.871+0000 7efe7b96e700 1 -- 172.21.15.2:0/2349650582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe740b19a0 msgr2=0x7efe7415e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:22.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.871+0000 7efe7b96e700 1 --2- 172.21.15.2:0/2349650582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe740b19a0 0x7efe7415e890 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efe740cc8b0 tx=0x7efe68009bd0 comp rx=0 tx=0).stop 2023-08-25T03:34:22.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.871+0000 7efe7b96e700 1 -- 172.21.15.2:0/2349650582 shutdown_connections 2023-08-25T03:34:22.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.871+0000 7efe7b96e700 1 --2- 172.21.15.2:0/2349650582 >> v2:172.21.15.31:3300/0 conn(0x7efe740caea0 0x7efe740cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:22.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.871+0000 7efe7b96e700 1 --2- 172.21.15.2:0/2349650582 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe740b19a0 0x7efe7415e890 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:22.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.871+0000 7efe7b96e700 1 -- 172.21.15.2:0/2349650582 >> 172.21.15.2:0/2349650582 conn(0x7efe740cfea0 msgr2=0x7efe740d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:22.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.871+0000 7efe7b96e700 1 -- 172.21.15.2:0/2349650582 shutdown_connections 2023-08-25T03:34:22.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.871+0000 7efe7b96e700 1 -- 172.21.15.2:0/2349650582 wait complete. 2023-08-25T03:34:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.873+0000 7efe7b96e700 1 Processor -- start 2023-08-25T03:34:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.873+0000 7efe7b96e700 1 -- start start 2023-08-25T03:34:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.874+0000 7efe7b96e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe740b19a0 0x7efe740af080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.874+0000 7efe7b96e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe740b1d80 0x7efe741f00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.874+0000 7efe7b96e700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efe740caea0 0x7efe740cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.874+0000 7efe7b96e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7415c080 con 0x7efe740b19a0 2023-08-25T03:34:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.874+0000 7efe7b96e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe741e35a0 con 0x7efe740b1d80 2023-08-25T03:34:22.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.874+0000 7efe7b96e700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe741f0630 con 0x7efe740caea0 2023-08-25T03:34:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.874+0000 7efe7970a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe740b19a0 0x7efe740af080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.874+0000 7efe7970a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe740b19a0 0x7efe740af080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59972/0 (socket says 172.21.15.2:59972) 2023-08-25T03:34:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.874+0000 7efe7970a700 1 -- 172.21.15.2:0/997564188 learned_addr learned my addr 172.21.15.2:0/997564188 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.874+0000 7efe78f09700 1 --2- 172.21.15.2:0/997564188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe740b1d80 0x7efe741f00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.874+0000 7efe7970a700 1 -- 172.21.15.2:0/997564188 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efe740caea0 msgr2=0x7efe740cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.874+0000 7efe79f0b700 1 --2- 172.21.15.2:0/997564188 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efe740caea0 0x7efe740cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:22.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.874+0000 7efe7970a700 1 --2- 172.21.15.2:0/997564188 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efe740caea0 0x7efe740cbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:22.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.874+0000 7efe7970a700 1 -- 172.21.15.2:0/997564188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe740b1d80 msgr2=0x7efe741f00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:22.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.874+0000 7efe7970a700 1 --2- 172.21.15.2:0/997564188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe740b1d80 0x7efe741f00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:22.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.874+0000 7efe7970a700 1 -- 172.21.15.2:0/997564188 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe68009550 con 0x7efe740b19a0 2023-08-25T03:34:22.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.874+0000 7efe78f09700 1 --2- 172.21.15.2:0/997564188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe740b1d80 0x7efe741f00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:22.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.874+0000 7efe7970a700 1 --2- 172.21.15.2:0/997564188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe740b19a0 0x7efe740af080 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efe68000c00 tx=0x7efe68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:22.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.874+0000 7efe65ffb700 1 -- 172.21.15.2:0/997564188 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe68004520 con 0x7efe740b19a0 2023-08-25T03:34:22.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.874+0000 7efe65ffb700 1 -- 172.21.15.2:0/997564188 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6801e070 con 0x7efe740b19a0 2023-08-25T03:34:22.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.874+0000 7efe7b96e700 1 -- 172.21.15.2:0/997564188 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe7415a9c0 con 0x7efe740b19a0 2023-08-25T03:34:22.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.874+0000 7efe65ffb700 1 -- 172.21.15.2:0/997564188 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe680038a0 con 0x7efe740b19a0 2023-08-25T03:34:22.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.875+0000 7efe7b96e700 1 -- 172.21.15.2:0/997564188 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe7415af80 con 0x7efe740b19a0 2023-08-25T03:34:22.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.875+0000 7efe7b96e700 1 -- 172.21.15.2:0/997564188 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe74066f70 con 0x7efe740b19a0 2023-08-25T03:34:22.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.876+0000 7efe65ffb700 1 -- 172.21.15.2:0/997564188 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7efe68003a00 con 0x7efe740b19a0 2023-08-25T03:34:22.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.876+0000 7efe65ffb700 1 --2- 172.21.15.2:0/997564188 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7efe5806a0e0 0x7efe5806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:22.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.876+0000 7efe65ffb700 1 -- 172.21.15.2:0/997564188 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efe68020070 con 0x7efe740b19a0 2023-08-25T03:34:22.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.877+0000 7efe78f09700 1 --2- 172.21.15.2:0/997564188 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7efe5806a0e0 0x7efe5806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:22.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.877+0000 7efe78f09700 1 --2- 172.21.15.2:0/997564188 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7efe5806a0e0 0x7efe5806c5a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efe740c6e40 tx=0x7efe70009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:22.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:22.881+0000 7efe65ffb700 1 -- 172.21.15.2:0/997564188 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7efe680552e0 con 0x7efe740b19a0 2023-08-25T03:34:23.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.125+0000 7efe7b96e700 1 -- 172.21.15.2:0/997564188 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe740746d0 con 0x7efe740b19a0 2023-08-25T03:34:23.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.126+0000 7efe65ffb700 1 -- 172.21.15.2:0/997564188 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4857 (secure 0 0 0) 0x7efe68003cb0 con 0x7efe740b19a0 2023-08-25T03:34:23.127 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:23.127 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2074312983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2074312983},{"type":"v1","addr":"172.21.15.2:6837","nonce":2074312983}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4059180922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4059180922},{"type":"v1","addr":"172.21.15.31:6835","nonce":4059180922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/922384230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":922384230},{"type":"v1","addr":"172.21.15.31:6834","nonce":922384230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T03:34:03.009109+0000","modified":"2023-08-25T03:34:06.613200+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.2:6835/193990892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":193990892},{"type":"v1","addr":"172.21.15.2:6835","nonce":193990892}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T03:34:23.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.128+0000 7efe7b96e700 1 -- 172.21.15.2:0/997564188 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7efe5806a0e0 msgr2=0x7efe5806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:23.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.128+0000 7efe7b96e700 1 --2- 172.21.15.2:0/997564188 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7efe5806a0e0 0x7efe5806c5a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efe740c6e40 tx=0x7efe70009040 comp rx=0 tx=0).stop 2023-08-25T03:34:23.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.128+0000 7efe7b96e700 1 -- 172.21.15.2:0/997564188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe740b19a0 msgr2=0x7efe740af080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:23.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.128+0000 7efe7b96e700 1 --2- 172.21.15.2:0/997564188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe740b19a0 0x7efe740af080 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7efe68000c00 tx=0x7efe68004750 comp rx=0 tx=0).stop 2023-08-25T03:34:23.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.128+0000 7efe7b96e700 1 -- 172.21.15.2:0/997564188 shutdown_connections 2023-08-25T03:34:23.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.128+0000 7efe7b96e700 1 --2- 172.21.15.2:0/997564188 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7efe5806a0e0 0x7efe5806c5a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.128+0000 7efe7b96e700 1 --2- 172.21.15.2:0/997564188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe740b19a0 0x7efe740af080 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.128+0000 7efe7b96e700 1 --2- 172.21.15.2:0/997564188 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efe740caea0 0x7efe740cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.128+0000 7efe7b96e700 1 --2- 172.21.15.2:0/997564188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe740b1d80 0x7efe741f00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.128+0000 7efe7b96e700 1 -- 172.21.15.2:0/997564188 >> 172.21.15.2:0/997564188 conn(0x7efe740cfea0 msgr2=0x7efe740b1070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:23.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.128+0000 7efe7b96e700 1 -- 172.21.15.2:0/997564188 shutdown_connections 2023-08-25T03:34:23.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.128+0000 7efe7b96e700 1 -- 172.21.15.2:0/997564188 wait complete. 2023-08-25T03:34:23.131 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 8 2023-08-25T03:34:23.543 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-25T03:34:03.009109+0000', 'modified': '2023-08-25T03:34:06.613200+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4299}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4299': {'gid': 4299, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.2:6835/193990892', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.2:6834', 'nonce': 193990892}, {'type': 'v1', 'addr': '172.21.15.2:6835', 'nonce': 193990892}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-25T03:34:23.543 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-25T03:34:23.544 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-08-25T03:34:23.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.677+0000 7f29fa389700 1 Processor -- start 2023-08-25T03:34:23.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.677+0000 7f29fa389700 1 -- start start 2023-08-25T03:34:23.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.678+0000 7f29fa389700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f29f415b060 0x7f29f415b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:23.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.678+0000 7f29fa389700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f29f40cb5a0 0x7f29f40ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:23.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.678+0000 7f29fa389700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29f40b36a0 con 0x7f29f40cd480 2023-08-25T03:34:23.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.678+0000 7f29fa389700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29f40b6b30 con 0x7f29f415b060 2023-08-25T03:34:23.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.678+0000 7f29fa389700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29f415b980 con 0x7f29f40cb5a0 2023-08-25T03:34:23.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.678+0000 7f29f3fff700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f29f415b060 0x7f29f415b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:23.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.678+0000 7f29f37fe700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f29f40cb5a0 0x7f29f40ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:23.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.678+0000 7f29f8926700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f29f40cd480 0x7f29f406dff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.2:54674/0 (socket says 172.21.15.2:54674) 2023-08-25T03:34:23.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.678+0000 7f29f8926700 1 -- 172.21.15.2:0/3122385503 learned_addr learned my addr 172.21.15.2:0/3122385503 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:23.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.679+0000 7f29f2ffd700 1 -- 172.21.15.2:0/3122385503 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3459042840 0 0) 0x7f29f40b36a0 con 0x7f29f40cd480 2023-08-25T03:34:23.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.679+0000 7f29f2ffd700 1 -- 172.21.15.2:0/3122385503 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29dc0035b0 con 0x7f29f40cd480 2023-08-25T03:34:23.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.679+0000 7f29f3fff700 1 -- 172.21.15.2:0/3122385503 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f29f40cb5a0 msgr2=0x7f29f40ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:23.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.679+0000 7f29f3fff700 1 --2- 172.21.15.2:0/3122385503 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f29f40cb5a0 0x7f29f40ccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.679+0000 7f29f3fff700 1 -- 172.21.15.2:0/3122385503 >> v1:172.21.15.31:6789/0 conn(0x7f29f40cd480 legacy=0x7f29f406dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:23.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.679+0000 7f29f3fff700 1 -- 172.21.15.2:0/3122385503 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f40b2a90 con 0x7f29f415b060 2023-08-25T03:34:23.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.679+0000 7f29f37fe700 1 --2- 172.21.15.2:0/3122385503 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f29f40cb5a0 0x7f29f40ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:34:23.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.679+0000 7f29f3fff700 1 --2- 172.21.15.2:0/3122385503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29f415b060 0x7f29f415b440 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f29e4009840 tx=0x7f29e4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45a571474acb618f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:23.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.679+0000 7f29f2ffd700 1 -- 172.21.15.2:0/3122385503 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e4011830 con 0x7f29f415b060 2023-08-25T03:34:23.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.679+0000 7f29f2ffd700 1 -- 172.21.15.2:0/3122385503 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e4011990 con 0x7f29f415b060 2023-08-25T03:34:23.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.679+0000 7f29f2ffd700 1 -- 172.21.15.2:0/3122385503 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e4011d00 con 0x7f29f415b060 2023-08-25T03:34:23.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.680+0000 7f29fa389700 1 -- 172.21.15.2:0/3122385503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29f415b060 msgr2=0x7f29f415b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:23.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.680+0000 7f29fa389700 1 --2- 172.21.15.2:0/3122385503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29f415b060 0x7f29f415b440 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f29e4009840 tx=0x7f29e4009b50 comp rx=0 tx=0).stop 2023-08-25T03:34:23.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.680+0000 7f29fa389700 1 -- 172.21.15.2:0/3122385503 shutdown_connections 2023-08-25T03:34:23.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.680+0000 7f29fa389700 1 --2- 172.21.15.2:0/3122385503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29f415b060 0x7f29f415b440 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.680+0000 7f29fa389700 1 --2- 172.21.15.2:0/3122385503 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f29f40cb5a0 0x7f29f40ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.680+0000 7f29fa389700 1 -- 172.21.15.2:0/3122385503 >> 172.21.15.2:0/3122385503 conn(0x7f29f40d0f70 msgr2=0x7f29f40d1350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:23.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.680+0000 7f29fa389700 1 -- 172.21.15.2:0/3122385503 shutdown_connections 2023-08-25T03:34:23.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.680+0000 7f29fa389700 1 -- 172.21.15.2:0/3122385503 wait complete. 2023-08-25T03:34:23.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.681+0000 7f29fa389700 1 Processor -- start 2023-08-25T03:34:23.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.681+0000 7f29fa389700 1 -- start start 2023-08-25T03:34:23.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.682+0000 7f29fa389700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f29f40cb5a0 0x7f29f40ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:23.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.682+0000 7f29fa389700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29f40cd480 0x7f29f415cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:23.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.682+0000 7f29fa389700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29f415b060 0x7f29f415d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:23.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.682+0000 7f29fa389700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f415bcd0 con 0x7f29f415b060 2023-08-25T03:34:23.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.682+0000 7f29fa389700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f415bf20 con 0x7f29f40cd480 2023-08-25T03:34:23.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.682+0000 7f29fa389700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29f415c200 con 0x7f29f40cb5a0 2023-08-25T03:34:23.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.682+0000 7f29f8926700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29f415b060 0x7f29f415d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:23.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.682+0000 7f29f8926700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29f415b060 0x7f29f415d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59990/0 (socket says 172.21.15.2:59990) 2023-08-25T03:34:23.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.682+0000 7f29f8926700 1 -- 172.21.15.2:0/3698350461 learned_addr learned my addr 172.21.15.2:0/3698350461 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:23.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.682+0000 7f29f3fff700 1 --2- 172.21.15.2:0/3698350461 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f29f40cb5a0 0x7f29f40ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:23.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.682+0000 7f29f37fe700 1 --2- 172.21.15.2:0/3698350461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29f40cd480 0x7f29f415cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:23.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.682+0000 7f29f8926700 1 -- 172.21.15.2:0/3698350461 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f29f40cb5a0 msgr2=0x7f29f40ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:23.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.682+0000 7f29f8926700 1 --2- 172.21.15.2:0/3698350461 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f29f40cb5a0 0x7f29f40ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.682+0000 7f29f8926700 1 -- 172.21.15.2:0/3698350461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29f40cd480 msgr2=0x7f29f415cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:23.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.682+0000 7f29f8926700 1 --2- 172.21.15.2:0/3698350461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29f40cd480 0x7f29f415cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.682+0000 7f29f8926700 1 -- 172.21.15.2:0/3698350461 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e80094a0 con 0x7f29f415b060 2023-08-25T03:34:23.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.683+0000 7f29f8926700 1 --2- 172.21.15.2:0/3698350461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29f415b060 0x7f29f415d220 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f29ec00cb40 tx=0x7f29ec00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:23.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.683+0000 7f29f0ff9700 1 -- 172.21.15.2:0/3698350461 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ec00ab50 con 0x7f29f415b060 2023-08-25T03:34:23.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.683+0000 7f29f0ff9700 1 -- 172.21.15.2:0/3698350461 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ec00acb0 con 0x7f29f415b060 2023-08-25T03:34:23.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.683+0000 7f29fa389700 1 -- 172.21.15.2:0/3698350461 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e40094a0 con 0x7f29f415b060 2023-08-25T03:34:23.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.683+0000 7f29f0ff9700 1 -- 172.21.15.2:0/3698350461 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ec012620 con 0x7f29f415b060 2023-08-25T03:34:23.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.683+0000 7f29fa389700 1 -- 172.21.15.2:0/3698350461 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29f415c7c0 con 0x7f29f415b060 2023-08-25T03:34:23.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.684+0000 7f29f0ff9700 1 -- 172.21.15.2:0/3698350461 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f29ec01b3b0 con 0x7f29f415b060 2023-08-25T03:34:23.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.684+0000 7f29f0ff9700 1 --2- 172.21.15.2:0/3698350461 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f29e006a0e0 0x7f29e006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:23.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.685+0000 7f29f0ff9700 1 -- 172.21.15.2:0/3698350461 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f29ec088370 con 0x7f29f415b060 2023-08-25T03:34:23.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.685+0000 7f29f3fff700 1 --2- 172.21.15.2:0/3698350461 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f29e006a0e0 0x7f29e006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:23.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.685+0000 7f29f3fff700 1 --2- 172.21.15.2:0/3698350461 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f29e006a0e0 0x7f29e006c5a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f29e4000c00 tx=0x7f29e401c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:23.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.687+0000 7f2a0652db80 1 Processor -- start 2023-08-25T03:34:23.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.687+0000 7f2a0652db80 1 -- start start 2023-08-25T03:34:23.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.687+0000 7f2a0652db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea14fcd60 0x556ea14fd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:23.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.687+0000 7f2a0652db80 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x556ea14faf80 0x556ea15241d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:23.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.687+0000 7f2a0652db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea1543da0 0x556ea1524710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:23.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.687+0000 7f2a0652db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea1464760 con 0x556ea14fcd60 2023-08-25T03:34:23.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.687+0000 7f2a0652db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea1455730 con 0x556ea1543da0 2023-08-25T03:34:23.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.687+0000 7f2a0652db80 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ea1453050 con 0x556ea14faf80 2023-08-25T03:34:23.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.687+0000 7f29f3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea14fcd60 0x556ea14fd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:23.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.687+0000 7f29f3fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea14fcd60 0x556ea14fd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:59994/0 (socket says 172.21.15.2:59994) 2023-08-25T03:34:23.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.688+0000 7f29f3fff700 1 -- 172.21.15.2:0/1948931964 learned_addr learned my addr 172.21.15.2:0/1948931964 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:23.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.688+0000 7f29f37fe700 1 --2- 172.21.15.2:0/1948931964 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x556ea14faf80 0x556ea15241d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:23.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.688+0000 7f29f3fff700 1 -- 172.21.15.2:0/1948931964 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x556ea14faf80 msgr2=0x556ea15241d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:23.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.688+0000 7f29f3fff700 1 --2- 172.21.15.2:0/1948931964 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x556ea14faf80 0x556ea15241d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.688+0000 7f29f3fff700 1 -- 172.21.15.2:0/1948931964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea1543da0 msgr2=0x556ea1524710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:23.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.688+0000 7f29f8926700 1 --2- 172.21.15.2:0/1948931964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea1543da0 0x556ea1524710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:23.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.688+0000 7f29f3fff700 1 --2- 172.21.15.2:0/1948931964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea1543da0 0x556ea1524710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.688+0000 7f29f3fff700 1 -- 172.21.15.2:0/1948931964 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ea12ce5e0 con 0x556ea14fcd60 2023-08-25T03:34:23.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.688+0000 7f29f8926700 1 --2- 172.21.15.2:0/1948931964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea1543da0 0x556ea1524710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:23.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.688+0000 7f29f3fff700 1 --2- 172.21.15.2:0/1948931964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea14fcd60 0x556ea14fd140 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f29e4009ea0 tx=0x7f29e4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beae969675895bc9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:23.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.688+0000 7f29d9ffb700 1 -- 172.21.15.2:0/1948931964 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e40036a0 con 0x556ea14fcd60 2023-08-25T03:34:23.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.688+0000 7f29d9ffb700 1 -- 172.21.15.2:0/1948931964 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e4003800 con 0x556ea14fcd60 2023-08-25T03:34:23.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.688+0000 7f29d9ffb700 1 -- 172.21.15.2:0/1948931964 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e4003b90 con 0x556ea14fcd60 2023-08-25T03:34:23.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.689+0000 7f2a0652db80 1 -- 172.21.15.2:0/1948931964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea14fcd60 msgr2=0x556ea14fd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:23.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.689+0000 7f2a0652db80 1 --2- 172.21.15.2:0/1948931964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea14fcd60 0x556ea14fd140 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f29e4009ea0 tx=0x7f29e4010000 comp rx=0 tx=0).stop 2023-08-25T03:34:23.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.689+0000 7f2a0652db80 1 -- 172.21.15.2:0/1948931964 shutdown_connections 2023-08-25T03:34:23.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.689+0000 7f2a0652db80 1 --2- 172.21.15.2:0/1948931964 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea14fcd60 0x556ea14fd140 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.689+0000 7f2a0652db80 1 --2- 172.21.15.2:0/1948931964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea1543da0 0x556ea1524710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.689+0000 7f2a0652db80 1 --2- 172.21.15.2:0/1948931964 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x556ea14faf80 0x556ea15241d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.689+0000 7f2a0652db80 1 -- 172.21.15.2:0/1948931964 >> 172.21.15.2:0/1948931964 conn(0x556ea1471090 msgr2=0x556ea1542920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:23.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.689+0000 7f2a0652db80 1 -- 172.21.15.2:0/1948931964 shutdown_connections 2023-08-25T03:34:23.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.689+0000 7f2a0652db80 1 -- 172.21.15.2:0/1948931964 wait complete. 2023-08-25T03:34:23.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.690+0000 7f2a0652db80 1 -- start start 2023-08-25T03:34:23.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.690+0000 7f2a0652db80 1 Processor -- start 2023-08-25T03:34:23.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.691+0000 7f2a0652db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea14faf80 0x556ea153f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:23.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.691+0000 7f2a0652db80 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x556ea14fcd60 0x556ea15208c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:23.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.691+0000 7f2a0652db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea1543da0 0x556ea153bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:23.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.691+0000 7f2a0652db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea144e940 con 0x556ea1543da0 2023-08-25T03:34:23.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.691+0000 7f2a0652db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea13ecb60 con 0x556ea14faf80 2023-08-25T03:34:23.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.691+0000 7f2a0652db80 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ea13ec860 con 0x556ea14fcd60 2023-08-25T03:34:23.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.691+0000 7f2a0652db80 10 client.0 _ll_register_callbacks cb 0x556ea1336a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T03:34:23.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.691+0000 7f29f8926700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea1543da0 0x556ea153bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:23.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.691+0000 7f29f8926700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea1543da0 0x556ea153bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60004/0 (socket says 172.21.15.2:60004) 2023-08-25T03:34:23.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.691+0000 7f29f8926700 1 -- 172.21.15.2:0/1082575780 learned_addr learned my addr 172.21.15.2:0/1082575780 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:23.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.691+0000 7f29f37fe700 1 --2- 172.21.15.2:0/1082575780 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x556ea14fcd60 0x556ea15208c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:23.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.691+0000 7f29f3fff700 1 --2- 172.21.15.2:0/1082575780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea14faf80 0x556ea153f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:23.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.691+0000 7f29f8926700 1 -- 172.21.15.2:0/1082575780 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x556ea14fcd60 msgr2=0x556ea15208c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:23.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.691+0000 7f29f8926700 1 --2- 172.21.15.2:0/1082575780 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x556ea14fcd60 0x556ea15208c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.691+0000 7f29f8926700 1 -- 172.21.15.2:0/1082575780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea14faf80 msgr2=0x556ea153f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:23.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.691+0000 7f29f8926700 1 --2- 172.21.15.2:0/1082575780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea14faf80 0x556ea153f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.691+0000 7f29f8926700 1 -- 172.21.15.2:0/1082575780 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29f40b2a90 con 0x556ea1543da0 2023-08-25T03:34:23.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.692+0000 7f29f8926700 1 --2- 172.21.15.2:0/1082575780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea1543da0 0x556ea153bd50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f29ec0866d0 tx=0x7f29ec057e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=758b481040250716 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:23.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.692+0000 7f29d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-08-25T03:34:23.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.692+0000 7f29d2ffd700 1 -- 172.21.15.2:0/1082575780 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ec05f420 con 0x556ea1543da0 2023-08-25T03:34:23.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.692+0000 7f29d2ffd700 1 -- 172.21.15.2:0/1082575780 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f29ec05ef90 con 0x556ea1543da0 2023-08-25T03:34:23.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.692+0000 7f2a0652db80 1 -- 172.21.15.2:0/1082575780 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ea14efe00 con 0x556ea1543da0 2023-08-25T03:34:23.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.692+0000 7f29d2ffd700 1 -- 172.21.15.2:0/1082575780 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29ec05f150 con 0x556ea1543da0 2023-08-25T03:34:23.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.692+0000 7f2a0652db80 10 client.4719 fetch_fsmap learned FSMap version 8 2023-08-25T03:34:23.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.692+0000 7f2a0652db80 1 -- 172.21.15.2:0/1082575780 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x556ea127ad90 con 0x556ea1543da0 2023-08-25T03:34:23.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.692+0000 7f29d2ffd700 1 -- 172.21.15.2:0/1082575780 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x556ea14efe00 con 0x556ea1543da0 2023-08-25T03:34:23.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.692+0000 7f29d2ffd700 1 -- 172.21.15.2:0/1082575780 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f29ec0861a0 con 0x556ea1543da0 2023-08-25T03:34:23.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.692+0000 7f2a0652db80 10 client.4719 fetch_fsmap finished waiting for FSMap version 8 2023-08-25T03:34:23.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.692+0000 7f2a0652db80 10 client.4719 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-25T03:34:23.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.692+0000 7f2a0652db80 20 client.4719 populate_metadata read hostname 'smithi002' 2023-08-25T03:34:23.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.692+0000 7f2a0652db80 1 --2- 172.21.15.2:0/1082575780 >> [v2:172.21.15.2:6834/193990892,v1:172.21.15.2:6835/193990892] conn(0x556ea1539f20 0x556ea153ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:23.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.692+0000 7f2a0652db80 4 client.4719 mds_command: new command op to 4299 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T03:34:23.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.692+0000 7f2a0652db80 1 -- 172.21.15.2:0/1082575780 --> [v2:172.21.15.2:6834/193990892,v1:172.21.15.2:6835/193990892] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ea13f4b50 con 0x556ea1539f20 2023-08-25T03:34:23.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.692+0000 7f29f3fff700 1 --2- 172.21.15.2:0/1082575780 >> [v2:172.21.15.2:6834/193990892,v1:172.21.15.2:6835/193990892] conn(0x556ea1539f20 0x556ea153ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:23.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.693+0000 7f29d2ffd700 10 client.4719 ms_handle_connect on v2:172.21.15.2:6834/193990892 2023-08-25T03:34:23.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.693+0000 7f29f3fff700 1 --2- 172.21.15.2:0/1082575780 >> [v2:172.21.15.2:6834/193990892,v1:172.21.15.2:6835/193990892] conn(0x556ea1539f20 0x556ea153ccc0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=752bdc34505e451c server_cookie=35802c180da1584 in_seq=0 out_seq=0 2023-08-25T03:34:23.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.693+0000 7f29d2ffd700 1 -- 172.21.15.2:0/1082575780 <== mds.0 v2:172.21.15.2:6834/193990892 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x556ea13f4b50 con 0x556ea1539f20 2023-08-25T03:34:23.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.693+0000 7f29d2ffd700 10 client.4719 handle_command_reply: tid=0 2023-08-25T03:34:23.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.693+0000 7f2a0652db80 1 client.4719 shutdown 2023-08-25T03:34:23.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.694+0000 7f2a0652db80 1 -- 172.21.15.2:0/1082575780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea1543da0 msgr2=0x556ea153bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:23.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.694+0000 7f2a0652db80 1 --2- 172.21.15.2:0/1082575780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea1543da0 0x556ea153bd50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f29ec0866d0 tx=0x7f29ec057e70 comp rx=0 tx=0).stop 2023-08-25T03:34:23.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.694+0000 7f2a0652db80 1 -- 172.21.15.2:0/1082575780 shutdown_connections 2023-08-25T03:34:23.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.694+0000 7f2a0652db80 1 --2- 172.21.15.2:0/1082575780 >> [v2:172.21.15.2:6834/193990892,v1:172.21.15.2:6835/193990892] conn(0x556ea1539f20 0x556ea153ccc0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.694+0000 7f2a0652db80 1 --2- 172.21.15.2:0/1082575780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea1543da0 0x556ea153bd50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.694+0000 7f2a0652db80 1 --2- 172.21.15.2:0/1082575780 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea14faf80 0x556ea153f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.694+0000 7f29d2ffd700 0 client.4719 ms_handle_reset on v2:172.21.15.2:6834/193990892 2023-08-25T03:34:23.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.694+0000 7f2a0652db80 1 --2- 172.21.15.2:0/1082575780 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x556ea14fcd60 0x556ea15208c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.694+0000 7f2a0652db80 1 -- 172.21.15.2:0/1082575780 >> 172.21.15.2:0/1082575780 conn(0x556ea1471090 msgr2=0x556ea14fd140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:23.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.694+0000 7f2a0652db80 1 -- 172.21.15.2:0/1082575780 shutdown_connections 2023-08-25T03:34:23.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.694+0000 7f2a0652db80 1 -- 172.21.15.2:0/1082575780 wait complete. 2023-08-25T03:34:23.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.694+0000 7f2a0652db80 20 client.4719 trim_cache size 0 max 16384 2023-08-25T03:34:23.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.712+0000 7f2a0652db80 1 Processor -- start 2023-08-25T03:34:23.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.712+0000 7f2a0652db80 1 -- start start 2023-08-25T03:34:23.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.713+0000 7f2a0652db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea14faf80 0x556ea1546440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:23.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.713+0000 7f2a0652db80 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x556ea14fcd60 0x556ea1522fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:23.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.713+0000 7f2a0652db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea1543da0 0x556ea15234e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:23.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.713+0000 7f2a0652db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea12d08d0 con 0x556ea1543da0 2023-08-25T03:34:23.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.713+0000 7f2a0652db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea15424a0 con 0x556ea14faf80 2023-08-25T03:34:23.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.713+0000 7f2a0652db80 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ea15426b0 con 0x556ea14fcd60 2023-08-25T03:34:23.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.713+0000 7f29f8926700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea1543da0 0x556ea15234e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:23.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.713+0000 7f29f8926700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea1543da0 0x556ea15234e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60010/0 (socket says 172.21.15.2:60010) 2023-08-25T03:34:23.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.713+0000 7f29f8926700 1 -- 172.21.15.2:0/1007094458 learned_addr learned my addr 172.21.15.2:0/1007094458 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:23.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.713+0000 7f29f3fff700 1 --2- 172.21.15.2:0/1007094458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea14faf80 0x556ea1546440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:23.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.713+0000 7f29f37fe700 1 --2- 172.21.15.2:0/1007094458 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x556ea14fcd60 0x556ea1522fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:23.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.713+0000 7f29f8926700 1 -- 172.21.15.2:0/1007094458 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x556ea14fcd60 msgr2=0x556ea1522fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:23.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.713+0000 7f29f8926700 1 --2- 172.21.15.2:0/1007094458 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x556ea14fcd60 0x556ea1522fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.713+0000 7f29f8926700 1 -- 172.21.15.2:0/1007094458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea14faf80 msgr2=0x556ea1546440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:23.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.713+0000 7f29f8926700 1 --2- 172.21.15.2:0/1007094458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea14faf80 0x556ea1546440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.713+0000 7f29f8926700 1 -- 172.21.15.2:0/1007094458 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ea1263c10 con 0x556ea1543da0 2023-08-25T03:34:23.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.713+0000 7f29f8926700 1 --2- 172.21.15.2:0/1007094458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea1543da0 0x556ea15234e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f29ec00cb10 tx=0x7f29ec014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=708516a0210e25e9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:23.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.713+0000 7f29da7fc700 1 -- 172.21.15.2:0/1007094458 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ec053a50 con 0x556ea1543da0 2023-08-25T03:34:23.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.714+0000 7f29da7fc700 1 -- 172.21.15.2:0/1007094458 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ec05f420 con 0x556ea1543da0 2023-08-25T03:34:23.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.714+0000 7f29da7fc700 1 -- 172.21.15.2:0/1007094458 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ec05ef90 con 0x556ea1543da0 2023-08-25T03:34:23.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.714+0000 7f2a0652db80 1 -- 172.21.15.2:0/1007094458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea1543da0 msgr2=0x556ea15234e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:23.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.714+0000 7f2a0652db80 1 --2- 172.21.15.2:0/1007094458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea1543da0 0x556ea15234e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f29ec00cb10 tx=0x7f29ec014000 comp rx=0 tx=0).stop 2023-08-25T03:34:23.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.714+0000 7f2a0652db80 1 -- 172.21.15.2:0/1007094458 shutdown_connections 2023-08-25T03:34:23.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.714+0000 7f2a0652db80 1 --2- 172.21.15.2:0/1007094458 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea1543da0 0x556ea15234e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.714+0000 7f2a0652db80 1 --2- 172.21.15.2:0/1007094458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea14faf80 0x556ea1546440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.714+0000 7f2a0652db80 1 --2- 172.21.15.2:0/1007094458 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x556ea14fcd60 0x556ea1522fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.714+0000 7f2a0652db80 1 -- 172.21.15.2:0/1007094458 >> 172.21.15.2:0/1007094458 conn(0x556ea1471090 msgr2=0x556ea1568eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:23.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.714+0000 7f2a0652db80 1 -- 172.21.15.2:0/1007094458 shutdown_connections 2023-08-25T03:34:23.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.714+0000 7f2a0652db80 1 -- 172.21.15.2:0/1007094458 wait complete. 2023-08-25T03:34:23.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.715+0000 7f2a0652db80 1 -- start start 2023-08-25T03:34:23.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.715+0000 7f2a0652db80 1 Processor -- start 2023-08-25T03:34:23.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.716+0000 7f2a0652db80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea14faf80 0x556ea15ad810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:23.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.716+0000 7f2a0652db80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea14fcd60 0x556ea1523310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:23.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.716+0000 7f2a0652db80 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x556ea1543da0 0x556ea14f6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:23.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.716+0000 7f2a0652db80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea1523850 con 0x556ea14fcd60 2023-08-25T03:34:23.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.716+0000 7f2a0652db80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ea15239c0 con 0x556ea14faf80 2023-08-25T03:34:23.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.716+0000 7f2a0652db80 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ea15ae880 con 0x556ea1543da0 2023-08-25T03:34:23.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.716+0000 7f2a0652db80 10 client.0 _ll_register_callbacks cb 0x7f29ec057120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T03:34:23.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.716+0000 7f29f37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea14fcd60 0x556ea1523310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:23.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.716+0000 7f29f37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea14fcd60 0x556ea1523310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60016/0 (socket says 172.21.15.2:60016) 2023-08-25T03:34:23.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.716+0000 7f29f37fe700 1 -- 172.21.15.2:0/1850849891 learned_addr learned my addr 172.21.15.2:0/1850849891 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:23.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.716+0000 7f29f3fff700 1 --2- 172.21.15.2:0/1850849891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea14faf80 0x556ea15ad810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:23.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.716+0000 7f29f8926700 1 --2- 172.21.15.2:0/1850849891 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x556ea1543da0 0x556ea14f6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:23.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.716+0000 7f29f37fe700 1 -- 172.21.15.2:0/1850849891 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x556ea1543da0 msgr2=0x556ea14f6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:23.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.716+0000 7f29f37fe700 1 --2- 172.21.15.2:0/1850849891 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x556ea1543da0 0x556ea14f6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.716+0000 7f29f37fe700 1 -- 172.21.15.2:0/1850849891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea14faf80 msgr2=0x556ea15ad810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:23.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.716+0000 7f29f37fe700 1 --2- 172.21.15.2:0/1850849891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea14faf80 0x556ea15ad810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.716+0000 7f29f37fe700 1 -- 172.21.15.2:0/1850849891 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ea127ad90 con 0x556ea14fcd60 2023-08-25T03:34:23.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.716+0000 7f29f37fe700 1 --2- 172.21.15.2:0/1850849891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea14fcd60 0x556ea1523310 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f29f40cb9e0 tx=0x7f29e8017720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2bbf333f7cb4145 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:23.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.716+0000 7f29d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-08-25T03:34:23.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.716+0000 7f29d2ffd700 1 -- 172.21.15.2:0/1850849891 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e801c070 con 0x556ea14fcd60 2023-08-25T03:34:23.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.717+0000 7f29d2ffd700 1 -- 172.21.15.2:0/1850849891 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f29e8022c30 con 0x556ea14fcd60 2023-08-25T03:34:23.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.717+0000 7f2a0652db80 1 -- 172.21.15.2:0/1850849891 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29e4003800 con 0x556ea14fcd60 2023-08-25T03:34:23.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.717+0000 7f29d2ffd700 1 -- 172.21.15.2:0/1850849891 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29e8022df0 con 0x556ea14fcd60 2023-08-25T03:34:23.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.717+0000 7f2a0652db80 10 client.4725 fetch_fsmap learned FSMap version 8 2023-08-25T03:34:23.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.717+0000 7f2a0652db80 1 -- 172.21.15.2:0/1850849891 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x556ea12a1200 con 0x556ea14fcd60 2023-08-25T03:34:23.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.717+0000 7f29d2ffd700 1 -- 172.21.15.2:0/1850849891 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e4003800 con 0x556ea14fcd60 2023-08-25T03:34:23.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.717+0000 7f29d2ffd700 1 -- 172.21.15.2:0/1850849891 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 8) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f29e8004240 con 0x556ea14fcd60 2023-08-25T03:34:23.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.717+0000 7f2a0652db80 10 client.4725 fetch_fsmap finished waiting for FSMap version 8 2023-08-25T03:34:23.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.717+0000 7f2a0652db80 10 client.4725 resolve_mds: resolved name 'a' to daemon mds.a 2023-08-25T03:34:23.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.717+0000 7f2a0652db80 20 client.4725 populate_metadata read hostname 'smithi002' 2023-08-25T03:34:23.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.717+0000 7f2a0652db80 1 --2- 172.21.15.2:0/1850849891 >> [v2:172.21.15.2:6834/193990892,v1:172.21.15.2:6835/193990892] conn(0x556ea1555680 0x556ea1555a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:23.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.717+0000 7f2a0652db80 4 client.4725 mds_command: new command op to 4299 tid=0[{"prefix": "status"}] 2023-08-25T03:34:23.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.717+0000 7f2a0652db80 1 -- 172.21.15.2:0/1850849891 --> [v2:172.21.15.2:6834/193990892,v1:172.21.15.2:6835/193990892] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556ea1447920 con 0x556ea1555680 2023-08-25T03:34:23.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.717+0000 7f29f3fff700 1 --2- 172.21.15.2:0/1850849891 >> [v2:172.21.15.2:6834/193990892,v1:172.21.15.2:6835/193990892] conn(0x556ea1555680 0x556ea1555a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:23.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.717+0000 7f29d2ffd700 10 client.4725 ms_handle_connect on v2:172.21.15.2:6834/193990892 2023-08-25T03:34:23.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.717+0000 7f29f3fff700 1 --2- 172.21.15.2:0/1850849891 >> [v2:172.21.15.2:6834/193990892,v1:172.21.15.2:6835/193990892] conn(0x556ea1555680 0x556ea1555a60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49b5cfdfd7eec74f server_cookie=32a97cd9dfe25c48 in_seq=0 out_seq=0 2023-08-25T03:34:23.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.718+0000 7f29d2ffd700 1 -- 172.21.15.2:0/1850849891 <== mds.0 v2:172.21.15.2:6834/193990892 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x556ea1447920 con 0x556ea1555680 2023-08-25T03:34:23.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.718+0000 7f29d2ffd700 10 client.4725 handle_command_reply: tid=0 2023-08-25T03:34:23.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.718+0000 7f2a0652db80 1 client.4725 shutdown 2023-08-25T03:34:23.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.718+0000 7f2a0652db80 1 -- 172.21.15.2:0/1850849891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea14fcd60 msgr2=0x556ea1523310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:23.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.718+0000 7f2a0652db80 1 --2- 172.21.15.2:0/1850849891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea14fcd60 0x556ea1523310 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f29f40cb9e0 tx=0x7f29e8017720 comp rx=0 tx=0).stop 2023-08-25T03:34:23.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.718+0000 7f2a0652db80 1 -- 172.21.15.2:0/1850849891 shutdown_connections 2023-08-25T03:34:23.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.718+0000 7f2a0652db80 1 --2- 172.21.15.2:0/1850849891 >> [v2:172.21.15.2:6834/193990892,v1:172.21.15.2:6835/193990892] conn(0x556ea1555680 0x556ea1555a60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.718+0000 7f2a0652db80 1 --2- 172.21.15.2:0/1850849891 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x556ea1543da0 0x556ea14f6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.718+0000 7f2a0652db80 1 --2- 172.21.15.2:0/1850849891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x556ea14faf80 0x556ea15ad810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.718+0000 7f2a0652db80 1 --2- 172.21.15.2:0/1850849891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x556ea14fcd60 0x556ea1523310 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.718+0000 7f29d2ffd700 0 client.4725 ms_handle_reset on v2:172.21.15.2:6834/193990892 2023-08-25T03:34:23.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.718+0000 7f2a0652db80 1 -- 172.21.15.2:0/1850849891 >> 172.21.15.2:0/1850849891 conn(0x556ea1471090 msgr2=0x556ea153a800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:23.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.718+0000 7f2a0652db80 1 -- 172.21.15.2:0/1850849891 shutdown_connections 2023-08-25T03:34:23.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.718+0000 7f2a0652db80 1 -- 172.21.15.2:0/1850849891 wait complete. 2023-08-25T03:34:23.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.718+0000 7f2a0652db80 20 client.4725 trim_cache size 0 max 16384 2023-08-25T03:34:23.729 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-08-25T03:34:23.729 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "dc54136d-3683-431b-8a68-994ec618505c", 2023-08-25T03:34:23.729 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 0, 2023-08-25T03:34:23.729 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 4299, 2023-08-25T03:34:23.729 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2023-08-25T03:34:23.730 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2023-08-25T03:34:23.730 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2023-08-25T03:34:23.730 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 20.114142337000001, 2023-08-25T03:34:23.730 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 8, 2023-08-25T03:34:23.730 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 24, 2023-08-25T03:34:23.730 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 24, 2023-08-25T03:34:23.730 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 26.039463212000001 2023-08-25T03:34:23.730 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-08-25T03:34:23.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.718+0000 7f29fa389700 1 -- 172.21.15.2:0/3698350461 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f29e006a0e0 msgr2=0x7f29e006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:23.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.718+0000 7f29fa389700 1 --2- 172.21.15.2:0/3698350461 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f29e006a0e0 0x7f29e006c5a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f29e4000c00 tx=0x7f29e401c040 comp rx=0 tx=0).stop 2023-08-25T03:34:23.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.718+0000 7f29fa389700 1 -- 172.21.15.2:0/3698350461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29f415b060 msgr2=0x7f29f415d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:23.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.718+0000 7f29fa389700 1 --2- 172.21.15.2:0/3698350461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29f415b060 0x7f29f415d220 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f29ec00cb40 tx=0x7f29ec00cf00 comp rx=0 tx=0).stop 2023-08-25T03:34:23.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.719+0000 7f29fa389700 1 -- 172.21.15.2:0/3698350461 shutdown_connections 2023-08-25T03:34:23.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.719+0000 7f29fa389700 1 --2- 172.21.15.2:0/3698350461 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f29e006a0e0 0x7f29e006c5a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.719+0000 7f29fa389700 1 --2- 172.21.15.2:0/3698350461 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f29f415b060 0x7f29f415d220 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.719+0000 7f29fa389700 1 --2- 172.21.15.2:0/3698350461 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f29f40cb5a0 0x7f29f40ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.719+0000 7f29fa389700 1 --2- 172.21.15.2:0/3698350461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29f40cd480 0x7f29f415cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:23.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.719+0000 7f29fa389700 1 -- 172.21.15.2:0/3698350461 >> 172.21.15.2:0/3698350461 conn(0x7f29f40d0f70 msgr2=0x7f29f4159cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:23.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.719+0000 7f29fa389700 1 -- 172.21.15.2:0/3698350461 shutdown_connections 2023-08-25T03:34:23.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:23.719+0000 7f29fa389700 1 -- 172.21.15.2:0/3698350461 wait complete. 2023-08-25T03:34:23.734 INFO:teuthology.run_tasks:Running task kclient... 2023-08-25T03:34:23.756 INFO:tasks.kclient:Mounting kernel clients... 2023-08-25T03:34:23.757 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-08-25T03:34:23.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:23.758 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2023-08-25T03:34:23.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:23.776 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link delete ceph-brx 2023-08-25T03:34:23.845 INFO:teuthology.orchestra.run.smithi002.stderr:Cannot find device "ceph-brx" 2023-08-25T03:34:23.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T03:34:23.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:23.847 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2023-08-25T03:34:23.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:23.904 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link delete ceph-brx 2023-08-25T03:34:23.971 INFO:teuthology.orchestra.run.smithi002.stderr:Cannot find device "ceph-brx" 2023-08-25T03:34:23.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T03:34:23.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:23.973 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-08-25T03:34:23.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:23.996 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2023-08-25T03:34:24.074 INFO:teuthology.orchestra.run.smithi031.stderr:Cannot find device "ceph-brx" 2023-08-25T03:34:24.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T03:34:24.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:24.076 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-08-25T03:34:24.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:24.137 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2023-08-25T03:34:24.208 INFO:teuthology.orchestra.run.smithi031.stderr:Cannot find device "ceph-brx" 2023-08-25T03:34:24.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T03:34:24.210 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2023-08-25T03:34:24.210 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-25T03:34:24.211 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-25T03:34:24.211 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T03:34:24.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.338+0000 7f15057e3700 1 Processor -- start 2023-08-25T03:34:24.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.338+0000 7f15057e3700 1 -- start start 2023-08-25T03:34:24.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.339+0000 7f15057e3700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f15000d24f0 0x7f15000d1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:24.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.339+0000 7f15057e3700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15000d3000 con 0x7f15000b5330 2023-08-25T03:34:24.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.339+0000 7f15057e3700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15000ce430 con 0x7f150015e980 2023-08-25T03:34:24.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.339+0000 7f15057e3700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f150006c7c0 con 0x7f15000d24f0 2023-08-25T03:34:24.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.339+0000 7f14fe7fc700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f15000b5330 0x7f150015ef10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46690/0 (socket says 172.21.15.2:46690) 2023-08-25T03:34:24.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.339+0000 7f14fe7fc700 1 -- 172.21.15.2:0/208137614 learned_addr learned my addr 172.21.15.2:0/208137614 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:24.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.339+0000 7f14ff7fe700 1 --2- 172.21.15.2:0/208137614 >> v2:172.21.15.31:3300/0 conn(0x7f15000d24f0 0x7f15000d1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:24.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.339+0000 7f14fdffb700 1 -- 172.21.15.2:0/208137614 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1915110240 0 0) 0x7f14f4001110 con 0x7f15000b5330 2023-08-25T03:34:24.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.339+0000 7f14fdffb700 1 -- 172.21.15.2:0/208137614 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14e80035b0 con 0x7f15000b5330 2023-08-25T03:34:24.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.339+0000 7f14fdffb700 1 -- 172.21.15.2:0/208137614 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2485257738 0 0) 0x7f15000ce430 con 0x7f150015e980 2023-08-25T03:34:24.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.339+0000 7f14fdffb700 1 -- 172.21.15.2:0/208137614 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14f4001110 con 0x7f150015e980 2023-08-25T03:34:24.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.339+0000 7f14fdffb700 1 -- 172.21.15.2:0/208137614 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 155091263 0 0) 0x7f14e80035b0 con 0x7f15000b5330 2023-08-25T03:34:24.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.339+0000 7f14ff7fe700 1 -- 172.21.15.2:0/208137614 >> v1:172.21.15.31:6789/0 conn(0x7f150015e980 legacy=0x7f15000b4c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:24.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.339+0000 7f14ff7fe700 1 -- 172.21.15.2:0/208137614 >> v1:172.21.15.2:6789/0 conn(0x7f15000b5330 legacy=0x7f150015ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:24.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.339+0000 7f14ff7fe700 1 -- 172.21.15.2:0/208137614 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15000d0a70 con 0x7f15000d24f0 2023-08-25T03:34:24.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.340+0000 7f14ff7fe700 1 --2- 172.21.15.2:0/208137614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15000d24f0 0x7f15000d1f30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009d90 tx=0x7f14f800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cc656f86fd80527 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:24.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.340+0000 7f14fdffb700 1 -- 172.21.15.2:0/208137614 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f800e070 con 0x7f15000d24f0 2023-08-25T03:34:24.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.340+0000 7f14fdffb700 1 -- 172.21.15.2:0/208137614 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f8003a20 con 0x7f15000d24f0 2023-08-25T03:34:24.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.340+0000 7f14fdffb700 1 -- 172.21.15.2:0/208137614 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f4001800 con 0x7f15000d24f0 2023-08-25T03:34:24.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.340+0000 7f15057e3700 1 -- 172.21.15.2:0/208137614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15000d24f0 msgr2=0x7f15000d1f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:24.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.340+0000 7f15057e3700 1 --2- 172.21.15.2:0/208137614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15000d24f0 0x7f15000d1f30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009d90 tx=0x7f14f800b330 comp rx=0 tx=0).stop 2023-08-25T03:34:24.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.340+0000 7f15057e3700 1 -- 172.21.15.2:0/208137614 shutdown_connections 2023-08-25T03:34:24.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.340+0000 7f15057e3700 1 --2- 172.21.15.2:0/208137614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15000d24f0 0x7f15000d1f30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:24.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.340+0000 7f15057e3700 1 -- 172.21.15.2:0/208137614 >> 172.21.15.2:0/208137614 conn(0x7f15000cfc20 msgr2=0x7f15000d0030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:24.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.340+0000 7f15057e3700 1 -- 172.21.15.2:0/208137614 shutdown_connections 2023-08-25T03:34:24.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.340+0000 7f15057e3700 1 -- 172.21.15.2:0/208137614 wait complete. 2023-08-25T03:34:24.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.342+0000 7f15057e3700 1 Processor -- start 2023-08-25T03:34:24.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.342+0000 7f15057e3700 1 -- start start 2023-08-25T03:34:24.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.343+0000 7f15057e3700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f15000b5330 0x7f150014a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:24.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.343+0000 7f15057e3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15000d24f0 0x7f150014b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:24.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.343+0000 7f15057e3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f150015e980 0x7f15000c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:24.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.343+0000 7f15057e3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15000caea0 con 0x7f15000d24f0 2023-08-25T03:34:24.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.343+0000 7f15057e3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15000cb180 con 0x7f150015e980 2023-08-25T03:34:24.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.343+0000 7f15057e3700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15000cb460 con 0x7f15000b5330 2023-08-25T03:34:24.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.343+0000 7f14fe7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15000d24f0 0x7f150014b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:24.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.343+0000 7f14fe7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15000d24f0 0x7f150014b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60030/0 (socket says 172.21.15.2:60030) 2023-08-25T03:34:24.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.343+0000 7f14fe7fc700 1 -- 172.21.15.2:0/759658451 learned_addr learned my addr 172.21.15.2:0/759658451 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:24.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.343+0000 7f14ff7fe700 1 --2- 172.21.15.2:0/759658451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f150015e980 0x7f15000c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:24.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.343+0000 7f14feffd700 1 --2- 172.21.15.2:0/759658451 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f15000b5330 0x7f150014a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:24.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.344+0000 7f14fe7fc700 1 -- 172.21.15.2:0/759658451 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f15000b5330 msgr2=0x7f150014a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:24.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.344+0000 7f14fe7fc700 1 --2- 172.21.15.2:0/759658451 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f15000b5330 0x7f150014a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:24.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.344+0000 7f14fe7fc700 1 -- 172.21.15.2:0/759658451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f150015e980 msgr2=0x7f15000c8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:24.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.344+0000 7f14fe7fc700 1 --2- 172.21.15.2:0/759658451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f150015e980 0x7f15000c8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:24.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.344+0000 7f14fe7fc700 1 -- 172.21.15.2:0/759658451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14f8009a70 con 0x7f15000d24f0 2023-08-25T03:34:24.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.344+0000 7f14fe7fc700 1 --2- 172.21.15.2:0/759658451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15000d24f0 0x7f150014b000 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f14f400cad0 tx=0x7f14f400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:24.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.344+0000 7f14e77fe700 1 -- 172.21.15.2:0/759658451 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f400ce50 con 0x7f15000d24f0 2023-08-25T03:34:24.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.344+0000 7f14e77fe700 1 -- 172.21.15.2:0/759658451 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f4004a80 con 0x7f15000d24f0 2023-08-25T03:34:24.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.344+0000 7f14e77fe700 1 -- 172.21.15.2:0/759658451 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f4004e30 con 0x7f15000d24f0 2023-08-25T03:34:24.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.344+0000 7f15057e3700 1 -- 172.21.15.2:0/759658451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15000cb740 con 0x7f15000d24f0 2023-08-25T03:34:24.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.344+0000 7f15057e3700 1 -- 172.21.15.2:0/759658451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15000c8ad0 con 0x7f15000d24f0 2023-08-25T03:34:24.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.345+0000 7f15057e3700 1 -- 172.21.15.2:0/759658451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15000d1e00 con 0x7f15000d24f0 2023-08-25T03:34:24.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.345+0000 7f14e77fe700 1 -- 172.21.15.2:0/759658451 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f14f40070d0 con 0x7f15000d24f0 2023-08-25T03:34:24.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.346+0000 7f14e77fe700 1 --2- 172.21.15.2:0/759658451 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f14ec06a0e0 0x7f14ec06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:24.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.346+0000 7f14e77fe700 1 -- 172.21.15.2:0/759658451 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f14f4087680 con 0x7f15000d24f0 2023-08-25T03:34:24.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.346+0000 7f14feffd700 1 --2- 172.21.15.2:0/759658451 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f14ec06a0e0 0x7f14ec06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:24.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.346+0000 7f14feffd700 1 --2- 172.21.15.2:0/759658451 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f14ec06a0e0 0x7f14ec06c5a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f14f0001220 tx=0x7f14f0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:24.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.350+0000 7f14e77fe700 1 -- 172.21.15.2:0/759658451 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f14f4052d70 con 0x7f15000d24f0 2023-08-25T03:34:24.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.594+0000 7f15057e3700 1 -- 172.21.15.2:0/759658451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f15000d3000 con 0x7f15000d24f0 2023-08-25T03:34:24.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.594+0000 7f14e77fe700 1 -- 172.21.15.2:0/759658451 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f15000d3000 con 0x7f15000d24f0 2023-08-25T03:34:24.595 INFO:teuthology.orchestra.run.smithi002.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T03:34:24.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.596+0000 7f15057e3700 1 -- 172.21.15.2:0/759658451 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f14ec06a0e0 msgr2=0x7f14ec06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:24.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.596+0000 7f15057e3700 1 --2- 172.21.15.2:0/759658451 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f14ec06a0e0 0x7f14ec06c5a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f14f0001220 tx=0x7f14f0009040 comp rx=0 tx=0).stop 2023-08-25T03:34:24.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.596+0000 7f15057e3700 1 -- 172.21.15.2:0/759658451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15000d24f0 msgr2=0x7f150014b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:24.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.596+0000 7f15057e3700 1 --2- 172.21.15.2:0/759658451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15000d24f0 0x7f150014b000 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f14f400cad0 tx=0x7f14f400cde0 comp rx=0 tx=0).stop 2023-08-25T03:34:24.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.596+0000 7f15057e3700 1 -- 172.21.15.2:0/759658451 shutdown_connections 2023-08-25T03:34:24.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.596+0000 7f15057e3700 1 --2- 172.21.15.2:0/759658451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f150015e980 0x7f15000c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:24.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.596+0000 7f15057e3700 1 --2- 172.21.15.2:0/759658451 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f15000b5330 0x7f150014a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:24.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.596+0000 7f15057e3700 1 --2- 172.21.15.2:0/759658451 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f14ec06a0e0 0x7f14ec06c5a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:24.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.596+0000 7f15057e3700 1 --2- 172.21.15.2:0/759658451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f15000d24f0 0x7f150014b000 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:24.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.596+0000 7f15057e3700 1 -- 172.21.15.2:0/759658451 >> 172.21.15.2:0/759658451 conn(0x7f15000cfc20 msgr2=0x7f150015b770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:24.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.596+0000 7f15057e3700 1 -- 172.21.15.2:0/759658451 shutdown_connections 2023-08-25T03:34:24.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:24.596+0000 7f15057e3700 1 -- 172.21.15.2:0/759658451 wait complete. 2023-08-25T03:34:24.609 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T03:34:24.610 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T03:34:24.610 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi002.front.sepia.ceph.com 2023-08-25T03:34:24.610 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-25T03:34:24.610 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-25T03:34:24.610 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T03:34:24.611 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T03:34:24.611 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T03:34:24.612 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-25T03:34:24.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:24.612 DEBUG:teuthology.orchestra.run.smithi002:> ip addr 2023-08-25T03:34:24.629 INFO:teuthology.orchestra.run.smithi002.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T03:34:24.630 INFO:teuthology.orchestra.run.smithi002.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T03:34:24.630 INFO:teuthology.orchestra.run.smithi002.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T03:34:24.630 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:24.630 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 ::1/128 scope host 2023-08-25T03:34:24.630 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:24.631 INFO:teuthology.orchestra.run.smithi002.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T03:34:24.631 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:02 brd ff:ff:ff:ff:ff:ff 2023-08-25T03:34:24.631 INFO:teuthology.orchestra.run.smithi002.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T03:34:24.631 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:03 brd ff:ff:ff:ff:ff:ff 2023-08-25T03:34:24.631 INFO:teuthology.orchestra.run.smithi002.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T03:34:24.631 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:20 brd ff:ff:ff:ff:ff:ff 2023-08-25T03:34:24.631 INFO:teuthology.orchestra.run.smithi002.stdout: inet 172.21.15.2/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-25T03:34:24.632 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft 42316sec preferred_lft 42316sec 2023-08-25T03:34:24.632 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 fe80::ec4:7aff:febd:1920/64 scope link 2023-08-25T03:34:24.632 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:24.632 INFO:teuthology.orchestra.run.smithi002.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T03:34:24.633 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:21 brd ff:ff:ff:ff:ff:ff 2023-08-25T03:34:24.633 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-08-25T03:34:24.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:24.634 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2023-08-25T03:34:24.634 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2023-08-25T03:34:24.634 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add name ceph-brx type bridge 2023-08-25T03:34:24.634 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip addr flush dev ceph-brx 2023-08-25T03:34:24.635 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set ceph-brx up 2023-08-25T03:34:24.635 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-08-25T03:34:24.635 DEBUG:teuthology.orchestra.run.smithi002:> ') 2023-08-25T03:34:24.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:24.789 DEBUG:teuthology.orchestra.run.smithi002:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T03:34:24.817 INFO:teuthology.orchestra.run.smithi002.stdout:1 2023-08-25T03:34:24.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:24.819 DEBUG:teuthology.orchestra.run.smithi002:> route 2023-08-25T03:34:24.897 INFO:teuthology.orchestra.run.smithi002.stdout:Kernel IP routing table 2023-08-25T03:34:24.898 INFO:teuthology.orchestra.run.smithi002.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T03:34:24.898 INFO:teuthology.orchestra.run.smithi002.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-25T03:34:24.898 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-25T03:34:24.899 INFO:teuthology.orchestra.run.smithi002.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T03:34:24.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:24.900 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2023-08-25T03:34:24.900 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2023-08-25T03:34:24.901 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-25T03:34:24.901 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-25T03:34:24.902 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-25T03:34:24.902 DEBUG:teuthology.orchestra.run.smithi002:> ') 2023-08-25T03:34:25.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:25.171 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2023-08-25T03:34:25.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:25.189 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list-id 2023-08-25T03:34:25.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:25.246 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2023-08-25T03:34:25.246 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2023-08-25T03:34:25.246 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-25T03:34:25.247 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-08-25T03:34:25.247 DEBUG:teuthology.orchestra.run.smithi002:> ') 2023-08-25T03:34:25.341 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-08-25T03:34:25.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:25.342 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2023-08-25T03:34:25.342 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2023-08-25T03:34:25.343 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-08-25T03:34:25.343 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-08-25T03:34:25.343 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-08-25T03:34:25.344 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-08-25T03:34:25.344 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-08-25T03:34:25.344 DEBUG:teuthology.orchestra.run.smithi002:> ') 2023-08-25T03:34:25.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:25.568 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2023-08-25T03:34:25.568 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2023-08-25T03:34:25.568 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set brx.0 up 2023-08-25T03:34:25.568 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set dev brx.0 master ceph-brx 2023-08-25T03:34:25.569 DEBUG:teuthology.orchestra.run.smithi002:> ') 2023-08-25T03:34:25.670 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T03:34:25.671 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-25T03:34:25.694 INFO:teuthology.orchestra.run.smithi002.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-25T03:34:25.695 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T03:34:25.695 DEBUG:teuthology.orchestra.run.smithi002:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-25T03:34:25.752 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-25T03:34:25.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:25.753 DEBUG:teuthology.orchestra.run.smithi002:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-25T03:34:25.872 INFO:teuthology.orchestra.run.smithi002.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-25T03:34:25.873 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-25T03:34:25.873 INFO:teuthology.orchestra.run.smithi002.stdout:invalid new device string format 2023-08-25T03:34:25.985 INFO:teuthology.orchestra.run.smithi002.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-25T03:34:25.985 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-25T03:34:25.986 INFO:teuthology.orchestra.run.smithi002.stdout:invalid new device string format 2023-08-25T03:34:25.986 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: resolved to: "172.21.15.2:3300,172.21.15.31:3300,172.21.15.31:3301" 2023-08-25T03:34:25.986 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-25T03:34:26.013 INFO:teuthology.orchestra.run.smithi002.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-25T03:34:26.013 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-25T03:34:26.014 INFO:teuthology.orchestra.run.smithi002.stdout:invalid new device string format 2023-08-25T03:34:26.014 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: resolved to: "172.21.15.2:3300,172.21.15.31:3300,172.21.15.31:3301" 2023-08-25T03:34:26.014 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-25T03:34:26.014 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: resolved to: "172.21.15.2:6789,172.21.15.31:6789,172.21.15.31:6790" 2023-08-25T03:34:26.014 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.2:6789,172.21.15.31:6789,172.21.15.31:6790:/ 2023-08-25T03:34:26.014 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=dc54136d-3683-431b-8a68-994ec618505c" will pass to kernel 2023-08-25T03:34:26.020 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-25T03:34:26.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:26.021 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-25T03:34:26.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:26.059 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-25T03:34:26.143 INFO:teuthology.orchestra.run.smithi002.stdout:client4734 2023-08-25T03:34:26.143 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:34:26.144 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:34:26.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:26.202 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-25T03:34:26.285 INFO:teuthology.orchestra.run.smithi002.stdout:client4734 2023-08-25T03:34:26.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:26.286 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-25T03:34:26.369 INFO:teuthology.orchestra.run.smithi002.stdout:client4734 2023-08-25T03:34:26.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:26.370 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/dc54136d-3683-431b-8a68-994ec618505c.client4734/status') 2023-08-25T03:34:26.461 INFO:teuthology.orchestra.run.smithi002.stdout:instance: client.4734 (3)192.168.0.1:0/2604350638 2023-08-25T03:34:26.462 INFO:teuthology.orchestra.run.smithi002.stdout:blocklisted: false 2023-08-25T03:34:26.463 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records in 2023-08-25T03:34:26.463 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records out 2023-08-25T03:34:26.463 INFO:teuthology.orchestra.run.smithi002.stderr:69 bytes copied, 2.3597e-05 s, 2.9 MB/s 2023-08-25T03:34:26.463 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2023-08-25T03:34:26.464 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-25T03:34:26.464 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-08-25T03:34:26.464 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T03:34:26.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.589+0000 7f1ac45a1700 1 Processor -- start 2023-08-25T03:34:26.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.589+0000 7f1ac45a1700 1 -- start start 2023-08-25T03:34:26.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.590+0000 7f1ac45a1700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1abc0b1fb0 0x7f1abc160f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:26.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.590+0000 7f1ac45a1700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1abc161450 0x7f1abc0cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:26.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.590+0000 7f1ac45a1700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f1abc0ce0b0 0x7f1abc06b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:26.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.590+0000 7f1ac45a1700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1abc06d450 con 0x7f1abc0ce0b0 2023-08-25T03:34:26.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.590+0000 7f1ac45a1700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1abc06d730 con 0x7f1abc161450 2023-08-25T03:34:26.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.590+0000 7f1ac45a1700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc06da10 con 0x7f1abc0b1fb0 2023-08-25T03:34:26.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.590+0000 7f1ac2b3e700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f1abc0ce0b0 0x7f1abc06b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:26.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.590+0000 7f1ac2b3e700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f1abc0ce0b0 0x7f1abc06b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60042/0 (socket says 172.21.15.2:60042) 2023-08-25T03:34:26.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.590+0000 7f1ac2b3e700 1 -- 172.21.15.2:0/837289521 learned_addr learned my addr 172.21.15.2:0/837289521 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:26.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.590+0000 7f1ac233d700 1 --2- 172.21.15.2:0/837289521 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1abc0b1fb0 0x7f1abc160f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:26.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.590+0000 7f1ac1b3c700 1 --2- 172.21.15.2:0/837289521 >> v2:172.21.15.31:3300/0 conn(0x7f1abc161450 0x7f1abc0cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:26.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.590+0000 7f1ac2b3e700 1 -- 172.21.15.2:0/837289521 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1abc0b1fb0 msgr2=0x7f1abc160f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:26.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.590+0000 7f1ac2b3e700 1 --2- 172.21.15.2:0/837289521 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1abc0b1fb0 0x7f1abc160f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:26.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.590+0000 7f1ac2b3e700 1 -- 172.21.15.2:0/837289521 >> v2:172.21.15.31:3300/0 conn(0x7f1abc161450 msgr2=0x7f1abc0cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:26.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.590+0000 7f1ac2b3e700 1 --2- 172.21.15.2:0/837289521 >> v2:172.21.15.31:3300/0 conn(0x7f1abc161450 0x7f1abc0cdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:26.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.590+0000 7f1ac2b3e700 1 -- 172.21.15.2:0/837289521 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1abc06dcf0 con 0x7f1abc0ce0b0 2023-08-25T03:34:26.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.590+0000 7f1ac2b3e700 1 --2- 172.21.15.2:0/837289521 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1abc0ce0b0 0x7f1abc06b8c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1abc06e1f0 tx=0x7f1ab400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def5f4254f0b3888 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:26.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.591+0000 7f1ac133b700 1 -- 172.21.15.2:0/837289521 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab400e070 con 0x7f1abc0ce0b0 2023-08-25T03:34:26.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.591+0000 7f1ac133b700 1 -- 172.21.15.2:0/837289521 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab400b910 con 0x7f1abc0ce0b0 2023-08-25T03:34:26.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.591+0000 7f1ac133b700 1 -- 172.21.15.2:0/837289521 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab400bcc0 con 0x7f1abc0ce0b0 2023-08-25T03:34:26.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.591+0000 7f1ac45a1700 1 -- 172.21.15.2:0/837289521 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1abc0ce0b0 msgr2=0x7f1abc06b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:26.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.591+0000 7f1ac45a1700 1 --2- 172.21.15.2:0/837289521 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1abc0ce0b0 0x7f1abc06b8c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1abc06e1f0 tx=0x7f1ab400b330 comp rx=0 tx=0).stop 2023-08-25T03:34:26.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.591+0000 7f1ac45a1700 1 -- 172.21.15.2:0/837289521 shutdown_connections 2023-08-25T03:34:26.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.591+0000 7f1ac45a1700 1 --2- 172.21.15.2:0/837289521 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1abc0ce0b0 0x7f1abc06b8c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:26.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.591+0000 7f1ac45a1700 1 --2- 172.21.15.2:0/837289521 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1abc0b1fb0 0x7f1abc160f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:26.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.591+0000 7f1ac45a1700 1 --2- 172.21.15.2:0/837289521 >> v2:172.21.15.31:3300/0 conn(0x7f1abc161450 0x7f1abc0cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:26.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.591+0000 7f1ac45a1700 1 -- 172.21.15.2:0/837289521 >> 172.21.15.2:0/837289521 conn(0x7f1abc0cfea0 msgr2=0x7f1abc0d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:26.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.591+0000 7f1ac45a1700 1 -- 172.21.15.2:0/837289521 shutdown_connections 2023-08-25T03:34:26.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.591+0000 7f1ac45a1700 1 -- 172.21.15.2:0/837289521 wait complete. 2023-08-25T03:34:26.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.593+0000 7f1ac45a1700 1 Processor -- start 2023-08-25T03:34:26.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.593+0000 7f1ac45a1700 1 -- start start 2023-08-25T03:34:26.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.593+0000 7f1ac45a1700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1abc0b1fb0 0x7f1abc0c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:26.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.593+0000 7f1ac45a1700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1abc0ce0b0 0x7f1abc0c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:26.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.594+0000 7f1ac45a1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1abc0c7de0 0x7f1abc159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:26.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.594+0000 7f1ac45a1700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc0c8260 con 0x7f1abc0c7de0 2023-08-25T03:34:26.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.594+0000 7f1ac45a1700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc0c8540 con 0x7f1abc0ce0b0 2023-08-25T03:34:26.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.594+0000 7f1ac45a1700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1abc0c8820 con 0x7f1abc0b1fb0 2023-08-25T03:34:26.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.594+0000 7f1ac2b3e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1abc0c7de0 0x7f1abc159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:26.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.594+0000 7f1ac2b3e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1abc0c7de0 0x7f1abc159150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60050/0 (socket says 172.21.15.2:60050) 2023-08-25T03:34:26.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.594+0000 7f1ac2b3e700 1 -- 172.21.15.2:0/2139951969 learned_addr learned my addr 172.21.15.2:0/2139951969 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:26.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.594+0000 7f1ac1b3c700 1 --2- 172.21.15.2:0/2139951969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1abc0ce0b0 0x7f1abc0c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:26.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.594+0000 7f1ac233d700 1 --2- 172.21.15.2:0/2139951969 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1abc0b1fb0 0x7f1abc0c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:26.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.594+0000 7f1ac2b3e700 1 -- 172.21.15.2:0/2139951969 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1abc0b1fb0 msgr2=0x7f1abc0c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:26.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.594+0000 7f1ac2b3e700 1 --2- 172.21.15.2:0/2139951969 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1abc0b1fb0 0x7f1abc0c9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:26.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.594+0000 7f1ac2b3e700 1 -- 172.21.15.2:0/2139951969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1abc0ce0b0 msgr2=0x7f1abc0c78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:26.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.594+0000 7f1ac2b3e700 1 --2- 172.21.15.2:0/2139951969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1abc0ce0b0 0x7f1abc0c78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:26.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.594+0000 7f1ac2b3e700 1 -- 172.21.15.2:0/2139951969 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab4009b30 con 0x7f1abc0c7de0 2023-08-25T03:34:26.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.594+0000 7f1ac1b3c700 1 --2- 172.21.15.2:0/2139951969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1abc0ce0b0 0x7f1abc0c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:26.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.594+0000 7f1ac2b3e700 1 --2- 172.21.15.2:0/2139951969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1abc0c7de0 0x7f1abc159150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4012000 tx=0x7f1ab4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:26.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.594+0000 7f1ab2ffd700 1 -- 172.21.15.2:0/2139951969 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab400ce80 con 0x7f1abc0c7de0 2023-08-25T03:34:26.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.594+0000 7f1ab2ffd700 1 -- 172.21.15.2:0/2139951969 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab4004250 con 0x7f1abc0c7de0 2023-08-25T03:34:26.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.594+0000 7f1ab2ffd700 1 -- 172.21.15.2:0/2139951969 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4004600 con 0x7f1abc0c7de0 2023-08-25T03:34:26.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.594+0000 7f1ac45a1700 1 -- 172.21.15.2:0/2139951969 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1abc0c8b00 con 0x7f1abc0c7de0 2023-08-25T03:34:26.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.594+0000 7f1ac45a1700 1 -- 172.21.15.2:0/2139951969 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1abc0c9840 con 0x7f1abc0c7de0 2023-08-25T03:34:26.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.595+0000 7f1ac45a1700 1 -- 172.21.15.2:0/2139951969 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1abc066f70 con 0x7f1abc0c7de0 2023-08-25T03:34:26.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.596+0000 7f1ab2ffd700 1 -- 172.21.15.2:0/2139951969 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1ab4004760 con 0x7f1abc0c7de0 2023-08-25T03:34:26.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.596+0000 7f1ab2ffd700 1 --2- 172.21.15.2:0/2139951969 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1aa006a0e0 0x7f1aa006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:26.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.596+0000 7f1ab2ffd700 1 -- 172.21.15.2:0/2139951969 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1ab40899d0 con 0x7f1abc0c7de0 2023-08-25T03:34:26.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.596+0000 7f1ac233d700 1 --2- 172.21.15.2:0/2139951969 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1aa006a0e0 0x7f1aa006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:26.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.597+0000 7f1ac233d700 1 --2- 172.21.15.2:0/2139951969 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1aa006a0e0 0x7f1aa006c5a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009a30 tx=0x7f1ab8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:26.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.601+0000 7f1ab2ffd700 1 -- 172.21.15.2:0/2139951969 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1ab4055200 con 0x7f1abc0c7de0 2023-08-25T03:34:26.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.849+0000 7f1ac45a1700 1 -- 172.21.15.2:0/2139951969 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1abc0c99d0 con 0x7f1abc0c7de0 2023-08-25T03:34:26.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.849+0000 7f1ab2ffd700 1 -- 172.21.15.2:0/2139951969 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1ab4020070 con 0x7f1abc0c7de0 2023-08-25T03:34:26.851 INFO:teuthology.orchestra.run.smithi002.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T03:34:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.851+0000 7f1ac45a1700 1 -- 172.21.15.2:0/2139951969 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1aa006a0e0 msgr2=0x7f1aa006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.851+0000 7f1ac45a1700 1 --2- 172.21.15.2:0/2139951969 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1aa006a0e0 0x7f1aa006c5a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f1ab8009a30 tx=0x7f1ab8008040 comp rx=0 tx=0).stop 2023-08-25T03:34:26.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.851+0000 7f1ac45a1700 1 -- 172.21.15.2:0/2139951969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1abc0c7de0 msgr2=0x7f1abc159150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:26.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.851+0000 7f1ac45a1700 1 --2- 172.21.15.2:0/2139951969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1abc0c7de0 0x7f1abc159150 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4012000 tx=0x7f1ab4004ad0 comp rx=0 tx=0).stop 2023-08-25T03:34:26.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.851+0000 7f1ac45a1700 1 -- 172.21.15.2:0/2139951969 shutdown_connections 2023-08-25T03:34:26.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.851+0000 7f1ac45a1700 1 --2- 172.21.15.2:0/2139951969 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1aa006a0e0 0x7f1aa006c5a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:26.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.851+0000 7f1ac45a1700 1 --2- 172.21.15.2:0/2139951969 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1abc0c7de0 0x7f1abc159150 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:26.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.851+0000 7f1ac45a1700 1 --2- 172.21.15.2:0/2139951969 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1abc0b1fb0 0x7f1abc0c9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:26.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.851+0000 7f1ac45a1700 1 --2- 172.21.15.2:0/2139951969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1abc0ce0b0 0x7f1abc0c78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:26.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.851+0000 7f1ac45a1700 1 -- 172.21.15.2:0/2139951969 >> 172.21.15.2:0/2139951969 conn(0x7f1abc0cfea0 msgr2=0x7f1abc15bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:26.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.851+0000 7f1ac45a1700 1 -- 172.21.15.2:0/2139951969 shutdown_connections 2023-08-25T03:34:26.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:26.851+0000 7f1ac45a1700 1 -- 172.21.15.2:0/2139951969 wait complete. 2023-08-25T03:34:26.864 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T03:34:26.864 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T03:34:26.865 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi002.front.sepia.ceph.com 2023-08-25T03:34:26.866 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-08-25T03:34:26.866 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-08-25T03:34:26.868 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T03:34:26.869 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T03:34:26.870 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T03:34:26.870 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-08-25T03:34:26.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:26.871 DEBUG:teuthology.orchestra.run.smithi002:> ip addr 2023-08-25T03:34:26.889 INFO:teuthology.orchestra.run.smithi002.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T03:34:26.890 INFO:teuthology.orchestra.run.smithi002.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T03:34:26.890 INFO:teuthology.orchestra.run.smithi002.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T03:34:26.890 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:26.890 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 ::1/128 scope host 2023-08-25T03:34:26.890 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:26.890 INFO:teuthology.orchestra.run.smithi002.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T03:34:26.891 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:02 brd ff:ff:ff:ff:ff:ff 2023-08-25T03:34:26.891 INFO:teuthology.orchestra.run.smithi002.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T03:34:26.891 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:03 brd ff:ff:ff:ff:ff:ff 2023-08-25T03:34:26.891 INFO:teuthology.orchestra.run.smithi002.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T03:34:26.891 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:20 brd ff:ff:ff:ff:ff:ff 2023-08-25T03:34:26.891 INFO:teuthology.orchestra.run.smithi002.stdout: inet 172.21.15.2/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-25T03:34:26.892 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft 42314sec preferred_lft 42314sec 2023-08-25T03:34:26.892 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 fe80::ec4:7aff:febd:1920/64 scope link 2023-08-25T03:34:26.892 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:26.892 INFO:teuthology.orchestra.run.smithi002.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T03:34:26.893 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:21 brd ff:ff:ff:ff:ff:ff 2023-08-25T03:34:26.893 INFO:teuthology.orchestra.run.smithi002.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T03:34:26.893 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether f2:17:15:9f:24:a0 brd ff:ff:ff:ff:ff:ff 2023-08-25T03:34:26.893 INFO:teuthology.orchestra.run.smithi002.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-25T03:34:26.893 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:26.894 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 fe80::4058:70ff:fea4:c945/64 scope link 2023-08-25T03:34:26.895 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:26.896 INFO:teuthology.orchestra.run.smithi002.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T03:34:26.896 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether f2:17:15:9f:24:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-25T03:34:26.896 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 fe80::f017:15ff:fe9f:24a0/64 scope link tentative 2023-08-25T03:34:26.896 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:26.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:26.897 DEBUG:teuthology.orchestra.run.smithi002:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T03:34:26.957 INFO:teuthology.orchestra.run.smithi002.stdout:1 2023-08-25T03:34:26.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:26.963 DEBUG:teuthology.orchestra.run.smithi002:> route 2023-08-25T03:34:27.022 INFO:teuthology.orchestra.run.smithi002.stdout:Kernel IP routing table 2023-08-25T03:34:27.023 INFO:teuthology.orchestra.run.smithi002.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T03:34:27.023 INFO:teuthology.orchestra.run.smithi002.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-25T03:34:27.023 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-25T03:34:27.023 INFO:teuthology.orchestra.run.smithi002.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T03:34:27.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:27.025 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2023-08-25T03:34:27.026 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2023-08-25T03:34:27.026 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-25T03:34:27.026 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-25T03:34:27.027 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-25T03:34:27.027 DEBUG:teuthology.orchestra.run.smithi002:> ') 2023-08-25T03:34:27.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:27.130 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2023-08-25T03:34:27.148 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-25T03:34:27.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:27.149 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list-id 2023-08-25T03:34:27.209 INFO:teuthology.orchestra.run.smithi002.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-08-25T03:34:27.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:27.211 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2023-08-25T03:34:27.211 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2023-08-25T03:34:27.211 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-25T03:34:27.211 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-08-25T03:34:27.212 DEBUG:teuthology.orchestra.run.smithi002:> ') 2023-08-25T03:34:27.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:27.305 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-08-25T03:34:27.347 INFO:teuthology.orchestra.run.smithi002.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T03:34:27.347 INFO:teuthology.orchestra.run.smithi002.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T03:34:27.348 INFO:teuthology.orchestra.run.smithi002.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T03:34:27.348 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:27.349 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 ::1/128 scope host 2023-08-25T03:34:27.349 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:27.349 INFO:teuthology.orchestra.run.smithi002.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T03:34:27.349 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether e6:f2:64:d1:2a:96 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-08-25T03:34:27.350 INFO:teuthology.orchestra.run.smithi002.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-08-25T03:34:27.350 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:27.350 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 fe80::e4f2:64ff:fed1:2a96/64 scope link tentative 2023-08-25T03:34:27.350 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:27.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:27.359 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-08-25T03:34:27.441 INFO:teuthology.orchestra.run.smithi002.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T03:34:27.442 INFO:teuthology.orchestra.run.smithi002.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T03:34:27.442 INFO:teuthology.orchestra.run.smithi002.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T03:34:27.442 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:27.442 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 ::1/128 scope host 2023-08-25T03:34:27.442 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:27.443 INFO:teuthology.orchestra.run.smithi002.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T03:34:27.443 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether e6:f2:64:d1:2a:96 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-08-25T03:34:27.443 INFO:teuthology.orchestra.run.smithi002.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-08-25T03:34:27.443 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:27.443 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 fe80::e4f2:64ff:fed1:2a96/64 scope link tentative 2023-08-25T03:34:27.443 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:27.463 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-08-25T03:34:27.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:27.464 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2023-08-25T03:34:27.464 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2023-08-25T03:34:27.464 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-08-25T03:34:27.464 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-08-25T03:34:27.465 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-08-25T03:34:27.465 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-08-25T03:34:27.465 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-08-25T03:34:27.465 DEBUG:teuthology.orchestra.run.smithi002:> ') 2023-08-25T03:34:27.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:27.670 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2023-08-25T03:34:27.670 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2023-08-25T03:34:27.670 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set brx.1 up 2023-08-25T03:34:27.670 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set dev brx.1 master ceph-brx 2023-08-25T03:34:27.671 DEBUG:teuthology.orchestra.run.smithi002:> ') 2023-08-25T03:34:27.790 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T03:34:27.790 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-08-25T03:34:27.812 INFO:teuthology.orchestra.run.smithi002.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-08-25T03:34:27.813 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T03:34:27.813 DEBUG:teuthology.orchestra.run.smithi002:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-08-25T03:34:27.870 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-25T03:34:27.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:27.872 DEBUG:teuthology.orchestra.run.smithi002:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-08-25T03:34:27.957 INFO:teuthology.orchestra.run.smithi002.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-08-25T03:34:27.958 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-08-25T03:34:27.958 INFO:teuthology.orchestra.run.smithi002.stdout:invalid new device string format 2023-08-25T03:34:27.968 INFO:teuthology.orchestra.run.smithi002.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-08-25T03:34:27.969 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-08-25T03:34:27.969 INFO:teuthology.orchestra.run.smithi002.stdout:invalid new device string format 2023-08-25T03:34:27.969 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: resolved to: "172.21.15.2:3300,172.21.15.31:3300,172.21.15.31:3301" 2023-08-25T03:34:27.969 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-25T03:34:28.039 INFO:teuthology.orchestra.run.smithi002.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2023-08-25T03:34:28.039 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2023-08-25T03:34:28.039 INFO:teuthology.orchestra.run.smithi002.stdout:invalid new device string format 2023-08-25T03:34:28.039 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: resolved to: "172.21.15.2:3300,172.21.15.31:3300,172.21.15.31:3301" 2023-08-25T03:34:28.039 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-25T03:34:28.040 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: resolved to: "172.21.15.2:6789,172.21.15.31:6789,172.21.15.31:6790" 2023-08-25T03:34:28.040 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.2:6789,172.21.15.31:6789,172.21.15.31:6790:/ 2023-08-25T03:34:28.040 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=dc54136d-3683-431b-8a68-994ec618505c" will pass to kernel 2023-08-25T03:34:28.041 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-25T03:34:28.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:28.041 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-08-25T03:34:28.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:28.069 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-25T03:34:28.141 INFO:teuthology.orchestra.run.smithi002.stdout:client4654 2023-08-25T03:34:28.142 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:34:28.142 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:34:28.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:28.200 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-25T03:34:28.272 INFO:teuthology.orchestra.run.smithi002.stdout:client4654 2023-08-25T03:34:28.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:28.272 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-25T03:34:28.345 INFO:teuthology.orchestra.run.smithi002.stdout:client4654 2023-08-25T03:34:28.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:28.345 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/dc54136d-3683-431b-8a68-994ec618505c.client4654/status') 2023-08-25T03:34:28.426 INFO:teuthology.orchestra.run.smithi002.stdout:instance: client.4654 (3)172.21.15.2:0/1178280324 2023-08-25T03:34:28.426 INFO:teuthology.orchestra.run.smithi002.stdout:blocklisted: false 2023-08-25T03:34:28.427 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records in 2023-08-25T03:34:28.427 INFO:teuthology.orchestra.run.smithi002.stderr:0+1 records out 2023-08-25T03:34:28.427 INFO:teuthology.orchestra.run.smithi002.stderr:69 bytes copied, 2.9398e-05 s, 2.3 MB/s 2023-08-25T03:34:28.428 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2023-08-25T03:34:28.429 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-25T03:34:28.429 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-25T03:34:28.430 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T03:34:28.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.593+0000 7f2f0bf30700 1 Processor -- start 2023-08-25T03:34:28.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.593+0000 7f2f0bf30700 1 -- start start 2023-08-25T03:34:28.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.594+0000 7f2f0bf30700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2f0415a410 0x7f2f040ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:28.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.594+0000 7f2f0bf30700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2f040ce6b0 0x7f2f040ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:28.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.594+0000 7f2f0bf30700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f04071dc0 con 0x7f2f0415a030 2023-08-25T03:34:28.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.594+0000 7f2f0bf30700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f040ceb00 con 0x7f2f040ce6b0 2023-08-25T03:34:28.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.594+0000 7f2f0bf30700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f040720a0 con 0x7f2f0415a410 2023-08-25T03:34:28.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.594+0000 7f2f0a4cd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2f040ce6b0 0x7f2f040ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:28.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.594+0000 7f2f09ccc700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f2f0415a030 0x7f2f040cbcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46730/0 (socket says 172.21.15.2:46730) 2023-08-25T03:34:28.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.594+0000 7f2f09ccc700 1 -- 172.21.15.2:0/2942438512 learned_addr learned my addr 172.21.15.2:0/2942438512 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:28.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.594+0000 7f2f094cb700 1 --2- 172.21.15.2:0/2942438512 >> v2:172.21.15.31:3300/0 conn(0x7f2f0415a410 0x7f2f040ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:28.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.594+0000 7f2f0a4cd700 1 -- 172.21.15.2:0/2942438512 >> v2:172.21.15.31:3300/0 conn(0x7f2f0415a410 msgr2=0x7f2f040ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:28.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.594+0000 7f2f0a4cd700 1 --2- 172.21.15.2:0/2942438512 >> v2:172.21.15.31:3300/0 conn(0x7f2f0415a410 0x7f2f040ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:28.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.594+0000 7f2f0a4cd700 1 -- 172.21.15.2:0/2942438512 >> v1:172.21.15.2:6789/0 conn(0x7f2f0415a030 legacy=0x7f2f040cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:28.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.594+0000 7f2f0a4cd700 1 -- 172.21.15.2:0/2942438512 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f040cb4b0 con 0x7f2f040ce6b0 2023-08-25T03:34:28.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.594+0000 7f2f08cca700 1 -- 172.21.15.2:0/2942438512 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 211049516 0 0) 0x7f2f04071dc0 con 0x7f2f0415a030 2023-08-25T03:34:28.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.595+0000 7f2f0a4cd700 1 --2- 172.21.15.2:0/2942438512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f040ce6b0 0x7f2f040ccd50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2f0406de90 tx=0x7f2efc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f8d0066c5f83b62 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:28.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.595+0000 7f2f08cca700 1 -- 172.21.15.2:0/2942438512 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc00e070 con 0x7f2f040ce6b0 2023-08-25T03:34:28.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.595+0000 7f2f08cca700 1 -- 172.21.15.2:0/2942438512 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2efc00bbd0 con 0x7f2f040ce6b0 2023-08-25T03:34:28.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.595+0000 7f2f08cca700 1 -- 172.21.15.2:0/2942438512 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc012430 con 0x7f2f040ce6b0 2023-08-25T03:34:28.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.595+0000 7f2f0bf30700 1 -- 172.21.15.2:0/2942438512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f040ce6b0 msgr2=0x7f2f040ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:28.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.595+0000 7f2f0bf30700 1 --2- 172.21.15.2:0/2942438512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f040ce6b0 0x7f2f040ccd50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2f0406de90 tx=0x7f2efc00b330 comp rx=0 tx=0).stop 2023-08-25T03:34:28.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.595+0000 7f2f0bf30700 1 -- 172.21.15.2:0/2942438512 shutdown_connections 2023-08-25T03:34:28.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.595+0000 7f2f0bf30700 1 --2- 172.21.15.2:0/2942438512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f040ce6b0 0x7f2f040ccd50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:28.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.595+0000 7f2f0bf30700 1 --2- 172.21.15.2:0/2942438512 >> v2:172.21.15.31:3300/0 conn(0x7f2f0415a410 0x7f2f040ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:28.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.595+0000 7f2f0bf30700 1 -- 172.21.15.2:0/2942438512 >> 172.21.15.2:0/2942438512 conn(0x7f2f040b1cf0 msgr2=0x7f2f040b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:28.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.595+0000 7f2f0bf30700 1 -- 172.21.15.2:0/2942438512 shutdown_connections 2023-08-25T03:34:28.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.595+0000 7f2f0bf30700 1 -- 172.21.15.2:0/2942438512 wait complete. 2023-08-25T03:34:28.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.597+0000 7f2f0bf30700 1 Processor -- start 2023-08-25T03:34:28.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.597+0000 7f2f0bf30700 1 -- start start 2023-08-25T03:34:28.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.597+0000 7f2f0bf30700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f040ce6b0 0x7f2f041563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:28.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.598+0000 7f2f0bf30700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f0415a030 0x7f2f040c56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:28.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.598+0000 7f2f0bf30700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2f0415a410 0x7f2f04158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.598+0000 7f2f0bf30700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f040c5260 con 0x7f2f040ce6b0 2023-08-25T03:34:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.598+0000 7f2f0bf30700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f040b5ca0 con 0x7f2f0415a030 2023-08-25T03:34:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.598+0000 7f2f0bf30700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f041568f0 con 0x7f2f0415a410 2023-08-25T03:34:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.598+0000 7f2f09ccc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f040ce6b0 0x7f2f041563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.598+0000 7f2f09ccc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f040ce6b0 0x7f2f041563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60066/0 (socket says 172.21.15.2:60066) 2023-08-25T03:34:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.598+0000 7f2f09ccc700 1 -- 172.21.15.2:0/3360284303 learned_addr learned my addr 172.21.15.2:0/3360284303 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.598+0000 7f2f094cb700 1 --2- 172.21.15.2:0/3360284303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f0415a030 0x7f2f040c56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:28.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.598+0000 7f2f0a4cd700 1 --2- 172.21.15.2:0/3360284303 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2f0415a410 0x7f2f04158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:28.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.598+0000 7f2f09ccc700 1 -- 172.21.15.2:0/3360284303 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2f0415a410 msgr2=0x7f2f04158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:28.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.598+0000 7f2f09ccc700 1 --2- 172.21.15.2:0/3360284303 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2f0415a410 0x7f2f04158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:28.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.598+0000 7f2f09ccc700 1 -- 172.21.15.2:0/3360284303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f0415a030 msgr2=0x7f2f040c56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:28.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.598+0000 7f2f09ccc700 1 --2- 172.21.15.2:0/3360284303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f0415a030 0x7f2f040c56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:28.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.598+0000 7f2f09ccc700 1 -- 172.21.15.2:0/3360284303 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2efc009a70 con 0x7f2f040ce6b0 2023-08-25T03:34:28.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.598+0000 7f2f09ccc700 1 --2- 172.21.15.2:0/3360284303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f040ce6b0 0x7f2f041563b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2ef40011e0 tx=0x7f2ef400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:28.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.598+0000 7f2efa7fc700 1 -- 172.21.15.2:0/3360284303 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4004a80 con 0x7f2f040ce6b0 2023-08-25T03:34:28.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.598+0000 7f2efa7fc700 1 -- 172.21.15.2:0/3360284303 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef4004c00 con 0x7f2f040ce6b0 2023-08-25T03:34:28.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.598+0000 7f2f0bf30700 1 -- 172.21.15.2:0/3360284303 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f040c4650 con 0x7f2f040ce6b0 2023-08-25T03:34:28.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.598+0000 7f2efa7fc700 1 -- 172.21.15.2:0/3360284303 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef40070d0 con 0x7f2f040ce6b0 2023-08-25T03:34:28.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.599+0000 7f2f0bf30700 1 -- 172.21.15.2:0/3360284303 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f040c4c10 con 0x7f2f040ce6b0 2023-08-25T03:34:28.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.599+0000 7f2f0bf30700 1 -- 172.21.15.2:0/3360284303 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f04066f70 con 0x7f2f040ce6b0 2023-08-25T03:34:28.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.600+0000 7f2efa7fc700 1 -- 172.21.15.2:0/3360284303 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2ef4007230 con 0x7f2f040ce6b0 2023-08-25T03:34:28.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.600+0000 7f2efa7fc700 1 --2- 172.21.15.2:0/3360284303 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2ee806a0e0 0x7f2ee806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:28.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.600+0000 7f2efa7fc700 1 -- 172.21.15.2:0/3360284303 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2ef408a9e0 con 0x7f2f040ce6b0 2023-08-25T03:34:28.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.600+0000 7f2f094cb700 1 --2- 172.21.15.2:0/3360284303 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2ee806a0e0 0x7f2ee806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:28.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.601+0000 7f2f094cb700 1 --2- 172.21.15.2:0/3360284303 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2ee806a0e0 0x7f2ee806c5a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2f040c86a0 tx=0x7f2f00008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:28.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.605+0000 7f2efa7fc700 1 -- 172.21.15.2:0/3360284303 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2ef40560d0 con 0x7f2f040ce6b0 2023-08-25T03:34:28.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.848+0000 7f2f0bf30700 1 -- 172.21.15.2:0/3360284303 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2f040c5540 con 0x7f2f040ce6b0 2023-08-25T03:34:28.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.848+0000 7f2efa7fc700 1 -- 172.21.15.2:0/3360284303 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2ef4015020 con 0x7f2f040ce6b0 2023-08-25T03:34:28.849 INFO:teuthology.orchestra.run.smithi002.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T03:34:28.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.850+0000 7f2f0bf30700 1 -- 172.21.15.2:0/3360284303 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2ee806a0e0 msgr2=0x7f2ee806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:28.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.850+0000 7f2f0bf30700 1 --2- 172.21.15.2:0/3360284303 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2ee806a0e0 0x7f2ee806c5a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f2f040c86a0 tx=0x7f2f00008040 comp rx=0 tx=0).stop 2023-08-25T03:34:28.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.850+0000 7f2f0bf30700 1 -- 172.21.15.2:0/3360284303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f040ce6b0 msgr2=0x7f2f041563b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:28.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.850+0000 7f2f0bf30700 1 --2- 172.21.15.2:0/3360284303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f040ce6b0 0x7f2f041563b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2ef40011e0 tx=0x7f2ef400ec40 comp rx=0 tx=0).stop 2023-08-25T03:34:28.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.850+0000 7f2f0bf30700 1 -- 172.21.15.2:0/3360284303 shutdown_connections 2023-08-25T03:34:28.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.850+0000 7f2f0bf30700 1 --2- 172.21.15.2:0/3360284303 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2ee806a0e0 0x7f2ee806c5a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:28.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.850+0000 7f2f0bf30700 1 --2- 172.21.15.2:0/3360284303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f040ce6b0 0x7f2f041563b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:28.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.850+0000 7f2f0bf30700 1 --2- 172.21.15.2:0/3360284303 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2f0415a410 0x7f2f04158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:28.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.850+0000 7f2f0bf30700 1 --2- 172.21.15.2:0/3360284303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f0415a030 0x7f2f040c56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:28.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.850+0000 7f2f0bf30700 1 -- 172.21.15.2:0/3360284303 >> 172.21.15.2:0/3360284303 conn(0x7f2f040b1cf0 msgr2=0x7f2f040b10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:28.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.850+0000 7f2f0bf30700 1 -- 172.21.15.2:0/3360284303 shutdown_connections 2023-08-25T03:34:28.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:28.850+0000 7f2f0bf30700 1 -- 172.21.15.2:0/3360284303 wait complete. 2023-08-25T03:34:28.863 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T03:34:28.864 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T03:34:28.864 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-08-25T03:34:28.864 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-25T03:34:28.864 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-25T03:34:28.864 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T03:34:28.865 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T03:34:28.865 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T03:34:28.865 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-25T03:34:28.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:28.866 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-08-25T03:34:28.889 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T03:34:28.889 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T03:34:28.889 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T03:34:28.890 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:28.890 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-08-25T03:34:28.890 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:28.890 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T03:34:28.890 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-08-25T03:34:28.890 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T03:34:28.890 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-08-25T03:34:28.891 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T03:34:28.891 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-08-25T03:34:28.891 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-25T03:34:28.891 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42296sec preferred_lft 42296sec 2023-08-25T03:34:28.891 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-08-25T03:34:28.892 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:28.892 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T03:34:28.892 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-08-25T03:34:28.892 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-08-25T03:34:28.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:28.894 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-08-25T03:34:28.895 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-08-25T03:34:28.896 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add name ceph-brx type bridge 2023-08-25T03:34:28.896 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr flush dev ceph-brx 2023-08-25T03:34:28.897 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set ceph-brx up 2023-08-25T03:34:28.898 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-08-25T03:34:28.898 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-08-25T03:34:29.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:29.098 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T03:34:29.129 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-08-25T03:34:29.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:29.131 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-08-25T03:34:29.202 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-08-25T03:34:29.202 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T03:34:29.202 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-25T03:34:29.202 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-25T03:34:29.202 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T03:34:29.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:29.203 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-08-25T03:34:29.203 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-08-25T03:34:29.204 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-25T03:34:29.204 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-25T03:34:29.204 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-25T03:34:29.205 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-08-25T03:34:29.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:29.443 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-08-25T03:34:29.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:29.461 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list-id 2023-08-25T03:34:29.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:29.518 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-08-25T03:34:29.519 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-08-25T03:34:29.519 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-25T03:34:29.519 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-08-25T03:34:29.519 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-08-25T03:34:29.620 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-08-25T03:34:29.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:29.621 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-08-25T03:34:29.621 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-08-25T03:34:29.621 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-08-25T03:34:29.622 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-08-25T03:34:29.622 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-08-25T03:34:29.622 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-08-25T03:34:29.622 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-08-25T03:34:29.622 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-08-25T03:34:29.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:29.823 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-08-25T03:34:29.824 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-08-25T03:34:29.824 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-08-25T03:34:29.825 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set dev brx.0 master ceph-brx 2023-08-25T03:34:29.826 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-08-25T03:34:29.928 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T03:34:29.929 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-25T03:34:29.952 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-25T03:34:29.952 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T03:34:29.953 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-25T03:34:30.008 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-25T03:34:30.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:30.009 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-25T03:34:30.104 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-25T03:34:30.105 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-25T03:34:30.105 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2023-08-25T03:34:30.222 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-25T03:34:30.223 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-25T03:34:30.223 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2023-08-25T03:34:30.223 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.2:3300,172.21.15.31:3300,172.21.15.31:3301" 2023-08-25T03:34:30.223 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-25T03:34:30.297 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-25T03:34:30.298 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-25T03:34:30.298 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2023-08-25T03:34:30.298 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.2:3300,172.21.15.31:3300,172.21.15.31:3301" 2023-08-25T03:34:30.298 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-25T03:34:30.298 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.2:6789,172.21.15.31:6789,172.21.15.31:6790" 2023-08-25T03:34:30.298 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.2:6789,172.21.15.31:6789,172.21.15.31:6790:/ 2023-08-25T03:34:30.299 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=dc54136d-3683-431b-8a68-994ec618505c" will pass to kernel 2023-08-25T03:34:30.300 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-25T03:34:30.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:30.300 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-25T03:34:30.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:30.329 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-25T03:34:30.402 INFO:teuthology.orchestra.run.smithi031.stdout:client4752 2023-08-25T03:34:30.402 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:34:30.403 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:34:30.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:30.463 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-25T03:34:30.535 INFO:teuthology.orchestra.run.smithi031.stdout:client4752 2023-08-25T03:34:30.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:30.536 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-25T03:34:30.608 INFO:teuthology.orchestra.run.smithi031.stdout:client4752 2023-08-25T03:34:30.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:30.609 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/dc54136d-3683-431b-8a68-994ec618505c.client4752/status') 2023-08-25T03:34:30.689 INFO:teuthology.orchestra.run.smithi031.stdout:instance: client.4752 (3)172.21.15.31:0/1324059625 2023-08-25T03:34:30.689 INFO:teuthology.orchestra.run.smithi031.stdout:blocklisted: false 2023-08-25T03:34:30.690 INFO:teuthology.orchestra.run.smithi031.stderr:0+1 records in 2023-08-25T03:34:30.690 INFO:teuthology.orchestra.run.smithi031.stderr:0+1 records out 2023-08-25T03:34:30.691 INFO:teuthology.orchestra.run.smithi031.stderr:70 bytes copied, 2.4082e-05 s, 2.9 MB/s 2023-08-25T03:34:30.691 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2023-08-25T03:34:30.691 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-08-25T03:34:30.691 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-08-25T03:34:30.692 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T03:34:30.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.829+0000 7fcb488db700 1 Processor -- start 2023-08-25T03:34:30.831 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.829+0000 7fcb488db700 1 -- start start 2023-08-25T03:34:30.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.830+0000 7fcb488db700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcb440b19a0 0x7fcb4415e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:30.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.830+0000 7fcb488db700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb44071b40 con 0x7fcb440caea0 2023-08-25T03:34:30.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.830+0000 7fcb488db700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb440720b0 con 0x7fcb440b1d80 2023-08-25T03:34:30.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.830+0000 7fcb488db700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb44071dd0 con 0x7fcb440b19a0 2023-08-25T03:34:30.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.830+0000 7fcb42d9d700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fcb440caea0 0x7fcb440cb330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46744/0 (socket says 172.21.15.2:46744) 2023-08-25T03:34:30.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.830+0000 7fcb42d9d700 1 -- 172.21.15.2:0/54840635 learned_addr learned my addr 172.21.15.2:0/54840635 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:30.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.830+0000 7fcb4259c700 1 --2- 172.21.15.2:0/54840635 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcb440b19a0 0x7fcb4415e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:30.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.830+0000 7fcb4159a700 1 -- 172.21.15.2:0/54840635 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 395840416 0 0) 0x7fcb44071b40 con 0x7fcb440caea0 2023-08-25T03:34:30.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.830+0000 7fcb4159a700 1 -- 172.21.15.2:0/54840635 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb2c0035b0 con 0x7fcb440caea0 2023-08-25T03:34:30.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.830+0000 7fcb4159a700 1 -- 172.21.15.2:0/54840635 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3429743592 0 0) 0x7fcb440720b0 con 0x7fcb440b1d80 2023-08-25T03:34:30.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.830+0000 7fcb4159a700 1 -- 172.21.15.2:0/54840635 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb44071b40 con 0x7fcb440b1d80 2023-08-25T03:34:30.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.830+0000 7fcb4159a700 1 -- 172.21.15.2:0/54840635 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1432271750 0 0) 0x7fcb2c0035b0 con 0x7fcb440caea0 2023-08-25T03:34:30.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.830+0000 7fcb4159a700 1 -- 172.21.15.2:0/54840635 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcb440b19a0 msgr2=0x7fcb4415e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:30.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.830+0000 7fcb4159a700 1 --2- 172.21.15.2:0/54840635 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcb440b19a0 0x7fcb4415e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:30.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.830+0000 7fcb4159a700 1 -- 172.21.15.2:0/54840635 >> v1:172.21.15.31:6789/0 conn(0x7fcb440b1d80 legacy=0x7fcb44071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:30.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.830+0000 7fcb4159a700 1 -- 172.21.15.2:0/54840635 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4415ddb0 con 0x7fcb440caea0 2023-08-25T03:34:30.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.830+0000 7fcb4159a700 1 -- 172.21.15.2:0/54840635 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7fcb3c0019f0 con 0x7fcb440caea0 2023-08-25T03:34:30.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.830+0000 7fcb4159a700 1 -- 172.21.15.2:0/54840635 >> v1:172.21.15.2:6789/0 conn(0x7fcb440caea0 legacy=0x7fcb440cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:30.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.830+0000 7fcb4159a700 1 --2- 172.21.15.2:0/54840635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb2c003bc0 0x7fcb2c003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:30.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.830+0000 7fcb4159a700 1 --2- 172.21.15.2:0/54840635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb2c004850 0x7fcb2c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:30.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.830+0000 7fcb4159a700 1 --2- 172.21.15.2:0/54840635 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcb440b19a0 0x7fcb2c005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:30.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.830+0000 7fcb4159a700 1 -- 172.21.15.2:0/54840635 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb44071dd0 con 0x7fcb2c003bc0 2023-08-25T03:34:30.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.831+0000 7fcb4159a700 1 -- 172.21.15.2:0/54840635 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c00bb20 con 0x7fcb2c004850 2023-08-25T03:34:30.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.831+0000 7fcb4159a700 1 -- 172.21.15.2:0/54840635 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c00bce0 con 0x7fcb440b19a0 2023-08-25T03:34:30.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.831+0000 7fcb4259c700 1 --2- 172.21.15.2:0/54840635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb2c003bc0 0x7fcb2c003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:30.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.831+0000 7fcb41d9b700 1 --2- 172.21.15.2:0/54840635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb2c004850 0x7fcb2c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:30.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.831+0000 7fcb42d9d700 1 --2- 172.21.15.2:0/54840635 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcb440b19a0 0x7fcb2c005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:30.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.831+0000 7fcb4259c700 1 -- 172.21.15.2:0/54840635 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcb440b19a0 msgr2=0x7fcb2c005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:30.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.831+0000 7fcb4259c700 1 --2- 172.21.15.2:0/54840635 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcb440b19a0 0x7fcb2c005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:30.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.831+0000 7fcb4259c700 1 -- 172.21.15.2:0/54840635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb2c004850 msgr2=0x7fcb2c004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:30.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.831+0000 7fcb4259c700 1 --2- 172.21.15.2:0/54840635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb2c004850 0x7fcb2c004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:30.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.831+0000 7fcb4259c700 1 -- 172.21.15.2:0/54840635 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb2c0035b0 con 0x7fcb2c003bc0 2023-08-25T03:34:30.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.831+0000 7fcb41d9b700 1 --2- 172.21.15.2:0/54840635 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb2c004850 0x7fcb2c004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:34:30.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.831+0000 7fcb4259c700 1 --2- 172.21.15.2:0/54840635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb2c003bc0 0x7fcb2c003fc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009b60 tx=0x7fcb2c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3717ca8d19876f91 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:30.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.831+0000 7fcb4159a700 1 -- 172.21.15.2:0/54840635 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3400e6c0 con 0x7fcb2c003bc0 2023-08-25T03:34:30.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.831+0000 7fcb4159a700 1 -- 172.21.15.2:0/54840635 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3400e820 con 0x7fcb2c003bc0 2023-08-25T03:34:30.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.831+0000 7fcb4159a700 1 -- 172.21.15.2:0/54840635 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3400eb90 con 0x7fcb2c003bc0 2023-08-25T03:34:30.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.832+0000 7fcb488db700 1 -- 172.21.15.2:0/54840635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb2c003bc0 msgr2=0x7fcb2c003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:30.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.832+0000 7fcb488db700 1 --2- 172.21.15.2:0/54840635 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb2c003bc0 0x7fcb2c003fc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009b60 tx=0x7fcb2c005880 comp rx=0 tx=0).stop 2023-08-25T03:34:30.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.832+0000 7fcb42d9d700 1 -- 172.21.15.2:0/54840635 reap_dead start 2023-08-25T03:34:30.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.832+0000 7fcb488db700 1 -- 172.21.15.2:0/54840635 shutdown_connections 2023-08-25T03:34:30.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.832+0000 7fcb488db700 1 -- 172.21.15.2:0/54840635 >> 172.21.15.2:0/54840635 conn(0x7fcb440cfea0 msgr2=0x7fcb440d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:30.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.832+0000 7fcb488db700 1 -- 172.21.15.2:0/54840635 shutdown_connections 2023-08-25T03:34:30.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.832+0000 7fcb488db700 1 -- 172.21.15.2:0/54840635 wait complete. 2023-08-25T03:34:30.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.833+0000 7fcb488db700 1 Processor -- start 2023-08-25T03:34:30.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.833+0000 7fcb488db700 1 -- start start 2023-08-25T03:34:30.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.834+0000 7fcb488db700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcb2c003bc0 0x7fcb4415b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:30.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.834+0000 7fcb488db700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb4414d530 0x7fcb440b3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:30.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.834+0000 7fcb488db700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb440b35a0 0x7fcb440b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:30.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.834+0000 7fcb488db700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb440b3a20 con 0x7fcb440b35a0 2023-08-25T03:34:30.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.834+0000 7fcb488db700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4415a4f0 con 0x7fcb4414d530 2023-08-25T03:34:30.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.834+0000 7fcb488db700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4415a760 con 0x7fcb2c003bc0 2023-08-25T03:34:30.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.834+0000 7fcb42d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb440b35a0 0x7fcb440b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:30.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.834+0000 7fcb42d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb440b35a0 0x7fcb440b4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60090/0 (socket says 172.21.15.2:60090) 2023-08-25T03:34:30.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.834+0000 7fcb42d9d700 1 -- 172.21.15.2:0/359369736 learned_addr learned my addr 172.21.15.2:0/359369736 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:30.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.834+0000 7fcb4259c700 1 --2- 172.21.15.2:0/359369736 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcb2c003bc0 0x7fcb4415b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:30.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.834+0000 7fcb41d9b700 1 --2- 172.21.15.2:0/359369736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb4414d530 0x7fcb440b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:30.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.835+0000 7fcb42d9d700 1 -- 172.21.15.2:0/359369736 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcb2c003bc0 msgr2=0x7fcb4415b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:30.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.835+0000 7fcb42d9d700 1 --2- 172.21.15.2:0/359369736 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcb2c003bc0 0x7fcb4415b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:30.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.835+0000 7fcb42d9d700 1 -- 172.21.15.2:0/359369736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb4414d530 msgr2=0x7fcb440b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:30.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.835+0000 7fcb42d9d700 1 --2- 172.21.15.2:0/359369736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb4414d530 0x7fcb440b3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:30.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.835+0000 7fcb42d9d700 1 -- 172.21.15.2:0/359369736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb4415aa40 con 0x7fcb440b35a0 2023-08-25T03:34:30.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.835+0000 7fcb42d9d700 1 --2- 172.21.15.2:0/359369736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb440b35a0 0x7fcb440b4fe0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcb4406d820 tx=0x7fcb3c00a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:30.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.835+0000 7fcb32ffd700 1 -- 172.21.15.2:0/359369736 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c00ad60 con 0x7fcb440b35a0 2023-08-25T03:34:30.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.835+0000 7fcb32ffd700 1 -- 172.21.15.2:0/359369736 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3c0133f0 con 0x7fcb440b35a0 2023-08-25T03:34:30.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.835+0000 7fcb32ffd700 1 -- 172.21.15.2:0/359369736 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3c0137a0 con 0x7fcb440b35a0 2023-08-25T03:34:30.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.835+0000 7fcb488db700 1 -- 172.21.15.2:0/359369736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb4415ad20 con 0x7fcb440b35a0 2023-08-25T03:34:30.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.835+0000 7fcb488db700 1 -- 172.21.15.2:0/359369736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb4414d9b0 con 0x7fcb440b35a0 2023-08-25T03:34:30.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.836+0000 7fcb488db700 1 -- 172.21.15.2:0/359369736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb44066f70 con 0x7fcb440b35a0 2023-08-25T03:34:30.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.836+0000 7fcb32ffd700 1 -- 172.21.15.2:0/359369736 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fcb3c013900 con 0x7fcb440b35a0 2023-08-25T03:34:30.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.837+0000 7fcb32ffd700 1 --2- 172.21.15.2:0/359369736 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fcb2406a0e0 0x7fcb2406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:30.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.837+0000 7fcb32ffd700 1 -- 172.21.15.2:0/359369736 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fcb3c088bd0 con 0x7fcb440b35a0 2023-08-25T03:34:30.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.837+0000 7fcb4259c700 1 --2- 172.21.15.2:0/359369736 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fcb2406a0e0 0x7fcb2406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:30.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.837+0000 7fcb4259c700 1 --2- 172.21.15.2:0/359369736 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fcb2406a0e0 0x7fcb2406c5a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009a80 tx=0x7fcb34008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:30.852 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:30.841+0000 7fcb32ffd700 1 -- 172.21.15.2:0/359369736 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fcb3c054350 con 0x7fcb440b35a0 2023-08-25T03:34:31.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:31.089+0000 7fcb488db700 1 -- 172.21.15.2:0/359369736 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcb4415af10 con 0x7fcb440b35a0 2023-08-25T03:34:31.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:31.089+0000 7fcb32ffd700 1 -- 172.21.15.2:0/359369736 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcb3c057970 con 0x7fcb440b35a0 2023-08-25T03:34:31.092 INFO:teuthology.orchestra.run.smithi002.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T03:34:31.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:31.091+0000 7fcb488db700 1 -- 172.21.15.2:0/359369736 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fcb2406a0e0 msgr2=0x7fcb2406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:31.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:31.091+0000 7fcb488db700 1 --2- 172.21.15.2:0/359369736 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fcb2406a0e0 0x7fcb2406c5a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009a80 tx=0x7fcb34008030 comp rx=0 tx=0).stop 2023-08-25T03:34:31.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:31.091+0000 7fcb488db700 1 -- 172.21.15.2:0/359369736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb440b35a0 msgr2=0x7fcb440b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:31.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:31.091+0000 7fcb488db700 1 --2- 172.21.15.2:0/359369736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb440b35a0 0x7fcb440b4fe0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcb4406d820 tx=0x7fcb3c00a5b0 comp rx=0 tx=0).stop 2023-08-25T03:34:31.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:31.091+0000 7fcb488db700 1 -- 172.21.15.2:0/359369736 shutdown_connections 2023-08-25T03:34:31.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:31.091+0000 7fcb488db700 1 --2- 172.21.15.2:0/359369736 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fcb2406a0e0 0x7fcb2406c5a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:31.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:31.091+0000 7fcb488db700 1 --2- 172.21.15.2:0/359369736 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fcb440b35a0 0x7fcb440b4fe0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:31.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:31.091+0000 7fcb488db700 1 --2- 172.21.15.2:0/359369736 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcb2c003bc0 0x7fcb4415b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:31.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:31.091+0000 7fcb488db700 1 --2- 172.21.15.2:0/359369736 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcb4414d530 0x7fcb440b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:31.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:31.091+0000 7fcb488db700 1 -- 172.21.15.2:0/359369736 >> 172.21.15.2:0/359369736 conn(0x7fcb440cfea0 msgr2=0x7fcb44071430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:31.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:31.092+0000 7fcb488db700 1 -- 172.21.15.2:0/359369736 shutdown_connections 2023-08-25T03:34:31.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:31.092+0000 7fcb488db700 1 -- 172.21.15.2:0/359369736 wait complete. 2023-08-25T03:34:31.107 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-08-25T03:34:31.108 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T03:34:31.108 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-08-25T03:34:31.108 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-08-25T03:34:31.108 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-08-25T03:34:31.109 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-08-25T03:34:31.109 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-08-25T03:34:31.109 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T03:34:31.109 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-08-25T03:34:31.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:31.110 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-08-25T03:34:31.129 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T03:34:31.130 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T03:34:31.130 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T03:34:31.130 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:31.130 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-08-25T03:34:31.130 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:31.130 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T03:34:31.130 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-08-25T03:34:31.131 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T03:34:31.131 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-08-25T03:34:31.131 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T03:34:31.131 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-08-25T03:34:31.131 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-25T03:34:31.132 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42294sec preferred_lft 42294sec 2023-08-25T03:34:31.132 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-08-25T03:34:31.132 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:31.132 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T03:34:31.132 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-08-25T03:34:31.132 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T03:34:31.132 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether c6:68:ce:93:64:ff brd ff:ff:ff:ff:ff:ff 2023-08-25T03:34:31.133 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-25T03:34:31.133 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:31.133 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::947b:51ff:fef5:313a/64 scope link 2023-08-25T03:34:31.133 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:31.133 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T03:34:31.133 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether c6:68:ce:93:64:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-25T03:34:31.134 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::c468:ceff:fe93:64ff/64 scope link tentative 2023-08-25T03:34:31.134 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:31.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:31.135 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T03:34:31.198 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-08-25T03:34:31.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:31.200 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-08-25T03:34:31.261 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-08-25T03:34:31.262 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T03:34:31.263 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-25T03:34:31.263 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-25T03:34:31.264 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T03:34:31.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:31.265 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-08-25T03:34:31.265 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-08-25T03:34:31.265 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-25T03:34:31.266 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-25T03:34:31.266 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-25T03:34:31.266 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-08-25T03:34:31.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:31.372 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-08-25T03:34:31.390 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-25T03:34:31.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:31.391 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list-id 2023-08-25T03:34:31.448 INFO:teuthology.orchestra.run.smithi031.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-08-25T03:34:31.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:31.449 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-08-25T03:34:31.449 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-08-25T03:34:31.449 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-25T03:34:31.450 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-08-25T03:34:31.450 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-08-25T03:34:31.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:31.546 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-08-25T03:34:31.584 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T03:34:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T03:34:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T03:34:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:31.585 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-08-25T03:34:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:31.586 INFO:teuthology.orchestra.run.smithi031.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T03:34:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 4e:7f:ce:ce:4a:b0 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-08-25T03:34:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-08-25T03:34:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::4c7f:ceff:fece:4ab0/64 scope link tentative 2023-08-25T03:34:31.586 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:31.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:31.595 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-08-25T03:34:31.670 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T03:34:31.670 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T03:34:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T03:34:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-08-25T03:34:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:31.671 INFO:teuthology.orchestra.run.smithi031.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T03:34:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 4e:7f:ce:ce:4a:b0 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-08-25T03:34:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-08-25T03:34:31.671 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:31.672 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::4c7f:ceff:fece:4ab0/64 scope link tentative 2023-08-25T03:34:31.672 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:34:31.694 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-08-25T03:34:31.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:31.696 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-08-25T03:34:31.696 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-08-25T03:34:31.698 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-08-25T03:34:31.699 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-08-25T03:34:31.700 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-08-25T03:34:31.701 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-08-25T03:34:31.702 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-08-25T03:34:31.703 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-08-25T03:34:31.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:31.940 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-08-25T03:34:31.940 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-08-25T03:34:31.940 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.1 up 2023-08-25T03:34:31.940 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set dev brx.1 master ceph-brx 2023-08-25T03:34:31.941 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-08-25T03:34:32.055 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T03:34:32.056 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-08-25T03:34:32.079 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-08-25T03:34:32.079 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T03:34:32.080 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-08-25T03:34:32.136 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-25T03:34:32.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:32.137 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-08-25T03:34:32.222 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-08-25T03:34:32.222 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-08-25T03:34:32.223 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2023-08-25T03:34:32.231 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-08-25T03:34:32.231 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-08-25T03:34:32.231 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2023-08-25T03:34:32.232 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.2:3300,172.21.15.31:3300,172.21.15.31:3301" 2023-08-25T03:34:32.232 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-25T03:34:32.261 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2023-08-25T03:34:32.262 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2023-08-25T03:34:32.262 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2023-08-25T03:34:32.262 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.2:3300,172.21.15.31:3300,172.21.15.31:3301" 2023-08-25T03:34:32.262 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-25T03:34:32.262 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.2:6789,172.21.15.31:6789,172.21.15.31:6790" 2023-08-25T03:34:32.263 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.2:6789,172.21.15.31:6789,172.21.15.31:6790:/ 2023-08-25T03:34:32.263 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=dc54136d-3683-431b-8a68-994ec618505c" will pass to kernel 2023-08-25T03:34:32.266 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-25T03:34:32.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:32.268 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-08-25T03:34:32.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:32.307 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-25T03:34:32.379 INFO:teuthology.orchestra.run.smithi031.stdout:client4490 2023-08-25T03:34:32.379 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:34:32.380 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:34:32.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:32.439 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-25T03:34:32.513 INFO:teuthology.orchestra.run.smithi031.stdout:client4490 2023-08-25T03:34:32.514 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:32.515 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-25T03:34:32.587 INFO:teuthology.orchestra.run.smithi031.stdout:client4490 2023-08-25T03:34:32.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:32.589 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/dc54136d-3683-431b-8a68-994ec618505c.client4490/status') 2023-08-25T03:34:32.669 INFO:teuthology.orchestra.run.smithi031.stdout:instance: client.4490 (3)192.168.0.2:0/2834564605 2023-08-25T03:34:32.670 INFO:teuthology.orchestra.run.smithi031.stdout:blocklisted: false 2023-08-25T03:34:32.670 INFO:teuthology.orchestra.run.smithi031.stderr:0+1 records in 2023-08-25T03:34:32.670 INFO:teuthology.orchestra.run.smithi031.stderr:0+1 records out 2023-08-25T03:34:32.670 INFO:teuthology.orchestra.run.smithi031.stderr:69 bytes copied, 2.438e-05 s, 2.8 MB/s 2023-08-25T03:34:32.671 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-08-25T03:34:32.749 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:34:32.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.887+0000 7fae04036700 1 Processor -- start 2023-08-25T03:34:32.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.888+0000 7fae04036700 1 -- start start 2023-08-25T03:34:32.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.888+0000 7fae04036700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fadfc0ae250 0x7fadfc06c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:32.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.888+0000 7fae04036700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fadfc0ce000 0x7fadfc0c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:32.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.888+0000 7fae04036700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadfc06c560 con 0x7fadfc070b20 2023-08-25T03:34:32.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.888+0000 7fae04036700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadfc0ccb30 con 0x7fadfc0ae250 2023-08-25T03:34:32.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.888+0000 7fae04036700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc0cce10 con 0x7fadfc0ce000 2023-08-25T03:34:32.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.889+0000 7fae01dd2700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fadfc0ae250 0x7fadfc06c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:32.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.889+0000 7fae015d1700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fadfc070b20 0x7fadfc0cd8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46760/0 (socket says 172.21.15.2:46760) 2023-08-25T03:34:32.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.889+0000 7fae015d1700 1 -- 172.21.15.2:0/2069622041 learned_addr learned my addr 172.21.15.2:0/2069622041 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:32.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.889+0000 7fae01dd2700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fadfc0ae250 0x7fadfc06c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60094/0 (socket says 172.21.15.2:60094) 2023-08-25T03:34:32.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.889+0000 7fae025d3700 1 --2- 172.21.15.2:0/2069622041 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fadfc0ce000 0x7fadfc0c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:32.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.889+0000 7fae01dd2700 1 -- 172.21.15.2:0/2069622041 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fadfc0ce000 msgr2=0x7fadfc0c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:32.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.889+0000 7fae01dd2700 1 --2- 172.21.15.2:0/2069622041 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fadfc0ce000 0x7fadfc0c9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:32.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.889+0000 7fae01dd2700 1 -- 172.21.15.2:0/2069622041 >> v1:172.21.15.2:6789/0 conn(0x7fadfc070b20 legacy=0x7fadfc0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:32.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.889+0000 7fae01dd2700 1 -- 172.21.15.2:0/2069622041 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadfc0cd0f0 con 0x7fadfc0ae250 2023-08-25T03:34:32.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.889+0000 7fae01dd2700 1 --2- 172.21.15.2:0/2069622041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadfc0ae250 0x7fadfc06c020 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fadf800a9e0 tx=0x7fadf800acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4e99025d05425b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:32.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.889+0000 7fae00dd0700 1 -- 172.21.15.2:0/2069622041 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf800cb70 con 0x7fadfc0ae250 2023-08-25T03:34:32.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.889+0000 7fae00dd0700 1 -- 172.21.15.2:0/2069622041 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf800ccd0 con 0x7fadfc0ae250 2023-08-25T03:34:32.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.889+0000 7fae00dd0700 1 -- 172.21.15.2:0/2069622041 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf8018620 con 0x7fadfc0ae250 2023-08-25T03:34:32.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.890+0000 7fae04036700 1 -- 172.21.15.2:0/2069622041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadfc0ae250 msgr2=0x7fadfc06c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:32.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.890+0000 7fae04036700 1 --2- 172.21.15.2:0/2069622041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadfc0ae250 0x7fadfc06c020 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fadf800a9e0 tx=0x7fadf800acf0 comp rx=0 tx=0).stop 2023-08-25T03:34:32.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.890+0000 7fae04036700 1 -- 172.21.15.2:0/2069622041 shutdown_connections 2023-08-25T03:34:32.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.890+0000 7fae04036700 1 --2- 172.21.15.2:0/2069622041 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fadfc0ce000 0x7fadfc0c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:32.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.890+0000 7fae04036700 1 --2- 172.21.15.2:0/2069622041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadfc0ae250 0x7fadfc06c020 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:32.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.890+0000 7fae04036700 1 -- 172.21.15.2:0/2069622041 >> 172.21.15.2:0/2069622041 conn(0x7fadfc0cbd30 msgr2=0x7fadfc0cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:32.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.890+0000 7fae04036700 1 -- 172.21.15.2:0/2069622041 shutdown_connections 2023-08-25T03:34:32.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.890+0000 7fae04036700 1 -- 172.21.15.2:0/2069622041 wait complete. 2023-08-25T03:34:32.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.892+0000 7fae04036700 1 Processor -- start 2023-08-25T03:34:32.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.892+0000 7fae04036700 1 -- start start 2023-08-25T03:34:32.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.893+0000 7fae04036700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadfc070b20 0x7fadfc0b4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:32.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.893+0000 7fae04036700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fadfc0ae250 0x7fadfc14ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:32.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.893+0000 7fae04036700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadfc0ce000 0x7fadfc0b2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:32.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.893+0000 7fae04036700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc14f220 con 0x7fadfc0ce000 2023-08-25T03:34:32.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.893+0000 7fae04036700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc09b250 con 0x7fadfc070b20 2023-08-25T03:34:32.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.893+0000 7fae04036700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc0c8f40 con 0x7fadfc0ae250 2023-08-25T03:34:32.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.893+0000 7fae025d3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadfc0ce000 0x7fadfc0b2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:32.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.893+0000 7fae025d3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadfc0ce000 0x7fadfc0b2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60104/0 (socket says 172.21.15.2:60104) 2023-08-25T03:34:32.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.893+0000 7fae025d3700 1 -- 172.21.15.2:0/1718177459 learned_addr learned my addr 172.21.15.2:0/1718177459 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:32.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.893+0000 7fae015d1700 1 --2- 172.21.15.2:0/1718177459 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fadfc0ae250 0x7fadfc14ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:32.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.893+0000 7fae01dd2700 1 --2- 172.21.15.2:0/1718177459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadfc070b20 0x7fadfc0b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:32.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.893+0000 7fae025d3700 1 -- 172.21.15.2:0/1718177459 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fadfc0ae250 msgr2=0x7fadfc14ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:32.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.893+0000 7fae025d3700 1 --2- 172.21.15.2:0/1718177459 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fadfc0ae250 0x7fadfc14ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:32.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.893+0000 7fae025d3700 1 -- 172.21.15.2:0/1718177459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadfc070b20 msgr2=0x7fadfc0b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:32.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.893+0000 7fae025d3700 1 --2- 172.21.15.2:0/1718177459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadfc070b20 0x7fadfc0b4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:32.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.893+0000 7fae025d3700 1 -- 172.21.15.2:0/1718177459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf800a640 con 0x7fadfc0ce000 2023-08-25T03:34:32.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.893+0000 7fae015d1700 1 --2- 172.21.15.2:0/1718177459 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fadfc0ae250 0x7fadfc14ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:32.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.893+0000 7fae01dd2700 1 --2- 172.21.15.2:0/1718177459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadfc070b20 0x7fadfc0b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:32.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.893+0000 7fae025d3700 1 --2- 172.21.15.2:0/1718177459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadfc0ce000 0x7fadfc0b2460 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fadf4009d30 tx=0x7fadf400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:32.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.893+0000 7fadee7fc700 1 -- 172.21.15.2:0/1718177459 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf4009ba0 con 0x7fadfc0ce000 2023-08-25T03:34:32.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.893+0000 7fadee7fc700 1 -- 172.21.15.2:0/1718177459 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf400f040 con 0x7fadfc0ce000 2023-08-25T03:34:32.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.893+0000 7fadee7fc700 1 -- 172.21.15.2:0/1718177459 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf4014480 con 0x7fadfc0ce000 2023-08-25T03:34:32.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.893+0000 7fae04036700 1 -- 172.21.15.2:0/1718177459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadfc0b29a0 con 0x7fadfc0ce000 2023-08-25T03:34:32.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.894+0000 7fae04036700 1 -- 172.21.15.2:0/1718177459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadfc0b2f00 con 0x7fadfc0ce000 2023-08-25T03:34:32.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.894+0000 7fae04036700 1 -- 172.21.15.2:0/1718177459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadfc0746d0 con 0x7fadfc0ce000 2023-08-25T03:34:32.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.895+0000 7fadee7fc700 1 -- 172.21.15.2:0/1718177459 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fadf40145e0 con 0x7fadfc0ce000 2023-08-25T03:34:32.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.896+0000 7fadee7fc700 1 --2- 172.21.15.2:0/1718177459 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fade006a0e0 0x7fade006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:32.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.896+0000 7fadee7fc700 1 -- 172.21.15.2:0/1718177459 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fadf40895d0 con 0x7fadfc0ce000 2023-08-25T03:34:32.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.896+0000 7fae01dd2700 1 --2- 172.21.15.2:0/1718177459 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fade006a0e0 0x7fade006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:32.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.896+0000 7fae01dd2700 1 --2- 172.21.15.2:0/1718177459 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fade006a0e0 0x7fade006c5a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fadf800ad30 tx=0x7fadf8012000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:32.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:32.900+0000 7fadee7fc700 1 -- 172.21.15.2:0/1718177459 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fadf4054da0 con 0x7fadfc0ce000 2023-08-25T03:34:32.952 DEBUG:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T03:34:32.954 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T03:34:33.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.102+0000 7fae04036700 1 -- 172.21.15.2:0/1718177459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadfc0c99f0 con 0x7fadfc0ce000 2023-08-25T03:34:33.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.104+0000 7fadee7fc700 1 -- 172.21.15.2:0/1718177459 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18135 (secure 0 0 0) 0x7fadf40583c0 con 0x7fadfc0ce000 2023-08-25T03:34:33.106 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:33.107 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":24,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:34:04.604059+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:34:00.869118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T03:34:01.987016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:34:33.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.106+0000 7fae04036700 1 -- 172.21.15.2:0/1718177459 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fade006a0e0 msgr2=0x7fade006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:33.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.106+0000 7fae04036700 1 --2- 172.21.15.2:0/1718177459 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fade006a0e0 0x7fade006c5a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fadf800ad30 tx=0x7fadf8012000 comp rx=0 tx=0).stop 2023-08-25T03:34:33.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.106+0000 7fae04036700 1 -- 172.21.15.2:0/1718177459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadfc0ce000 msgr2=0x7fadfc0b2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:33.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.106+0000 7fae04036700 1 --2- 172.21.15.2:0/1718177459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadfc0ce000 0x7fadfc0b2460 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fadf4009d30 tx=0x7fadf400c3a0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.106+0000 7fae04036700 1 -- 172.21.15.2:0/1718177459 shutdown_connections 2023-08-25T03:34:33.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.106+0000 7fae04036700 1 --2- 172.21.15.2:0/1718177459 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fade006a0e0 0x7fade006c5a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.106+0000 7fae04036700 1 --2- 172.21.15.2:0/1718177459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fadfc0ce000 0x7fadfc0b2460 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.106+0000 7fae04036700 1 --2- 172.21.15.2:0/1718177459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadfc070b20 0x7fadfc0b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.106+0000 7fae04036700 1 --2- 172.21.15.2:0/1718177459 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fadfc0ae250 0x7fadfc14ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.106+0000 7fae04036700 1 -- 172.21.15.2:0/1718177459 >> 172.21.15.2:0/1718177459 conn(0x7fadfc0cbd30 msgr2=0x7fadfc0cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:33.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.106+0000 7fae04036700 1 -- 172.21.15.2:0/1718177459 shutdown_connections 2023-08-25T03:34:33.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.106+0000 7fae04036700 1 -- 172.21.15.2:0/1718177459 wait complete. 2023-08-25T03:34:33.124 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T03:33:49.560643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T03:33:56.265184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T03:34:00.869118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T03:34:01.987016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T03:34:33.125 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T03:34:33.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.251+0000 7f82af9c9700 1 Processor -- start 2023-08-25T03:34:33.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.251+0000 7f82af9c9700 1 -- start start 2023-08-25T03:34:33.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.252+0000 7f82af9c9700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f82a806a8f0 0x7f82a80af380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:33.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.252+0000 7f82af9c9700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f82a80cdc20 0x7f82a815e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:33.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.252+0000 7f82af9c9700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a80cf4e0 con 0x7f82a815d540 2023-08-25T03:34:33.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.252+0000 7f82af9c9700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a815eb70 con 0x7f82a80cdc20 2023-08-25T03:34:33.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.252+0000 7f82af9c9700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a80712a0 con 0x7f82a806a8f0 2023-08-25T03:34:33.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.252+0000 7f82adf66700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f82a80cdc20 0x7f82a815e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:33.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.252+0000 7f82ad765700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f82a815d540 0x7f82a806a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46774/0 (socket says 172.21.15.2:46774) 2023-08-25T03:34:33.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.252+0000 7f82ad765700 1 -- 172.21.15.2:0/3192279471 learned_addr learned my addr 172.21.15.2:0/3192279471 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:33.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.252+0000 7f82acf64700 1 --2- 172.21.15.2:0/3192279471 >> v2:172.21.15.31:3300/0 conn(0x7f82a806a8f0 0x7f82a80af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:33.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.252+0000 7f82adf66700 1 -- 172.21.15.2:0/3192279471 >> v2:172.21.15.31:3300/0 conn(0x7f82a806a8f0 msgr2=0x7f82a80af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:33.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.252+0000 7f82adf66700 1 --2- 172.21.15.2:0/3192279471 >> v2:172.21.15.31:3300/0 conn(0x7f82a806a8f0 0x7f82a80af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.252+0000 7f82adf66700 1 -- 172.21.15.2:0/3192279471 >> v1:172.21.15.2:6789/0 conn(0x7f82a815d540 legacy=0x7f82a806a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:33.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.252+0000 7f829ffff700 1 -- 172.21.15.2:0/3192279471 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1693997247 0 0) 0x7f82a80cf4e0 con 0x7f82a815d540 2023-08-25T03:34:33.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.252+0000 7f82adf66700 1 -- 172.21.15.2:0/3192279471 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a80cf710 con 0x7f82a80cdc20 2023-08-25T03:34:33.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.253+0000 7f82adf66700 1 --2- 172.21.15.2:0/3192279471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82a80cdc20 0x7f82a815e630 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f82a8072a20 tx=0x7f82a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4fd83e1ebc73a1c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:33.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.253+0000 7f829ffff700 1 -- 172.21.15.2:0/3192279471 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a000e070 con 0x7f82a80cdc20 2023-08-25T03:34:33.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.253+0000 7f829ffff700 1 -- 172.21.15.2:0/3192279471 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a000bbd0 con 0x7f82a80cdc20 2023-08-25T03:34:33.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.253+0000 7f829ffff700 1 -- 172.21.15.2:0/3192279471 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a0012430 con 0x7f82a80cdc20 2023-08-25T03:34:33.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.253+0000 7f82af9c9700 1 -- 172.21.15.2:0/3192279471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82a80cdc20 msgr2=0x7f82a815e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:33.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.253+0000 7f82af9c9700 1 --2- 172.21.15.2:0/3192279471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82a80cdc20 0x7f82a815e630 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f82a8072a20 tx=0x7f82a000b330 comp rx=0 tx=0).stop 2023-08-25T03:34:33.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.253+0000 7f82af9c9700 1 -- 172.21.15.2:0/3192279471 shutdown_connections 2023-08-25T03:34:33.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.253+0000 7f82af9c9700 1 --2- 172.21.15.2:0/3192279471 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82a80cdc20 0x7f82a815e630 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.253+0000 7f82af9c9700 1 --2- 172.21.15.2:0/3192279471 >> v2:172.21.15.31:3300/0 conn(0x7f82a806a8f0 0x7f82a80af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.253+0000 7f82af9c9700 1 -- 172.21.15.2:0/3192279471 >> 172.21.15.2:0/3192279471 conn(0x7f82a80b19a0 msgr2=0x7f82a80b1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:33.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.253+0000 7f82af9c9700 1 -- 172.21.15.2:0/3192279471 shutdown_connections 2023-08-25T03:34:33.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.253+0000 7f82af9c9700 1 -- 172.21.15.2:0/3192279471 wait complete. 2023-08-25T03:34:33.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.255+0000 7f82af9c9700 1 Processor -- start 2023-08-25T03:34:33.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.255+0000 7f82af9c9700 1 -- start start 2023-08-25T03:34:33.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.255+0000 7f82af9c9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82a806a8f0 0x7f82a8159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:33.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.255+0000 7f82af9c9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a80cdc20 0x7f82a80b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:33.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.255+0000 7f82af9c9700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f82a815d540 0x7f82a80c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:33.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.255+0000 7f82af9c9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a80cce20 con 0x7f82a806a8f0 2023-08-25T03:34:33.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.255+0000 7f82af9c9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a80c6b20 con 0x7f82a80cdc20 2023-08-25T03:34:33.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.255+0000 7f82af9c9700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a80c6150 con 0x7f82a815d540 2023-08-25T03:34:33.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.256+0000 7f82ad765700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82a806a8f0 0x7f82a8159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:33.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.256+0000 7f82ad765700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82a806a8f0 0x7f82a8159570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60114/0 (socket says 172.21.15.2:60114) 2023-08-25T03:34:33.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.256+0000 7f82ad765700 1 -- 172.21.15.2:0/4149599790 learned_addr learned my addr 172.21.15.2:0/4149599790 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:33.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.256+0000 7f82acf64700 1 --2- 172.21.15.2:0/4149599790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a80cdc20 0x7f82a80b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:33.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.256+0000 7f82adf66700 1 --2- 172.21.15.2:0/4149599790 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f82a815d540 0x7f82a80c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:33.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.256+0000 7f82ad765700 1 -- 172.21.15.2:0/4149599790 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f82a815d540 msgr2=0x7f82a80c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:33.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.256+0000 7f82ad765700 1 --2- 172.21.15.2:0/4149599790 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f82a815d540 0x7f82a80c65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.256+0000 7f82ad765700 1 -- 172.21.15.2:0/4149599790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a80cdc20 msgr2=0x7f82a80b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:33.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.256+0000 7f82ad765700 1 --2- 172.21.15.2:0/4149599790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a80cdc20 0x7f82a80b3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.256+0000 7f82ad765700 1 -- 172.21.15.2:0/4149599790 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a0009a70 con 0x7f82a806a8f0 2023-08-25T03:34:33.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.256+0000 7f82ad765700 1 --2- 172.21.15.2:0/4149599790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82a806a8f0 0x7f82a8159570 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f829800c780 tx=0x7f829800ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:33.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.256+0000 7f829dffb700 1 -- 172.21.15.2:0/4149599790 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8298004c80 con 0x7f82a806a8f0 2023-08-25T03:34:33.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.256+0000 7f829dffb700 1 -- 172.21.15.2:0/4149599790 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8298004e00 con 0x7f82a806a8f0 2023-08-25T03:34:33.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.256+0000 7f82af9c9700 1 -- 172.21.15.2:0/4149599790 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a80c3550 con 0x7f82a806a8f0 2023-08-25T03:34:33.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.256+0000 7f829dffb700 1 -- 172.21.15.2:0/4149599790 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8298007310 con 0x7f82a806a8f0 2023-08-25T03:34:33.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.257+0000 7f82af9c9700 1 -- 172.21.15.2:0/4149599790 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a80c5de0 con 0x7f82a806a8f0 2023-08-25T03:34:33.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.257+0000 7f82af9c9700 1 -- 172.21.15.2:0/4149599790 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a815e2f0 con 0x7f82a806a8f0 2023-08-25T03:34:33.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.258+0000 7f829dffb700 1 -- 172.21.15.2:0/4149599790 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8298007470 con 0x7f82a806a8f0 2023-08-25T03:34:33.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.258+0000 7f829dffb700 1 --2- 172.21.15.2:0/4149599790 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f828c06a110 0x7f828c06c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:33.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.258+0000 7f829dffb700 1 -- 172.21.15.2:0/4149599790 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f82980881b0 con 0x7f82a806a8f0 2023-08-25T03:34:33.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.258+0000 7f82acf64700 1 --2- 172.21.15.2:0/4149599790 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f828c06a110 0x7f828c06c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:33.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.259+0000 7f82acf64700 1 --2- 172.21.15.2:0/4149599790 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f828c06a110 0x7f828c06c5d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f82a81f62a0 tx=0x7f82a4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:33.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.263+0000 7f829dffb700 1 -- 172.21.15.2:0/4149599790 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f82980538a0 con 0x7f82a806a8f0 2023-08-25T03:34:33.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.462+0000 7f82af9c9700 1 -- 172.21.15.2:0/4149599790 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f82a80746d0 con 0x7f82a806a8f0 2023-08-25T03:34:33.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.463+0000 7f829dffb700 1 -- 172.21.15.2:0/4149599790 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8298056ec0 con 0x7f82a806a8f0 2023-08-25T03:34:33.465 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-08-25T03:34:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.465+0000 7f82af9c9700 1 -- 172.21.15.2:0/4149599790 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f828c06a110 msgr2=0x7f828c06c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.465+0000 7f82af9c9700 1 --2- 172.21.15.2:0/4149599790 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f828c06a110 0x7f828c06c5d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f82a81f62a0 tx=0x7f82a4008040 comp rx=0 tx=0).stop 2023-08-25T03:34:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.465+0000 7f82af9c9700 1 -- 172.21.15.2:0/4149599790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82a806a8f0 msgr2=0x7f82a8159570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.465+0000 7f82af9c9700 1 --2- 172.21.15.2:0/4149599790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82a806a8f0 0x7f82a8159570 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f829800c780 tx=0x7f829800ca90 comp rx=0 tx=0).stop 2023-08-25T03:34:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.465+0000 7f82af9c9700 1 -- 172.21.15.2:0/4149599790 shutdown_connections 2023-08-25T03:34:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.465+0000 7f82af9c9700 1 --2- 172.21.15.2:0/4149599790 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f828c06a110 0x7f828c06c5d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.465+0000 7f82af9c9700 1 --2- 172.21.15.2:0/4149599790 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82a806a8f0 0x7f82a8159570 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.465+0000 7f82af9c9700 1 --2- 172.21.15.2:0/4149599790 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f82a815d540 0x7f82a80c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.465+0000 7f82af9c9700 1 --2- 172.21.15.2:0/4149599790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a80cdc20 0x7f82a80b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.465+0000 7f82af9c9700 1 -- 172.21.15.2:0/4149599790 >> 172.21.15.2:0/4149599790 conn(0x7f82a80b19a0 msgr2=0x7f82a8161890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:33.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.465+0000 7f82af9c9700 1 -- 172.21.15.2:0/4149599790 shutdown_connections 2023-08-25T03:34:33.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.465+0000 7f82af9c9700 1 -- 172.21.15.2:0/4149599790 wait complete. 2023-08-25T03:34:33.479 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T03:34:33.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.605+0000 7fb61348a700 1 Processor -- start 2023-08-25T03:34:33.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.605+0000 7fb61348a700 1 -- start start 2023-08-25T03:34:33.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.606+0000 7fb61348a700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fb60c0af180 0x7fb60c070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:33.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.606+0000 7fb61348a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb60c06a1e0 0x7fb60c06a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:33.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.606+0000 7fb61348a700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb60c160bd0 con 0x7fb60c1607f0 2023-08-25T03:34:33.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.606+0000 7fb61348a700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb60c06abd0 con 0x7fb60c0af180 2023-08-25T03:34:33.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.606+0000 7fb61348a700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb60c0713a0 con 0x7fb60c06a1e0 2023-08-25T03:34:33.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.606+0000 7fb610a25700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fb60c0af180 0x7fb60c070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.606+0000 7fb611226700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fb60c1607f0 0x7fb60c0ac8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46788/0 (socket says 172.21.15.2:46788) 2023-08-25T03:34:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.606+0000 7fb610a25700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fb60c0af180 0x7fb60c070e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60124/0 (socket says 172.21.15.2:60124) 2023-08-25T03:34:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.606+0000 7fb611226700 1 -- 172.21.15.2:0/1625172356 learned_addr learned my addr 172.21.15.2:0/1625172356 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.606+0000 7fb611a27700 1 --2- 172.21.15.2:0/1625172356 >> v2:172.21.15.31:3300/0 conn(0x7fb60c06a1e0 0x7fb60c06a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.607+0000 7fb610a25700 1 -- 172.21.15.2:0/1625172356 >> v2:172.21.15.31:3300/0 conn(0x7fb60c06a1e0 msgr2=0x7fb60c06a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.607+0000 7fb610a25700 1 --2- 172.21.15.2:0/1625172356 >> v2:172.21.15.31:3300/0 conn(0x7fb60c06a1e0 0x7fb60c06a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.607+0000 7fb610a25700 1 -- 172.21.15.2:0/1625172356 >> v1:172.21.15.2:6789/0 conn(0x7fb60c1607f0 legacy=0x7fb60c0ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:33.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.607+0000 7fb603fff700 1 -- 172.21.15.2:0/1625172356 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1257968081 0 0) 0x7fb60c160bd0 con 0x7fb60c1607f0 2023-08-25T03:34:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.607+0000 7fb610a25700 1 -- 172.21.15.2:0/1625172356 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb60c0cdd00 con 0x7fb60c0af180 2023-08-25T03:34:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.607+0000 7fb610a25700 1 --2- 172.21.15.2:0/1625172356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb60c0af180 0x7fb60c070e60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0cdc70 tx=0x7fb608009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf7923f0f6a3801 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.607+0000 7fb603fff700 1 -- 172.21.15.2:0/1625172356 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb608011830 con 0x7fb60c0af180 2023-08-25T03:34:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.607+0000 7fb603fff700 1 -- 172.21.15.2:0/1625172356 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb608011990 con 0x7fb60c0af180 2023-08-25T03:34:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.607+0000 7fb603fff700 1 -- 172.21.15.2:0/1625172356 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb608011d20 con 0x7fb60c0af180 2023-08-25T03:34:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.608+0000 7fb61348a700 1 -- 172.21.15.2:0/1625172356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb60c0af180 msgr2=0x7fb60c070e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:33.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.608+0000 7fb61348a700 1 --2- 172.21.15.2:0/1625172356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb60c0af180 0x7fb60c070e60 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0cdc70 tx=0x7fb608009b20 comp rx=0 tx=0).stop 2023-08-25T03:34:33.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.608+0000 7fb61348a700 1 -- 172.21.15.2:0/1625172356 shutdown_connections 2023-08-25T03:34:33.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.608+0000 7fb61348a700 1 --2- 172.21.15.2:0/1625172356 >> v2:172.21.15.31:3300/0 conn(0x7fb60c06a1e0 0x7fb60c06a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.608+0000 7fb61348a700 1 --2- 172.21.15.2:0/1625172356 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb60c0af180 0x7fb60c070e60 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.608+0000 7fb61348a700 1 -- 172.21.15.2:0/1625172356 >> 172.21.15.2:0/1625172356 conn(0x7fb60c0cfd90 msgr2=0x7fb60c0d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:33.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.608+0000 7fb61348a700 1 -- 172.21.15.2:0/1625172356 shutdown_connections 2023-08-25T03:34:33.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.608+0000 7fb61348a700 1 -- 172.21.15.2:0/1625172356 wait complete. 2023-08-25T03:34:33.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.609+0000 7fb61348a700 1 Processor -- start 2023-08-25T03:34:33.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.609+0000 7fb61348a700 1 -- start start 2023-08-25T03:34:33.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.610+0000 7fb61348a700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb60c06a1e0 0x7fb60c1f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:33.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.610+0000 7fb61348a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb60c0af180 0x7fb60c0c5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:33.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.610+0000 7fb61348a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb60c1607f0 0x7fb60c0c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:33.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.610+0000 7fb61348a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c0c36f0 con 0x7fb60c0af180 2023-08-25T03:34:33.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.610+0000 7fb61348a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c0c7d50 con 0x7fb60c1607f0 2023-08-25T03:34:33.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.610+0000 7fb61348a700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c0c8030 con 0x7fb60c06a1e0 2023-08-25T03:34:33.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.610+0000 7fb610a25700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb60c0af180 0x7fb60c0c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:33.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.610+0000 7fb610a25700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb60c0af180 0x7fb60c0c5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60130/0 (socket says 172.21.15.2:60130) 2023-08-25T03:34:33.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.610+0000 7fb610a25700 1 -- 172.21.15.2:0/4127970217 learned_addr learned my addr 172.21.15.2:0/4127970217 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:33.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.610+0000 7fb611a27700 1 --2- 172.21.15.2:0/4127970217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb60c1607f0 0x7fb60c0c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:33.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.610+0000 7fb611226700 1 --2- 172.21.15.2:0/4127970217 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb60c06a1e0 0x7fb60c1f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:33.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.610+0000 7fb610a25700 1 -- 172.21.15.2:0/4127970217 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb60c06a1e0 msgr2=0x7fb60c1f1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:33.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.610+0000 7fb610a25700 1 --2- 172.21.15.2:0/4127970217 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb60c06a1e0 0x7fb60c1f1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.610+0000 7fb610a25700 1 -- 172.21.15.2:0/4127970217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb60c1607f0 msgr2=0x7fb60c0c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:33.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.610+0000 7fb610a25700 1 --2- 172.21.15.2:0/4127970217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb60c1607f0 0x7fb60c0c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.610+0000 7fb610a25700 1 -- 172.21.15.2:0/4127970217 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6080094a0 con 0x7fb60c0af180 2023-08-25T03:34:33.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.610+0000 7fb611a27700 1 --2- 172.21.15.2:0/4127970217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb60c1607f0 0x7fb60c0c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:33.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.610+0000 7fb611226700 1 --2- 172.21.15.2:0/4127970217 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb60c06a1e0 0x7fb60c1f1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:33.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.610+0000 7fb610a25700 1 --2- 172.21.15.2:0/4127970217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb60c0af180 0x7fb60c0c5860 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb6080090b0 tx=0x7fb608004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:33.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.610+0000 7fb601ffb700 1 -- 172.21.15.2:0/4127970217 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6080038a0 con 0x7fb60c0af180 2023-08-25T03:34:33.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.611+0000 7fb601ffb700 1 -- 172.21.15.2:0/4127970217 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60801f070 con 0x7fb60c0af180 2023-08-25T03:34:33.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.611+0000 7fb601ffb700 1 -- 172.21.15.2:0/4127970217 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb608003c20 con 0x7fb60c0af180 2023-08-25T03:34:33.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.611+0000 7fb61348a700 1 -- 172.21.15.2:0/4127970217 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb60c0c8310 con 0x7fb60c0af180 2023-08-25T03:34:33.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.611+0000 7fb61348a700 1 -- 172.21.15.2:0/4127970217 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb60c1f2510 con 0x7fb60c0af180 2023-08-25T03:34:33.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.612+0000 7fb61348a700 1 -- 172.21.15.2:0/4127970217 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb60c0746d0 con 0x7fb60c0af180 2023-08-25T03:34:33.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.612+0000 7fb601ffb700 1 -- 172.21.15.2:0/4127970217 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb60801a430 con 0x7fb60c0af180 2023-08-25T03:34:33.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.612+0000 7fb601ffb700 1 --2- 172.21.15.2:0/4127970217 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb5f006a0e0 0x7fb5f006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:33.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.613+0000 7fb601ffb700 1 -- 172.21.15.2:0/4127970217 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb608021070 con 0x7fb60c0af180 2023-08-25T03:34:33.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.613+0000 7fb611226700 1 --2- 172.21.15.2:0/4127970217 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb5f006a0e0 0x7fb5f006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:33.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.613+0000 7fb611226700 1 --2- 172.21.15.2:0/4127970217 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb5f006a0e0 0x7fb5f006c5a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc000f60 tx=0x7fb5fc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:33.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.617+0000 7fb601ffb700 1 -- 172.21.15.2:0/4127970217 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb60808f020 con 0x7fb60c0af180 2023-08-25T03:34:33.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.817+0000 7fb61348a700 1 -- 172.21.15.2:0/4127970217 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb60c0c8830 con 0x7fb60c0af180 2023-08-25T03:34:33.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.817+0000 7fb601ffb700 1 -- 172.21.15.2:0/4127970217 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb60802b0a0 con 0x7fb60c0af180 2023-08-25T03:34:33.819 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 8 2023-08-25T03:34:33.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.819+0000 7fb61348a700 1 -- 172.21.15.2:0/4127970217 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb5f006a0e0 msgr2=0x7fb5f006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:33.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.819+0000 7fb61348a700 1 --2- 172.21.15.2:0/4127970217 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb5f006a0e0 0x7fb5f006c5a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc000f60 tx=0x7fb5fc009040 comp rx=0 tx=0).stop 2023-08-25T03:34:33.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.819+0000 7fb61348a700 1 -- 172.21.15.2:0/4127970217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb60c0af180 msgr2=0x7fb60c0c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:33.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.819+0000 7fb61348a700 1 --2- 172.21.15.2:0/4127970217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb60c0af180 0x7fb60c0c5860 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb6080090b0 tx=0x7fb608004750 comp rx=0 tx=0).stop 2023-08-25T03:34:33.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.819+0000 7fb61348a700 1 -- 172.21.15.2:0/4127970217 shutdown_connections 2023-08-25T03:34:33.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.819+0000 7fb61348a700 1 --2- 172.21.15.2:0/4127970217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb60c1607f0 0x7fb60c0c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.819+0000 7fb61348a700 1 --2- 172.21.15.2:0/4127970217 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb60c06a1e0 0x7fb60c1f1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.820+0000 7fb61348a700 1 --2- 172.21.15.2:0/4127970217 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb5f006a0e0 0x7fb5f006c5a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.820+0000 7fb61348a700 1 --2- 172.21.15.2:0/4127970217 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb60c0af180 0x7fb60c0c5860 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.820+0000 7fb61348a700 1 -- 172.21.15.2:0/4127970217 >> 172.21.15.2:0/4127970217 conn(0x7fb60c0cfd90 msgr2=0x7fb60c0ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:33.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.820+0000 7fb61348a700 1 -- 172.21.15.2:0/4127970217 shutdown_connections 2023-08-25T03:34:33.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.820+0000 7fb61348a700 1 -- 172.21.15.2:0/4127970217 wait complete. 2023-08-25T03:34:33.833 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T03:34:33.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.960+0000 7f288b71e700 1 Processor -- start 2023-08-25T03:34:33.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.960+0000 7f288b71e700 1 -- start start 2023-08-25T03:34:33.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.961+0000 7f288b71e700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f288415b170 0x7f28840cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:33.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.961+0000 7f288b71e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f28840cc640 0x7f28840ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:33.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.961+0000 7f288b71e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28840cf6f0 con 0x7f28840cd7f0 2023-08-25T03:34:33.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.961+0000 7f288b71e700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28840d1d50 con 0x7f288415b170 2023-08-25T03:34:33.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.961+0000 7f288b71e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f288415b550 con 0x7f28840cc640 2023-08-25T03:34:33.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.961+0000 7f28894ba700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f288415b170 0x7f28840cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:33.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.961+0000 7f28894ba700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f288415b170 0x7f28840cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60136/0 (socket says 172.21.15.2:60136) 2023-08-25T03:34:33.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.961+0000 7f28894ba700 1 -- 172.21.15.2:0/2200506364 learned_addr learned my addr 172.21.15.2:0/2200506364 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:33.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.961+0000 7f2888cb9700 1 --2- 172.21.15.2:0/2200506364 >> v2:172.21.15.31:3300/0 conn(0x7f28840cc640 0x7f28840ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:33.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.961+0000 7f28894ba700 1 -- 172.21.15.2:0/2200506364 >> v2:172.21.15.31:3300/0 conn(0x7f28840cc640 msgr2=0x7f28840ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:33.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.961+0000 7f28894ba700 1 --2- 172.21.15.2:0/2200506364 >> v2:172.21.15.31:3300/0 conn(0x7f28840cc640 0x7f28840ccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.961+0000 7f28894ba700 1 -- 172.21.15.2:0/2200506364 >> v1:172.21.15.31:6789/0 conn(0x7f28840cd7f0 legacy=0x7f288406d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:33.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.961+0000 7f28894ba700 1 -- 172.21.15.2:0/2200506364 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28840ceae0 con 0x7f288415b170 2023-08-25T03:34:33.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.961+0000 7f2888cb9700 1 --2- 172.21.15.2:0/2200506364 >> v2:172.21.15.31:3300/0 conn(0x7f28840cc640 0x7f28840ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:33.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.962+0000 7f28894ba700 1 --2- 172.21.15.2:0/2200506364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f288415b170 0x7f28840cfb80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f28840c7ef0 tx=0x7f2874009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8499b1647a36081c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:33.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.962+0000 7f287bfff700 1 -- 172.21.15.2:0/2200506364 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2874005e30 con 0x7f288415b170 2023-08-25T03:34:33.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.962+0000 7f287bfff700 1 -- 172.21.15.2:0/2200506364 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2874006030 con 0x7f288415b170 2023-08-25T03:34:33.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.962+0000 7f287bfff700 1 -- 172.21.15.2:0/2200506364 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28740063e0 con 0x7f288415b170 2023-08-25T03:34:33.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.962+0000 7f288b71e700 1 -- 172.21.15.2:0/2200506364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f288415b170 msgr2=0x7f28840cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:33.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.962+0000 7f288b71e700 1 --2- 172.21.15.2:0/2200506364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f288415b170 0x7f28840cfb80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f28840c7ef0 tx=0x7f2874009bd0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.962+0000 7f288b71e700 1 -- 172.21.15.2:0/2200506364 shutdown_connections 2023-08-25T03:34:33.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.962+0000 7f288b71e700 1 --2- 172.21.15.2:0/2200506364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f288415b170 0x7f28840cfb80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.962+0000 7f288b71e700 1 --2- 172.21.15.2:0/2200506364 >> v2:172.21.15.31:3300/0 conn(0x7f28840cc640 0x7f28840ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.962+0000 7f288b71e700 1 -- 172.21.15.2:0/2200506364 >> 172.21.15.2:0/2200506364 conn(0x7f288409a200 msgr2=0x7f28840d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:33.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.962+0000 7f288b71e700 1 -- 172.21.15.2:0/2200506364 shutdown_connections 2023-08-25T03:34:33.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.962+0000 7f288b71e700 1 -- 172.21.15.2:0/2200506364 wait complete. 2023-08-25T03:34:33.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.964+0000 7f288b71e700 1 Processor -- start 2023-08-25T03:34:33.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.964+0000 7f288b71e700 1 -- start start 2023-08-25T03:34:33.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.964+0000 7f288b71e700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28840cc640 0x7f28841f5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:33.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.964+0000 7f288b71e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28840cd7f0 0x7f28841f2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:33.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.964+0000 7f288b71e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f288415b170 0x7f28841f2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:33.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.964+0000 7f288b71e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28840b39f0 con 0x7f288415b170 2023-08-25T03:34:33.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.964+0000 7f288b71e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28841f2f60 con 0x7f28840cd7f0 2023-08-25T03:34:33.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.964+0000 7f288b71e700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28841f3240 con 0x7f28840cc640 2023-08-25T03:34:33.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.965+0000 7f2889cbb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f288415b170 0x7f28841f2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:33.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.965+0000 7f2889cbb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f288415b170 0x7f28841f2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60144/0 (socket says 172.21.15.2:60144) 2023-08-25T03:34:33.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.965+0000 7f2889cbb700 1 -- 172.21.15.2:0/596624811 learned_addr learned my addr 172.21.15.2:0/596624811 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:33.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.965+0000 7f2888cb9700 1 --2- 172.21.15.2:0/596624811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28840cd7f0 0x7f28841f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:33.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.965+0000 7f28894ba700 1 --2- 172.21.15.2:0/596624811 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28840cc640 0x7f28841f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:33.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.965+0000 7f2889cbb700 1 -- 172.21.15.2:0/596624811 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28840cc640 msgr2=0x7f28841f5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:33.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.965+0000 7f2889cbb700 1 --2- 172.21.15.2:0/596624811 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28840cc640 0x7f28841f5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.965+0000 7f2889cbb700 1 -- 172.21.15.2:0/596624811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28840cd7f0 msgr2=0x7f28841f2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:33.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.965+0000 7f2889cbb700 1 --2- 172.21.15.2:0/596624811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28840cd7f0 0x7f28841f2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:33.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.965+0000 7f2889cbb700 1 -- 172.21.15.2:0/596624811 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2874009500 con 0x7f288415b170 2023-08-25T03:34:33.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.965+0000 7f2889cbb700 1 --2- 172.21.15.2:0/596624811 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f288415b170 0x7f28841f2970 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f287c00cc10 tx=0x7f287c00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:33.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.965+0000 7f2879ffb700 1 -- 172.21.15.2:0/596624811 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c00aa60 con 0x7f288415b170 2023-08-25T03:34:33.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.965+0000 7f2879ffb700 1 -- 172.21.15.2:0/596624811 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f287c00abc0 con 0x7f288415b170 2023-08-25T03:34:33.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.965+0000 7f288b71e700 1 -- 172.21.15.2:0/596624811 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28840b3c40 con 0x7f288415b170 2023-08-25T03:34:33.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.965+0000 7f2879ffb700 1 -- 172.21.15.2:0/596624811 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f287c012420 con 0x7f288415b170 2023-08-25T03:34:33.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.966+0000 7f288b71e700 1 -- 172.21.15.2:0/596624811 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f288406ce40 con 0x7f288415b170 2023-08-25T03:34:33.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.967+0000 7f288b71e700 1 -- 172.21.15.2:0/596624811 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28840b4160 con 0x7f288415b170 2023-08-25T03:34:33.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.967+0000 7f2879ffb700 1 -- 172.21.15.2:0/596624811 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f287c012580 con 0x7f288415b170 2023-08-25T03:34:33.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.967+0000 7f2879ffb700 1 --2- 172.21.15.2:0/596624811 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f286806a0e0 0x7f286806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:33.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.967+0000 7f2879ffb700 1 -- 172.21.15.2:0/596624811 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f287c088770 con 0x7f288415b170 2023-08-25T03:34:33.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.967+0000 7f28894ba700 1 --2- 172.21.15.2:0/596624811 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f286806a0e0 0x7f286806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:33.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.968+0000 7f28894ba700 1 --2- 172.21.15.2:0/596624811 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f286806a0e0 0x7f286806c5a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f28840c7ef0 tx=0x7f2874008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:33.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:33.972+0000 7f2879ffb700 1 -- 172.21.15.2:0/596624811 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f28840b4160 con 0x7f288415b170 2023-08-25T03:34:34.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.172+0000 7f288b71e700 1 -- 172.21.15.2:0/596624811 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f28840b4160 con 0x7f288415b170 2023-08-25T03:34:34.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.173+0000 7f2879ffb700 1 -- 172.21.15.2:0/596624811 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f28840b4160 con 0x7f288415b170 2023-08-25T03:34:34.175 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-08-25T03:34:34.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.175+0000 7f288b71e700 1 -- 172.21.15.2:0/596624811 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f286806a0e0 msgr2=0x7f286806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:34.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.175+0000 7f288b71e700 1 --2- 172.21.15.2:0/596624811 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f286806a0e0 0x7f286806c5a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f28840c7ef0 tx=0x7f2874008070 comp rx=0 tx=0).stop 2023-08-25T03:34:34.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.175+0000 7f288b71e700 1 -- 172.21.15.2:0/596624811 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f288415b170 msgr2=0x7f28841f2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:34.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.175+0000 7f288b71e700 1 --2- 172.21.15.2:0/596624811 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f288415b170 0x7f28841f2970 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f287c00cc10 tx=0x7f287c00cf20 comp rx=0 tx=0).stop 2023-08-25T03:34:34.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.175+0000 7f288b71e700 1 -- 172.21.15.2:0/596624811 shutdown_connections 2023-08-25T03:34:34.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.175+0000 7f288b71e700 1 --2- 172.21.15.2:0/596624811 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f286806a0e0 0x7f286806c5a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:34.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.175+0000 7f288b71e700 1 --2- 172.21.15.2:0/596624811 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f288415b170 0x7f28841f2970 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:34.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.175+0000 7f288b71e700 1 --2- 172.21.15.2:0/596624811 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28840cc640 0x7f28841f5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:34.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.175+0000 7f288b71e700 1 --2- 172.21.15.2:0/596624811 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28840cd7f0 0x7f28841f2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:34.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.175+0000 7f288b71e700 1 -- 172.21.15.2:0/596624811 >> 172.21.15.2:0/596624811 conn(0x7f288409a200 msgr2=0x7f28840d0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:34.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.175+0000 7f288b71e700 1 -- 172.21.15.2:0/596624811 shutdown_connections 2023-08-25T03:34:34.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.175+0000 7f288b71e700 1 -- 172.21.15.2:0/596624811 wait complete. 2023-08-25T03:34:34.189 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T03:34:34.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.314+0000 7f509966f700 1 Processor -- start 2023-08-25T03:34:34.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.314+0000 7f509966f700 1 -- start start 2023-08-25T03:34:34.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.315+0000 7f509966f700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f50941607f0 0x7f5094070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:34.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.315+0000 7f509966f700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5094160bd0 con 0x7f50940af180 2023-08-25T03:34:34.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.315+0000 7f509966f700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50940cdd00 con 0x7f509406a1e0 2023-08-25T03:34:34.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.315+0000 7f509966f700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50940713a0 con 0x7f50941607f0 2023-08-25T03:34:34.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.315+0000 7f50927fc700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f50940af180 0x7f50940ac8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46814/0 (socket says 172.21.15.2:46814) 2023-08-25T03:34:34.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.315+0000 7f5092ffd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f50941607f0 0x7f5094070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:34.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.315+0000 7f50927fc700 1 -- 172.21.15.2:0/1616307630 learned_addr learned my addr 172.21.15.2:0/1616307630 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:34.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.316+0000 7f5092ffd700 1 -- 172.21.15.2:0/1616307630 >> v1:172.21.15.31:6789/0 conn(0x7f509406a1e0 legacy=0x7f509406ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:34.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.316+0000 7f5092ffd700 1 -- 172.21.15.2:0/1616307630 >> v1:172.21.15.2:6789/0 conn(0x7f50940af180 legacy=0x7f50940ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:34.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.316+0000 7f5092ffd700 1 -- 172.21.15.2:0/1616307630 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50940cda00 con 0x7f50941607f0 2023-08-25T03:34:34.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.316+0000 7f5091ffb700 1 -- 172.21.15.2:0/1616307630 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3214675069 0 0) 0x7f5094160bd0 con 0x7f50940af180 2023-08-25T03:34:34.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.316+0000 7f5092ffd700 1 --2- 172.21.15.2:0/1616307630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50941607f0 0x7f5094070e60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5088009840 tx=0x7f5088009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e29aba68b0e0c2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:34.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.316+0000 7f5091ffb700 1 -- 172.21.15.2:0/1616307630 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5088004230 con 0x7f50941607f0 2023-08-25T03:34:34.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.316+0000 7f5091ffb700 1 -- 172.21.15.2:0/1616307630 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5088004390 con 0x7f50941607f0 2023-08-25T03:34:34.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.316+0000 7f5091ffb700 1 -- 172.21.15.2:0/1616307630 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50880036a0 con 0x7f50941607f0 2023-08-25T03:34:34.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.316+0000 7f509966f700 1 -- 172.21.15.2:0/1616307630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50941607f0 msgr2=0x7f5094070e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:34.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.316+0000 7f509966f700 1 --2- 172.21.15.2:0/1616307630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50941607f0 0x7f5094070e60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5088009840 tx=0x7f5088009b50 comp rx=0 tx=0).stop 2023-08-25T03:34:34.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.316+0000 7f509966f700 1 -- 172.21.15.2:0/1616307630 shutdown_connections 2023-08-25T03:34:34.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.316+0000 7f509966f700 1 --2- 172.21.15.2:0/1616307630 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50941607f0 0x7f5094070e60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:34.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.317+0000 7f509966f700 1 -- 172.21.15.2:0/1616307630 >> 172.21.15.2:0/1616307630 conn(0x7f50940cfd90 msgr2=0x7f50940d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:34.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.317+0000 7f509966f700 1 -- 172.21.15.2:0/1616307630 shutdown_connections 2023-08-25T03:34:34.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.317+0000 7f509966f700 1 -- 172.21.15.2:0/1616307630 wait complete. 2023-08-25T03:34:34.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.318+0000 7f509966f700 1 Processor -- start 2023-08-25T03:34:34.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.318+0000 7f509966f700 1 -- start start 2023-08-25T03:34:34.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.319+0000 7f509966f700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f509406a1e0 0x7f50940c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:34.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.319+0000 7f509966f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50940af180 0x7f509415fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:34.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.319+0000 7f509966f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50941607f0 0x7f50940b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:34.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.319+0000 7f509966f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50940c7670 con 0x7f50940af180 2023-08-25T03:34:34.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.319+0000 7f509966f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50940c9a80 con 0x7f50941607f0 2023-08-25T03:34:34.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.319+0000 7f509966f700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f509415ff70 con 0x7f509406a1e0 2023-08-25T03:34:34.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.319+0000 7f50927fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50940af180 0x7f509415fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:34.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.319+0000 7f50927fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50940af180 0x7f509415fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60156/0 (socket says 172.21.15.2:60156) 2023-08-25T03:34:34.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.319+0000 7f50927fc700 1 -- 172.21.15.2:0/2717908110 learned_addr learned my addr 172.21.15.2:0/2717908110 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:34.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.320+0000 7f50937fe700 1 --2- 172.21.15.2:0/2717908110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50941607f0 0x7f50940b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:34.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.320+0000 7f5092ffd700 1 --2- 172.21.15.2:0/2717908110 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f509406a1e0 0x7f50940c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:34.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.320+0000 7f50927fc700 1 -- 172.21.15.2:0/2717908110 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f509406a1e0 msgr2=0x7f50940c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:34.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.320+0000 7f50927fc700 1 --2- 172.21.15.2:0/2717908110 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f509406a1e0 0x7f50940c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:34.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.320+0000 7f50927fc700 1 -- 172.21.15.2:0/2717908110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50941607f0 msgr2=0x7f50940b4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:34.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.320+0000 7f50927fc700 1 --2- 172.21.15.2:0/2717908110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50941607f0 0x7f50940b4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:34.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.320+0000 7f50927fc700 1 -- 172.21.15.2:0/2717908110 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50880094a0 con 0x7f50940af180 2023-08-25T03:34:34.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.320+0000 7f50927fc700 1 --2- 172.21.15.2:0/2717908110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50940af180 0x7f509415fa30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f50840011e0 tx=0x7f508400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:34.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.320+0000 7f507b7fe700 1 -- 172.21.15.2:0/2717908110 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084004c80 con 0x7f50940af180 2023-08-25T03:34:34.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.320+0000 7f507b7fe700 1 -- 172.21.15.2:0/2717908110 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5084004e00 con 0x7f50940af180 2023-08-25T03:34:34.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.320+0000 7f507b7fe700 1 -- 172.21.15.2:0/2717908110 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084007310 con 0x7f50940af180 2023-08-25T03:34:34.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.320+0000 7f509966f700 1 -- 172.21.15.2:0/2717908110 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50940c97a0 con 0x7f50940af180 2023-08-25T03:34:34.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.320+0000 7f509966f700 1 -- 172.21.15.2:0/2717908110 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50940c7300 con 0x7f50940af180 2023-08-25T03:34:34.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.321+0000 7f509966f700 1 -- 172.21.15.2:0/2717908110 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50940c6e40 con 0x7f50940af180 2023-08-25T03:34:34.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.322+0000 7f507b7fe700 1 -- 172.21.15.2:0/2717908110 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5084007470 con 0x7f50940af180 2023-08-25T03:34:34.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.322+0000 7f507b7fe700 1 --2- 172.21.15.2:0/2717908110 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f508006a0e0 0x7f508006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:34.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.322+0000 7f507b7fe700 1 -- 172.21.15.2:0/2717908110 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5084088140 con 0x7f50940af180 2023-08-25T03:34:34.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.322+0000 7f5092ffd700 1 --2- 172.21.15.2:0/2717908110 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f508006a0e0 0x7f508006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:34.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.323+0000 7f5092ffd700 1 --2- 172.21.15.2:0/2717908110 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f508006a0e0 0x7f508006c5a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f50880090b0 tx=0x7f5088008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:34.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.327+0000 7f507b7fe700 1 -- 172.21.15.2:0/2717908110 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5084053840 con 0x7f50940af180 2023-08-25T03:34:34.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.526+0000 7f509966f700 1 -- 172.21.15.2:0/2717908110 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f50940746d0 con 0x7f50940af180 2023-08-25T03:34:34.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.527+0000 7f507b7fe700 1 -- 172.21.15.2:0/2717908110 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5084056e60 con 0x7f50940af180 2023-08-25T03:34:34.530 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-08-25T03:34:34.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.529+0000 7f509966f700 1 -- 172.21.15.2:0/2717908110 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f508006a0e0 msgr2=0x7f508006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:34.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.529+0000 7f509966f700 1 --2- 172.21.15.2:0/2717908110 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f508006a0e0 0x7f508006c5a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f50880090b0 tx=0x7f5088008030 comp rx=0 tx=0).stop 2023-08-25T03:34:34.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.529+0000 7f509966f700 1 -- 172.21.15.2:0/2717908110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50940af180 msgr2=0x7f509415fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:34.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.529+0000 7f509966f700 1 --2- 172.21.15.2:0/2717908110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50940af180 0x7f509415fa30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f50840011e0 tx=0x7f508400ca30 comp rx=0 tx=0).stop 2023-08-25T03:34:34.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.529+0000 7f509966f700 1 -- 172.21.15.2:0/2717908110 shutdown_connections 2023-08-25T03:34:34.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.529+0000 7f509966f700 1 --2- 172.21.15.2:0/2717908110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50941607f0 0x7f50940b4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:34.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.529+0000 7f509966f700 1 --2- 172.21.15.2:0/2717908110 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f509406a1e0 0x7f50940c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:34.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.529+0000 7f509966f700 1 --2- 172.21.15.2:0/2717908110 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f508006a0e0 0x7f508006c5a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:34.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.529+0000 7f509966f700 1 --2- 172.21.15.2:0/2717908110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f50940af180 0x7f509415fa30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:34.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.529+0000 7f509966f700 1 -- 172.21.15.2:0/2717908110 >> 172.21.15.2:0/2717908110 conn(0x7f50940cfd90 msgr2=0x7f50940af800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:34.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.529+0000 7f509966f700 1 -- 172.21.15.2:0/2717908110 shutdown_connections 2023-08-25T03:34:34.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.529+0000 7f509966f700 1 -- 172.21.15.2:0/2717908110 wait complete. 2023-08-25T03:34:34.543 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:34:34.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.669+0000 7f19de0c6700 1 Processor -- start 2023-08-25T03:34:34.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.669+0000 7f19de0c6700 1 -- start start 2023-08-25T03:34:34.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.670+0000 7f19de0c6700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f19d8159ff0 0x7f19d80cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:34.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.670+0000 7f19de0c6700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f19d8071db0 0x7f19d806b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:34.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.670+0000 7f19de0c6700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f19d815a3d0 0x7f19d806b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:34.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.670+0000 7f19de0c6700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19d80cf0a0 con 0x7f19d8071db0 2023-08-25T03:34:34.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.670+0000 7f19de0c6700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19d80ce770 con 0x7f19d8159ff0 2023-08-25T03:34:34.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.670+0000 7f19de0c6700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d80cea50 con 0x7f19d815a3d0 2023-08-25T03:34:34.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.670+0000 7f19d6ffd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f19d8071db0 0x7f19d806b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:34.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.670+0000 7f19d6ffd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f19d8071db0 0x7f19d806b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60164/0 (socket says 172.21.15.2:60164) 2023-08-25T03:34:34.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.670+0000 7f19d6ffd700 1 -- 172.21.15.2:0/3533338096 learned_addr learned my addr 172.21.15.2:0/3533338096 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:34.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.670+0000 7f19d7fff700 1 --2- 172.21.15.2:0/3533338096 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f19d815a3d0 0x7f19d806b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:34.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.670+0000 7f19d77fe700 1 --2- 172.21.15.2:0/3533338096 >> v2:172.21.15.31:3300/0 conn(0x7f19d8159ff0 0x7f19d80cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:34.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.670+0000 7f19d6ffd700 1 -- 172.21.15.2:0/3533338096 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f19d815a3d0 msgr2=0x7f19d806b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:34.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.670+0000 7f19d6ffd700 1 --2- 172.21.15.2:0/3533338096 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f19d815a3d0 0x7f19d806b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:34.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.670+0000 7f19d6ffd700 1 -- 172.21.15.2:0/3533338096 >> v2:172.21.15.31:3300/0 conn(0x7f19d8159ff0 msgr2=0x7f19d80cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:34.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.670+0000 7f19d6ffd700 1 --2- 172.21.15.2:0/3533338096 >> v2:172.21.15.31:3300/0 conn(0x7f19d8159ff0 0x7f19d80cde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:34.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.670+0000 7f19d6ffd700 1 -- 172.21.15.2:0/3533338096 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d80ced30 con 0x7f19d8071db0 2023-08-25T03:34:34.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.670+0000 7f19d6ffd700 1 --2- 172.21.15.2:0/3533338096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19d8071db0 0x7f19d806b2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f19d806d550 tx=0x7f19c8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e829f9fa997179 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:34.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.671+0000 7f19d67fc700 1 -- 172.21.15.2:0/3533338096 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c800b870 con 0x7f19d8071db0 2023-08-25T03:34:34.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.671+0000 7f19d67fc700 1 -- 172.21.15.2:0/3533338096 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c800b9d0 con 0x7f19d8071db0 2023-08-25T03:34:34.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.671+0000 7f19d67fc700 1 -- 172.21.15.2:0/3533338096 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c800bd60 con 0x7f19d8071db0 2023-08-25T03:34:34.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.671+0000 7f19de0c6700 1 -- 172.21.15.2:0/3533338096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19d8071db0 msgr2=0x7f19d806b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:34.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.671+0000 7f19de0c6700 1 --2- 172.21.15.2:0/3533338096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19d8071db0 0x7f19d806b2f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f19d806d550 tx=0x7f19c8009bd0 comp rx=0 tx=0).stop 2023-08-25T03:34:34.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.671+0000 7f19de0c6700 1 -- 172.21.15.2:0/3533338096 shutdown_connections 2023-08-25T03:34:34.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.671+0000 7f19de0c6700 1 --2- 172.21.15.2:0/3533338096 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f19d815a3d0 0x7f19d806b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:34.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.671+0000 7f19de0c6700 1 --2- 172.21.15.2:0/3533338096 >> v2:172.21.15.31:3300/0 conn(0x7f19d8159ff0 0x7f19d80cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:34.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.671+0000 7f19de0c6700 1 --2- 172.21.15.2:0/3533338096 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19d8071db0 0x7f19d806b2f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:34.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.671+0000 7f19de0c6700 1 -- 172.21.15.2:0/3533338096 >> 172.21.15.2:0/3533338096 conn(0x7f19d80b1bf0 msgr2=0x7f19d80b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:34.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.671+0000 7f19de0c6700 1 -- 172.21.15.2:0/3533338096 shutdown_connections 2023-08-25T03:34:34.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.671+0000 7f19de0c6700 1 -- 172.21.15.2:0/3533338096 wait complete. 2023-08-25T03:34:34.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.672+0000 7f19de0c6700 1 Processor -- start 2023-08-25T03:34:34.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.673+0000 7f19de0c6700 1 -- start start 2023-08-25T03:34:34.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.673+0000 7f19de0c6700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f19d8071db0 0x7f19d80c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:34.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.673+0000 7f19de0c6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19d8159ff0 0x7f19d80b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:34.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.673+0000 7f19de0c6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d815a3d0 0x7f19d80af980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:34.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.673+0000 7f19de0c6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d814b6a0 con 0x7f19d8159ff0 2023-08-25T03:34:34.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.673+0000 7f19de0c6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d80b5a20 con 0x7f19d815a3d0 2023-08-25T03:34:34.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.673+0000 7f19de0c6700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d80c9b10 con 0x7f19d8071db0 2023-08-25T03:34:34.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.673+0000 7f19d6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19d8159ff0 0x7f19d80b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:34.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.673+0000 7f19d6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19d8159ff0 0x7f19d80b54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60170/0 (socket says 172.21.15.2:60170) 2023-08-25T03:34:34.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.673+0000 7f19d6ffd700 1 -- 172.21.15.2:0/2973838323 learned_addr learned my addr 172.21.15.2:0/2973838323 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:34.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.674+0000 7f19d77fe700 1 --2- 172.21.15.2:0/2973838323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f19d8071db0 0x7f19d80c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:34.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.674+0000 7f19d7fff700 1 --2- 172.21.15.2:0/2973838323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d815a3d0 0x7f19d80af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:34.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.674+0000 7f19d6ffd700 1 -- 172.21.15.2:0/2973838323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f19d8071db0 msgr2=0x7f19d80c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:34.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.674+0000 7f19d6ffd700 1 --2- 172.21.15.2:0/2973838323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f19d8071db0 0x7f19d80c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:34.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.674+0000 7f19d6ffd700 1 -- 172.21.15.2:0/2973838323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d815a3d0 msgr2=0x7f19d80af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:34.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.674+0000 7f19d6ffd700 1 --2- 172.21.15.2:0/2973838323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d815a3d0 0x7f19d80af980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:34.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.674+0000 7f19d6ffd700 1 -- 172.21.15.2:0/2973838323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c8009550 con 0x7f19d8159ff0 2023-08-25T03:34:34.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.674+0000 7f19d77fe700 1 --2- 172.21.15.2:0/2973838323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f19d8071db0 0x7f19d80c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:34.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.674+0000 7f19d7fff700 1 --2- 172.21.15.2:0/2973838323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d815a3d0 0x7f19d80af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:34.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.674+0000 7f19d6ffd700 1 --2- 172.21.15.2:0/2973838323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19d8159ff0 0x7f19d80b54e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f19c8000c00 tx=0x7f19c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:34.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.674+0000 7f19cffff700 1 -- 172.21.15.2:0/2973838323 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c8004540 con 0x7f19d8159ff0 2023-08-25T03:34:34.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.674+0000 7f19cffff700 1 -- 172.21.15.2:0/2973838323 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c80036a0 con 0x7f19d8159ff0 2023-08-25T03:34:34.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.674+0000 7f19cffff700 1 -- 172.21.15.2:0/2973838323 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c8003a50 con 0x7f19d8159ff0 2023-08-25T03:34:34.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.674+0000 7f19de0c6700 1 -- 172.21.15.2:0/2973838323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d80c9df0 con 0x7f19d8159ff0 2023-08-25T03:34:34.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.674+0000 7f19de0c6700 1 -- 172.21.15.2:0/2973838323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d80b4fc0 con 0x7f19d8159ff0 2023-08-25T03:34:34.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.675+0000 7f19de0c6700 1 -- 172.21.15.2:0/2973838323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d8066f70 con 0x7f19d8159ff0 2023-08-25T03:34:34.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.676+0000 7f19cffff700 1 -- 172.21.15.2:0/2973838323 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f19c8021070 con 0x7f19d8159ff0 2023-08-25T03:34:34.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.676+0000 7f19cffff700 1 --2- 172.21.15.2:0/2973838323 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f19b806a0e0 0x7f19b806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:34.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.676+0000 7f19cffff700 1 -- 172.21.15.2:0/2973838323 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f19c8089c40 con 0x7f19d8159ff0 2023-08-25T03:34:34.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.676+0000 7f19d77fe700 1 --2- 172.21.15.2:0/2973838323 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f19b806a0e0 0x7f19b806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:34.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.676+0000 7f19d77fe700 1 --2- 172.21.15.2:0/2973838323 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f19b806a0e0 0x7f19b806c5a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f19d80ca0d0 tx=0x7f19c0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:34.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.680+0000 7f19cffff700 1 -- 172.21.15.2:0/2973838323 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f19c80553c0 con 0x7f19d8159ff0 2023-08-25T03:34:34.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.883+0000 7f19de0c6700 1 -- 172.21.15.2:0/2973838323 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19d80746d0 con 0x7f19d8159ff0 2023-08-25T03:34:34.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.884+0000 7f19cffff700 1 -- 172.21.15.2:0/2973838323 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18135 (secure 0 0 0) 0x7f19c8029090 con 0x7f19d8159ff0 2023-08-25T03:34:34.887 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:34.887 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":24,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:34:04.604059+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:34:00.869118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T03:34:01.987016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:34:34.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.886+0000 7f19de0c6700 1 -- 172.21.15.2:0/2973838323 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f19b806a0e0 msgr2=0x7f19b806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:34.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.886+0000 7f19de0c6700 1 --2- 172.21.15.2:0/2973838323 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f19b806a0e0 0x7f19b806c5a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f19d80ca0d0 tx=0x7f19c0008040 comp rx=0 tx=0).stop 2023-08-25T03:34:34.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.886+0000 7f19de0c6700 1 -- 172.21.15.2:0/2973838323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19d8159ff0 msgr2=0x7f19d80b54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:34.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.886+0000 7f19de0c6700 1 --2- 172.21.15.2:0/2973838323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19d8159ff0 0x7f19d80b54e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f19c8000c00 tx=0x7f19c8004750 comp rx=0 tx=0).stop 2023-08-25T03:34:34.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.886+0000 7f19de0c6700 1 -- 172.21.15.2:0/2973838323 shutdown_connections 2023-08-25T03:34:34.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.886+0000 7f19de0c6700 1 --2- 172.21.15.2:0/2973838323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f19d815a3d0 0x7f19d80af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:34.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.886+0000 7f19de0c6700 1 --2- 172.21.15.2:0/2973838323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f19d8071db0 0x7f19d80c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:34.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.886+0000 7f19de0c6700 1 --2- 172.21.15.2:0/2973838323 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f19b806a0e0 0x7f19b806c5a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:34.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.886+0000 7f19de0c6700 1 --2- 172.21.15.2:0/2973838323 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f19d8159ff0 0x7f19d80b54e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:34.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.886+0000 7f19de0c6700 1 -- 172.21.15.2:0/2973838323 >> 172.21.15.2:0/2973838323 conn(0x7f19d80b1bf0 msgr2=0x7f19d80b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:34.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.886+0000 7f19de0c6700 1 -- 172.21.15.2:0/2973838323 shutdown_connections 2023-08-25T03:34:34.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:34.886+0000 7f19de0c6700 1 -- 172.21.15.2:0/2973838323 wait complete. 2023-08-25T03:34:34.901 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T03:33:49.560643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T03:33:56.265184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T03:34:00.869118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T03:34:01.987016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T03:34:34.902 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T03:34:35.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.029+0000 7f90a4507700 1 Processor -- start 2023-08-25T03:34:35.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.029+0000 7f90a4507700 1 -- start start 2023-08-25T03:34:35.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.030+0000 7f90a4507700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f909c1449a0 0x7f909c06fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:35.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.030+0000 7f90a4507700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f909c0ce7e0 0x7f909c0cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:35.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.030+0000 7f90a4507700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f909c0cc8e0 con 0x7f909c06be40 2023-08-25T03:34:35.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.030+0000 7f90a4507700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f909c0c9360 con 0x7f909c0ce7e0 2023-08-25T03:34:35.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.030+0000 7f90a4507700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f909c0c9640 con 0x7f909c1449a0 2023-08-25T03:34:35.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.030+0000 7f90a1aa2700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f909c0ce7e0 0x7f909c0cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:35.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.030+0000 7f90a1aa2700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f909c0ce7e0 0x7f909c0cb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60176/0 (socket says 172.21.15.2:60176) 2023-08-25T03:34:35.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.030+0000 7f90a1aa2700 1 -- 172.21.15.2:0/128899292 learned_addr learned my addr 172.21.15.2:0/128899292 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:35.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.030+0000 7f90a22a3700 1 --2- 172.21.15.2:0/128899292 >> v2:172.21.15.31:3300/0 conn(0x7f909c1449a0 0x7f909c06fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:35.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.030+0000 7f90a1aa2700 1 -- 172.21.15.2:0/128899292 >> v2:172.21.15.31:3300/0 conn(0x7f909c1449a0 msgr2=0x7f909c06fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:35.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.030+0000 7f90a1aa2700 1 --2- 172.21.15.2:0/128899292 >> v2:172.21.15.31:3300/0 conn(0x7f909c1449a0 0x7f909c06fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.030+0000 7f90a1aa2700 1 -- 172.21.15.2:0/128899292 >> v1:172.21.15.31:6789/0 conn(0x7f909c06be40 legacy=0x7f909c0cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:35.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.030+0000 7f90a1aa2700 1 -- 172.21.15.2:0/128899292 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f909c0ccb80 con 0x7f909c0ce7e0 2023-08-25T03:34:35.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.030+0000 7f90a22a3700 1 --2- 172.21.15.2:0/128899292 >> v2:172.21.15.31:3300/0 conn(0x7f909c1449a0 0x7f909c06fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:35.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.030+0000 7f90a1aa2700 1 --2- 172.21.15.2:0/128899292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f909c0ce7e0 0x7f909c0cb9d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f909c06e230 tx=0x7f908c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25dfce5bf94b64be server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:35.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.031+0000 7f90a12a1700 1 -- 172.21.15.2:0/128899292 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c004230 con 0x7f909c0ce7e0 2023-08-25T03:34:35.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.031+0000 7f90a12a1700 1 -- 172.21.15.2:0/128899292 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f908c004390 con 0x7f909c0ce7e0 2023-08-25T03:34:35.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.031+0000 7f90a12a1700 1 -- 172.21.15.2:0/128899292 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908c0036a0 con 0x7f909c0ce7e0 2023-08-25T03:34:35.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.031+0000 7f90a4507700 1 -- 172.21.15.2:0/128899292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f909c0ce7e0 msgr2=0x7f909c0cb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:35.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.031+0000 7f90a4507700 1 --2- 172.21.15.2:0/128899292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f909c0ce7e0 0x7f909c0cb9d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f909c06e230 tx=0x7f908c009bd0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.031+0000 7f90a4507700 1 -- 172.21.15.2:0/128899292 shutdown_connections 2023-08-25T03:34:35.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.031+0000 7f90a4507700 1 --2- 172.21.15.2:0/128899292 >> v2:172.21.15.31:3300/0 conn(0x7f909c1449a0 0x7f909c06fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.031+0000 7f90a4507700 1 --2- 172.21.15.2:0/128899292 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f909c0ce7e0 0x7f909c0cb9d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.031+0000 7f90a4507700 1 -- 172.21.15.2:0/128899292 >> 172.21.15.2:0/128899292 conn(0x7f909c0b1240 msgr2=0x7f909c06a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:35.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.031+0000 7f90a4507700 1 -- 172.21.15.2:0/128899292 shutdown_connections 2023-08-25T03:34:35.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.031+0000 7f90a4507700 1 -- 172.21.15.2:0/128899292 wait complete. 2023-08-25T03:34:35.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.032+0000 7f90a4507700 1 Processor -- start 2023-08-25T03:34:35.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.032+0000 7f90a4507700 1 -- start start 2023-08-25T03:34:35.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.033+0000 7f90a4507700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f909c06be40 0x7f909c13fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:35.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.033+0000 7f90a4507700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f909c0ce7e0 0x7f909c15c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:35.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.033+0000 7f90a4507700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f909c1449a0 0x7f909c0c5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:35.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.033+0000 7f90a4507700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c144100 con 0x7f909c1449a0 2023-08-25T03:34:35.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.033+0000 7f90a4507700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f909c0c5de0 con 0x7f909c0ce7e0 2023-08-25T03:34:35.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.033+0000 7f90a4507700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f909c1f32d0 con 0x7f909c06be40 2023-08-25T03:34:35.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.033+0000 7f90a2aa4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f909c1449a0 0x7f909c0c5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:35.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.033+0000 7f90a2aa4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f909c1449a0 0x7f909c0c5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60182/0 (socket says 172.21.15.2:60182) 2023-08-25T03:34:35.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.033+0000 7f90a2aa4700 1 -- 172.21.15.2:0/3234794247 learned_addr learned my addr 172.21.15.2:0/3234794247 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:35.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.033+0000 7f90a1aa2700 1 --2- 172.21.15.2:0/3234794247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f909c0ce7e0 0x7f909c15c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:35.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.033+0000 7f90a22a3700 1 --2- 172.21.15.2:0/3234794247 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f909c06be40 0x7f909c13fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:35.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.034+0000 7f90a2aa4700 1 -- 172.21.15.2:0/3234794247 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f909c06be40 msgr2=0x7f909c13fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:35.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.034+0000 7f90a2aa4700 1 --2- 172.21.15.2:0/3234794247 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f909c06be40 0x7f909c13fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.034+0000 7f90a2aa4700 1 -- 172.21.15.2:0/3234794247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f909c0ce7e0 msgr2=0x7f909c15c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:35.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.034+0000 7f90a2aa4700 1 --2- 172.21.15.2:0/3234794247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f909c0ce7e0 0x7f909c15c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.034+0000 7f90a2aa4700 1 -- 172.21.15.2:0/3234794247 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f908c009550 con 0x7f909c1449a0 2023-08-25T03:34:35.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.034+0000 7f90a2aa4700 1 --2- 172.21.15.2:0/3234794247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f909c1449a0 0x7f909c0c5050 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f909400cc10 tx=0x7f909400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:35.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.034+0000 7f9092ffd700 1 -- 172.21.15.2:0/3234794247 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909400aa60 con 0x7f909c1449a0 2023-08-25T03:34:35.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.034+0000 7f9092ffd700 1 -- 172.21.15.2:0/3234794247 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909400abc0 con 0x7f909c1449a0 2023-08-25T03:34:35.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.034+0000 7f90a4507700 1 -- 172.21.15.2:0/3234794247 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f909c0c6050 con 0x7f909c1449a0 2023-08-25T03:34:35.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.034+0000 7f9092ffd700 1 -- 172.21.15.2:0/3234794247 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9094012420 con 0x7f909c1449a0 2023-08-25T03:34:35.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.034+0000 7f90a4507700 1 -- 172.21.15.2:0/3234794247 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f909c0c63e0 con 0x7f909c1449a0 2023-08-25T03:34:35.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.035+0000 7f90a4507700 1 -- 172.21.15.2:0/3234794247 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f909c0c6570 con 0x7f909c1449a0 2023-08-25T03:34:35.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.035+0000 7f9092ffd700 1 -- 172.21.15.2:0/3234794247 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9094012580 con 0x7f909c1449a0 2023-08-25T03:34:35.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.036+0000 7f9092ffd700 1 --2- 172.21.15.2:0/3234794247 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f908006a0e0 0x7f908006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:35.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.036+0000 7f9092ffd700 1 -- 172.21.15.2:0/3234794247 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9094088820 con 0x7f909c1449a0 2023-08-25T03:34:35.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.036+0000 7f90a22a3700 1 --2- 172.21.15.2:0/3234794247 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f908006a0e0 0x7f908006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:35.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.036+0000 7f90a22a3700 1 --2- 172.21.15.2:0/3234794247 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f908006a0e0 0x7f908006c5a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9098006fd0 tx=0x7f9098008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:35.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.041+0000 7f9092ffd700 1 -- 172.21.15.2:0/3234794247 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f909c0c6570 con 0x7f909c1449a0 2023-08-25T03:34:35.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.241+0000 7f90a4507700 1 -- 172.21.15.2:0/3234794247 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f909c0c6570 con 0x7f909c1449a0 2023-08-25T03:34:35.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.241+0000 7f9092ffd700 1 -- 172.21.15.2:0/3234794247 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f909c0c6570 con 0x7f909c1449a0 2023-08-25T03:34:35.243 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-08-25T03:34:35.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.243+0000 7f90a4507700 1 -- 172.21.15.2:0/3234794247 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f908006a0e0 msgr2=0x7f908006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:35.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.243+0000 7f90a4507700 1 --2- 172.21.15.2:0/3234794247 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f908006a0e0 0x7f908006c5a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9098006fd0 tx=0x7f9098008040 comp rx=0 tx=0).stop 2023-08-25T03:34:35.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.243+0000 7f90a4507700 1 -- 172.21.15.2:0/3234794247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f909c1449a0 msgr2=0x7f909c0c5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:35.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.243+0000 7f90a4507700 1 --2- 172.21.15.2:0/3234794247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f909c1449a0 0x7f909c0c5050 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f909400cc10 tx=0x7f909400cf20 comp rx=0 tx=0).stop 2023-08-25T03:34:35.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.243+0000 7f90a4507700 1 -- 172.21.15.2:0/3234794247 shutdown_connections 2023-08-25T03:34:35.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.243+0000 7f90a4507700 1 --2- 172.21.15.2:0/3234794247 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f908006a0e0 0x7f908006c5a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.243+0000 7f90a4507700 1 --2- 172.21.15.2:0/3234794247 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f909c1449a0 0x7f909c0c5050 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.243+0000 7f90a4507700 1 --2- 172.21.15.2:0/3234794247 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f909c06be40 0x7f909c13fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.243+0000 7f90a4507700 1 --2- 172.21.15.2:0/3234794247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f909c0ce7e0 0x7f909c15c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.243+0000 7f90a4507700 1 -- 172.21.15.2:0/3234794247 >> 172.21.15.2:0/3234794247 conn(0x7f909c0b1240 msgr2=0x7f909c0cd130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:35.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.243+0000 7f90a4507700 1 -- 172.21.15.2:0/3234794247 shutdown_connections 2023-08-25T03:34:35.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.243+0000 7f90a4507700 1 -- 172.21.15.2:0/3234794247 wait complete. 2023-08-25T03:34:35.257 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T03:34:35.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.383+0000 7f6d4a5df700 1 Processor -- start 2023-08-25T03:34:35.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.383+0000 7f6d4a5df700 1 -- start start 2023-08-25T03:34:35.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.384+0000 7f6d4a5df700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6d441607f0 0x7f6d44070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:35.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.384+0000 7f6d4a5df700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d44160bd0 con 0x7f6d440af180 2023-08-25T03:34:35.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.384+0000 7f6d4a5df700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d440cdd00 con 0x7f6d4406a1e0 2023-08-25T03:34:35.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.384+0000 7f6d4a5df700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d440713a0 con 0x7f6d441607f0 2023-08-25T03:34:35.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.384+0000 7f6d437fe700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f6d440af180 0x7f6d440ac8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46856/0 (socket says 172.21.15.2:46856) 2023-08-25T03:34:35.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.384+0000 7f6d437fe700 1 -- 172.21.15.2:0/3583008805 learned_addr learned my addr 172.21.15.2:0/3583008805 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:35.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.384+0000 7f6d43fff700 1 --2- 172.21.15.2:0/3583008805 >> v2:172.21.15.31:3300/0 conn(0x7f6d441607f0 0x7f6d44070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:35.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.384+0000 7f6d42ffd700 1 -- 172.21.15.2:0/3583008805 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 26587529 0 0) 0x7f6d44160bd0 con 0x7f6d440af180 2023-08-25T03:34:35.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.384+0000 7f6d42ffd700 1 -- 172.21.15.2:0/3583008805 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d300035b0 con 0x7f6d440af180 2023-08-25T03:34:35.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.384+0000 7f6d43fff700 1 -- 172.21.15.2:0/3583008805 >> v1:172.21.15.31:6789/0 conn(0x7f6d4406a1e0 legacy=0x7f6d4406ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:35.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.384+0000 7f6d43fff700 1 -- 172.21.15.2:0/3583008805 >> v1:172.21.15.2:6789/0 conn(0x7f6d440af180 legacy=0x7f6d440ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:35.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.384+0000 7f6d43fff700 1 -- 172.21.15.2:0/3583008805 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d440cda00 con 0x7f6d441607f0 2023-08-25T03:34:35.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.385+0000 7f6d43fff700 1 --2- 172.21.15.2:0/3583008805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d441607f0 0x7f6d44070e60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6d34009840 tx=0x7f6d34009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=476e286884867793 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:35.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.385+0000 7f6d42ffd700 1 -- 172.21.15.2:0/3583008805 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d34011830 con 0x7f6d441607f0 2023-08-25T03:34:35.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.385+0000 7f6d42ffd700 1 -- 172.21.15.2:0/3583008805 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d34011990 con 0x7f6d441607f0 2023-08-25T03:34:35.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.385+0000 7f6d42ffd700 1 -- 172.21.15.2:0/3583008805 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d34011d00 con 0x7f6d441607f0 2023-08-25T03:34:35.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.385+0000 7f6d4a5df700 1 -- 172.21.15.2:0/3583008805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d441607f0 msgr2=0x7f6d44070e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:35.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.385+0000 7f6d4a5df700 1 --2- 172.21.15.2:0/3583008805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d441607f0 0x7f6d44070e60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f6d34009840 tx=0x7f6d34009b50 comp rx=0 tx=0).stop 2023-08-25T03:34:35.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.385+0000 7f6d4a5df700 1 -- 172.21.15.2:0/3583008805 shutdown_connections 2023-08-25T03:34:35.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.385+0000 7f6d4a5df700 1 --2- 172.21.15.2:0/3583008805 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d441607f0 0x7f6d44070e60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.385+0000 7f6d4a5df700 1 -- 172.21.15.2:0/3583008805 >> 172.21.15.2:0/3583008805 conn(0x7f6d440cfd90 msgr2=0x7f6d440d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:35.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.385+0000 7f6d4a5df700 1 -- 172.21.15.2:0/3583008805 shutdown_connections 2023-08-25T03:34:35.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.385+0000 7f6d4a5df700 1 -- 172.21.15.2:0/3583008805 wait complete. 2023-08-25T03:34:35.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.387+0000 7f6d4a5df700 1 Processor -- start 2023-08-25T03:34:35.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.387+0000 7f6d4a5df700 1 -- start start 2023-08-25T03:34:35.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.388+0000 7f6d4a5df700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d4406a1e0 0x7f6d440c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:35.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.388+0000 7f6d4a5df700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6d440af180 0x7f6d440c67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:35.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.388+0000 7f6d4a5df700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d441607f0 0x7f6d440cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:35.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.388+0000 7f6d4a5df700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d440c36f0 con 0x7f6d4406a1e0 2023-08-25T03:34:35.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.388+0000 7f6d4a5df700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d440c38d0 con 0x7f6d441607f0 2023-08-25T03:34:35.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.388+0000 7f6d4a5df700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d440c3bb0 con 0x7f6d440af180 2023-08-25T03:34:35.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.388+0000 7f6d43fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d4406a1e0 0x7f6d440c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:35.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.388+0000 7f6d43fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d4406a1e0 0x7f6d440c9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60194/0 (socket says 172.21.15.2:60194) 2023-08-25T03:34:35.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.388+0000 7f6d43fff700 1 -- 172.21.15.2:0/1089299690 learned_addr learned my addr 172.21.15.2:0/1089299690 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:35.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.388+0000 7f6d48b7c700 1 --2- 172.21.15.2:0/1089299690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d441607f0 0x7f6d440cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:35.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.388+0000 7f6d437fe700 1 --2- 172.21.15.2:0/1089299690 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6d440af180 0x7f6d440c67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:35.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.388+0000 7f6d43fff700 1 -- 172.21.15.2:0/1089299690 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6d440af180 msgr2=0x7f6d440c67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:35.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.388+0000 7f6d43fff700 1 --2- 172.21.15.2:0/1089299690 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6d440af180 0x7f6d440c67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.388+0000 7f6d43fff700 1 -- 172.21.15.2:0/1089299690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d441607f0 msgr2=0x7f6d440cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:35.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.388+0000 7f6d43fff700 1 --2- 172.21.15.2:0/1089299690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d441607f0 0x7f6d440cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.388+0000 7f6d43fff700 1 -- 172.21.15.2:0/1089299690 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d340094a0 con 0x7f6d4406a1e0 2023-08-25T03:34:35.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.388+0000 7f6d43fff700 1 --2- 172.21.15.2:0/1089299690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d4406a1e0 0x7f6d440c9380 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6d34009fd0 tx=0x7f6d34004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:35.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.388+0000 7f6d40ff9700 1 -- 172.21.15.2:0/1089299690 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d340036a0 con 0x7f6d4406a1e0 2023-08-25T03:34:35.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.388+0000 7f6d40ff9700 1 -- 172.21.15.2:0/1089299690 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3401e070 con 0x7f6d4406a1e0 2023-08-25T03:34:35.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.388+0000 7f6d4a5df700 1 -- 172.21.15.2:0/1089299690 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d440c3e90 con 0x7f6d4406a1e0 2023-08-25T03:34:35.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.388+0000 7f6d40ff9700 1 -- 172.21.15.2:0/1089299690 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d34003a40 con 0x7f6d4406a1e0 2023-08-25T03:34:35.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.388+0000 7f6d4a5df700 1 -- 172.21.15.2:0/1089299690 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d440c4450 con 0x7f6d4406a1e0 2023-08-25T03:34:35.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.389+0000 7f6d4a5df700 1 -- 172.21.15.2:0/1089299690 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d440c3cf0 con 0x7f6d4406a1e0 2023-08-25T03:34:35.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.390+0000 7f6d40ff9700 1 -- 172.21.15.2:0/1089299690 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6d34003ba0 con 0x7f6d4406a1e0 2023-08-25T03:34:35.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.390+0000 7f6d40ff9700 1 --2- 172.21.15.2:0/1089299690 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6d2406a0e0 0x7f6d2406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:35.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.390+0000 7f6d40ff9700 1 -- 172.21.15.2:0/1089299690 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6d34089610 con 0x7f6d4406a1e0 2023-08-25T03:34:35.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.390+0000 7f6d437fe700 1 --2- 172.21.15.2:0/1089299690 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6d2406a0e0 0x7f6d2406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:35.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.391+0000 7f6d437fe700 1 --2- 172.21.15.2:0/1089299690 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6d2406a0e0 0x7f6d2406c5a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6d440712f0 tx=0x7f6d2c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:35.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.395+0000 7f6d40ff9700 1 -- 172.21.15.2:0/1089299690 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6d440c3cf0 con 0x7f6d4406a1e0 2023-08-25T03:34:35.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.595+0000 7f6d4a5df700 1 -- 172.21.15.2:0/1089299690 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6d440c3cf0 con 0x7f6d4406a1e0 2023-08-25T03:34:35.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.595+0000 7f6d40ff9700 1 -- 172.21.15.2:0/1089299690 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6d440c3cf0 con 0x7f6d4406a1e0 2023-08-25T03:34:35.597 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 8 2023-08-25T03:34:35.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.597+0000 7f6d4a5df700 1 -- 172.21.15.2:0/1089299690 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6d2406a0e0 msgr2=0x7f6d2406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:35.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.597+0000 7f6d4a5df700 1 --2- 172.21.15.2:0/1089299690 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6d2406a0e0 0x7f6d2406c5a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6d440712f0 tx=0x7f6d2c009040 comp rx=0 tx=0).stop 2023-08-25T03:34:35.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.597+0000 7f6d4a5df700 1 -- 172.21.15.2:0/1089299690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d4406a1e0 msgr2=0x7f6d440c9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:35.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.597+0000 7f6d4a5df700 1 --2- 172.21.15.2:0/1089299690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d4406a1e0 0x7f6d440c9380 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6d34009fd0 tx=0x7f6d34004750 comp rx=0 tx=0).stop 2023-08-25T03:34:35.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.597+0000 7f6d4a5df700 1 -- 172.21.15.2:0/1089299690 shutdown_connections 2023-08-25T03:34:35.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.597+0000 7f6d4a5df700 1 --2- 172.21.15.2:0/1089299690 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6d2406a0e0 0x7f6d2406c5a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.597+0000 7f6d4a5df700 1 --2- 172.21.15.2:0/1089299690 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d4406a1e0 0x7f6d440c9380 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.597+0000 7f6d4a5df700 1 --2- 172.21.15.2:0/1089299690 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d441607f0 0x7f6d440cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.597+0000 7f6d4a5df700 1 --2- 172.21.15.2:0/1089299690 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6d440af180 0x7f6d440c67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.597+0000 7f6d4a5df700 1 -- 172.21.15.2:0/1089299690 >> 172.21.15.2:0/1089299690 conn(0x7f6d440cfd90 msgr2=0x7f6d440af7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:35.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.597+0000 7f6d4a5df700 1 -- 172.21.15.2:0/1089299690 shutdown_connections 2023-08-25T03:34:35.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.597+0000 7f6d4a5df700 1 -- 172.21.15.2:0/1089299690 wait complete. 2023-08-25T03:34:35.611 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T03:34:35.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.739+0000 7f4af2403700 1 Processor -- start 2023-08-25T03:34:35.741 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.739+0000 7f4af2403700 1 -- start start 2023-08-25T03:34:35.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.740+0000 7f4af2403700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4aec159f10 0x7f4aec15dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:35.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.740+0000 7f4af2403700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4aec0cacf0 0x7f4aec0cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:35.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.740+0000 7f4af2403700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aec06eaf0 con 0x7f4aec0cedf0 2023-08-25T03:34:35.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.740+0000 7f4af2403700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aec06e850 con 0x7f4aec0cacf0 2023-08-25T03:34:35.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.740+0000 7f4af2403700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec15e020 con 0x7f4aec159f10 2023-08-25T03:34:35.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.740+0000 7f4aeb7fe700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f4aec0cedf0 0x7f4aec06ef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46870/0 (socket says 172.21.15.2:46870) 2023-08-25T03:34:35.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.740+0000 7f4aeb7fe700 1 -- 172.21.15.2:0/1087086707 learned_addr learned my addr 172.21.15.2:0/1087086707 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:35.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.740+0000 7f4af09a0700 1 --2- 172.21.15.2:0/1087086707 >> v2:172.21.15.31:3300/0 conn(0x7f4aec0cacf0 0x7f4aec0cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:35.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.740+0000 7f4aebfff700 1 --2- 172.21.15.2:0/1087086707 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4aec159f10 0x7f4aec15dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:35.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.740+0000 7f4aeaffd700 1 -- 172.21.15.2:0/1087086707 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 775813783 0 0) 0x7f4aec06eaf0 con 0x7f4aec0cedf0 2023-08-25T03:34:35.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.740+0000 7f4aeaffd700 1 -- 172.21.15.2:0/1087086707 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ad80057c0 con 0x7f4aec0cedf0 2023-08-25T03:34:35.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.740+0000 7f4aeaffd700 1 -- 172.21.15.2:0/1087086707 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1807174039 0 0) 0x7f4ad80057c0 con 0x7f4aec0cedf0 2023-08-25T03:34:35.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.740+0000 7f4aeaffd700 1 -- 172.21.15.2:0/1087086707 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4aec159f10 msgr2=0x7f4aec15dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:35.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.740+0000 7f4aeaffd700 1 --2- 172.21.15.2:0/1087086707 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4aec159f10 0x7f4aec15dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.740+0000 7f4aeaffd700 1 -- 172.21.15.2:0/1087086707 >> v2:172.21.15.31:3300/0 conn(0x7f4aec0cacf0 msgr2=0x7f4aec0cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:35.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.740+0000 7f4aeaffd700 1 --2- 172.21.15.2:0/1087086707 >> v2:172.21.15.31:3300/0 conn(0x7f4aec0cacf0 0x7f4aec0cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.740+0000 7f4aeaffd700 1 -- 172.21.15.2:0/1087086707 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aec15e2c0 con 0x7f4aec0cedf0 2023-08-25T03:34:35.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.740+0000 7f4aeaffd700 1 -- 172.21.15.2:0/1087086707 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f4adc001670 con 0x7f4aec0cedf0 2023-08-25T03:34:35.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.740+0000 7f4af09a0700 1 --2- 172.21.15.2:0/1087086707 >> v2:172.21.15.31:3300/0 conn(0x7f4aec0cacf0 0x7f4aec0cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:34:35.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.741+0000 7f4aeaffd700 1 -- 172.21.15.2:0/1087086707 >> v1:172.21.15.2:6789/0 conn(0x7f4aec0cedf0 legacy=0x7f4aec06ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:35.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.741+0000 7f4aeaffd700 1 --2- 172.21.15.2:0/1087086707 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4aec159f10 0x7f4ad8005c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:35.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.741+0000 7f4aeaffd700 1 --2- 172.21.15.2:0/1087086707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ad8006190 0x7f4ad8006a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:35.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.741+0000 7f4aeaffd700 1 --2- 172.21.15.2:0/1087086707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ad8006fa0 0x7f4ad8007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:35.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.741+0000 7f4aeaffd700 1 -- 172.21.15.2:0/1087086707 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec06e850 con 0x7f4ad8006fa0 2023-08-25T03:34:35.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.741+0000 7f4aeaffd700 1 -- 172.21.15.2:0/1087086707 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec15e020 con 0x7f4ad8006190 2023-08-25T03:34:35.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.741+0000 7f4aeaffd700 1 -- 172.21.15.2:0/1087086707 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad800dc20 con 0x7f4aec159f10 2023-08-25T03:34:35.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.741+0000 7f4af09a0700 1 --2- 172.21.15.2:0/1087086707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ad8006fa0 0x7f4ad8007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:35.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.741+0000 7f4aebfff700 1 --2- 172.21.15.2:0/1087086707 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4aec159f10 0x7f4ad8005c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:35.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.741+0000 7f4aeb7fe700 1 --2- 172.21.15.2:0/1087086707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ad8006190 0x7f4ad8006a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:35.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.741+0000 7f4af09a0700 1 -- 172.21.15.2:0/1087086707 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4aec159f10 msgr2=0x7f4ad8005c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:35.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.741+0000 7f4af09a0700 1 --2- 172.21.15.2:0/1087086707 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4aec159f10 0x7f4ad8005c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.741+0000 7f4af09a0700 1 -- 172.21.15.2:0/1087086707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ad8006190 msgr2=0x7f4ad8006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:35.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.741+0000 7f4af09a0700 1 --2- 172.21.15.2:0/1087086707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ad8006190 0x7f4ad8006a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.741+0000 7f4af09a0700 1 -- 172.21.15.2:0/1087086707 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad80057c0 con 0x7f4ad8006fa0 2023-08-25T03:34:35.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.741+0000 7f4af09a0700 1 --2- 172.21.15.2:0/1087086707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ad8006fa0 0x7f4ad8007420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400e350 tx=0x7f4ad8007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0b9e7a997633ee server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:35.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.741+0000 7f4aeaffd700 1 -- 172.21.15.2:0/1087086707 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae400ed90 con 0x7f4ad8006fa0 2023-08-25T03:34:35.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.741+0000 7f4aeaffd700 1 -- 172.21.15.2:0/1087086707 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae400f040 con 0x7f4ad8006fa0 2023-08-25T03:34:35.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.741+0000 7f4aeaffd700 1 -- 172.21.15.2:0/1087086707 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae40146a0 con 0x7f4ad8006fa0 2023-08-25T03:34:35.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.742+0000 7f4af2403700 1 -- 172.21.15.2:0/1087086707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ad8006fa0 msgr2=0x7f4ad8007420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:35.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.742+0000 7f4af2403700 1 --2- 172.21.15.2:0/1087086707 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ad8006fa0 0x7f4ad8007420 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f4ae400e350 tx=0x7f4ad8007a90 comp rx=0 tx=0).stop 2023-08-25T03:34:35.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.742+0000 7f4af09a0700 1 -- 172.21.15.2:0/1087086707 reap_dead start 2023-08-25T03:34:35.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.742+0000 7f4af2403700 1 -- 172.21.15.2:0/1087086707 shutdown_connections 2023-08-25T03:34:35.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.742+0000 7f4af2403700 1 -- 172.21.15.2:0/1087086707 >> 172.21.15.2:0/1087086707 conn(0x7f4aec0cfd90 msgr2=0x7f4aec0d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:35.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.742+0000 7f4af2403700 1 -- 172.21.15.2:0/1087086707 shutdown_connections 2023-08-25T03:34:35.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.742+0000 7f4af2403700 1 -- 172.21.15.2:0/1087086707 wait complete. 2023-08-25T03:34:35.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.743+0000 7f4af2403700 1 Processor -- start 2023-08-25T03:34:35.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.743+0000 7f4af2403700 1 -- start start 2023-08-25T03:34:35.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.744+0000 7f4af2403700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ad8006fa0 0x7f4aec14d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:35.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.744+0000 7f4af2403700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4aec0ada20 0x7f4aec15cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:35.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.744+0000 7f4af2403700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4aec15d190 0x7f4aec070e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:35.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.744+0000 7f4af2403700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec15c7c0 con 0x7f4aec15d190 2023-08-25T03:34:35.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.744+0000 7f4af2403700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec15bbb0 con 0x7f4ad8006fa0 2023-08-25T03:34:35.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.744+0000 7f4af2403700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aec15be90 con 0x7f4aec0ada20 2023-08-25T03:34:35.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.744+0000 7f4af09a0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4aec15d190 0x7f4aec070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:35.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.744+0000 7f4aebfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ad8006fa0 0x7f4aec14d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:35.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.744+0000 7f4aebfff700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ad8006fa0 0x7f4aec14d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.2:51504/0 (socket says 172.21.15.2:51504) 2023-08-25T03:34:35.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.744+0000 7f4af09a0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4aec15d190 0x7f4aec070e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60218/0 (socket says 172.21.15.2:60218) 2023-08-25T03:34:35.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.744+0000 7f4aebfff700 1 -- 172.21.15.2:0/4119299214 learned_addr learned my addr 172.21.15.2:0/4119299214 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:35.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.744+0000 7f4aeb7fe700 1 --2- 172.21.15.2:0/4119299214 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4aec0ada20 0x7f4aec15cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:35.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.744+0000 7f4af09a0700 1 -- 172.21.15.2:0/4119299214 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4aec0ada20 msgr2=0x7f4aec15cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:35.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.744+0000 7f4af09a0700 1 --2- 172.21.15.2:0/4119299214 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4aec0ada20 0x7f4aec15cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.744+0000 7f4af09a0700 1 -- 172.21.15.2:0/4119299214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ad8006fa0 msgr2=0x7f4aec14d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:35.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.744+0000 7f4af09a0700 1 --2- 172.21.15.2:0/4119299214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ad8006fa0 0x7f4aec14d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.744+0000 7f4af09a0700 1 -- 172.21.15.2:0/4119299214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aec15c170 con 0x7f4aec15d190 2023-08-25T03:34:35.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.745+0000 7f4af09a0700 1 --2- 172.21.15.2:0/4119299214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4aec15d190 0x7f4aec070e40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4019000 tx=0x7f4ae4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:35.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.745+0000 7f4ae8ff9700 1 -- 172.21.15.2:0/4119299214 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4007000 con 0x7f4aec15d190 2023-08-25T03:34:35.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.745+0000 7f4ae8ff9700 1 -- 172.21.15.2:0/4119299214 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae4004470 con 0x7f4aec15d190 2023-08-25T03:34:35.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.745+0000 7f4af2403700 1 -- 172.21.15.2:0/4119299214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aec15c450 con 0x7f4aec15d190 2023-08-25T03:34:35.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.745+0000 7f4ae8ff9700 1 -- 172.21.15.2:0/4119299214 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae4004820 con 0x7f4aec15d190 2023-08-25T03:34:35.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.745+0000 7f4af2403700 1 -- 172.21.15.2:0/4119299214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aec0adea0 con 0x7f4aec15d190 2023-08-25T03:34:35.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.746+0000 7f4af2403700 1 -- 172.21.15.2:0/4119299214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aec0746d0 con 0x7f4aec15d190 2023-08-25T03:34:35.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.746+0000 7f4ae8ff9700 1 -- 172.21.15.2:0/4119299214 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4ae4014480 con 0x7f4aec15d190 2023-08-25T03:34:35.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.747+0000 7f4ae8ff9700 1 --2- 172.21.15.2:0/4119299214 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f4acc06a0e0 0x7f4acc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:35.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.747+0000 7f4ae8ff9700 1 -- 172.21.15.2:0/4119299214 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4ae408a090 con 0x7f4aec15d190 2023-08-25T03:34:35.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.747+0000 7f4aebfff700 1 --2- 172.21.15.2:0/4119299214 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f4acc06a0e0 0x7f4acc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:35.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.747+0000 7f4aebfff700 1 --2- 172.21.15.2:0/4119299214 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f4acc06a0e0 0x7f4acc06c5a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009d70 tx=0x7f4ad4009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:35.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.751+0000 7f4ae8ff9700 1 -- 172.21.15.2:0/4119299214 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4ae4059340 con 0x7f4aec15d190 2023-08-25T03:34:35.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.952+0000 7f4af2403700 1 -- 172.21.15.2:0/4119299214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4aec15c900 con 0x7f4aec15d190 2023-08-25T03:34:35.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.952+0000 7f4ae8ff9700 1 -- 172.21.15.2:0/4119299214 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4ae4009de0 con 0x7f4aec15d190 2023-08-25T03:34:35.956 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-08-25T03:34:35.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.954+0000 7f4af2403700 1 -- 172.21.15.2:0/4119299214 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f4acc06a0e0 msgr2=0x7f4acc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:35.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.954+0000 7f4af2403700 1 --2- 172.21.15.2:0/4119299214 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f4acc06a0e0 0x7f4acc06c5a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4009d70 tx=0x7f4ad4009210 comp rx=0 tx=0).stop 2023-08-25T03:34:35.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.954+0000 7f4af2403700 1 -- 172.21.15.2:0/4119299214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4aec15d190 msgr2=0x7f4aec070e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:35.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.954+0000 7f4af2403700 1 --2- 172.21.15.2:0/4119299214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4aec15d190 0x7f4aec070e40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4ae4019000 tx=0x7f4ae4009750 comp rx=0 tx=0).stop 2023-08-25T03:34:35.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.955+0000 7f4af2403700 1 -- 172.21.15.2:0/4119299214 shutdown_connections 2023-08-25T03:34:35.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.955+0000 7f4af2403700 1 --2- 172.21.15.2:0/4119299214 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f4acc06a0e0 0x7f4acc06c5a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.955+0000 7f4af2403700 1 --2- 172.21.15.2:0/4119299214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4aec15d190 0x7f4aec070e40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.955+0000 7f4af2403700 1 --2- 172.21.15.2:0/4119299214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ad8006fa0 0x7f4aec14d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.955+0000 7f4af2403700 1 --2- 172.21.15.2:0/4119299214 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4aec0ada20 0x7f4aec15cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:35.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.955+0000 7f4af2403700 1 -- 172.21.15.2:0/4119299214 >> 172.21.15.2:0/4119299214 conn(0x7f4aec0cfd90 msgr2=0x7f4aec0af770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:35.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.955+0000 7f4af2403700 1 -- 172.21.15.2:0/4119299214 shutdown_connections 2023-08-25T03:34:35.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:35.955+0000 7f4af2403700 1 -- 172.21.15.2:0/4119299214 wait complete. 2023-08-25T03:34:35.969 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T03:34:36.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.096+0000 7f4842829700 1 Processor -- start 2023-08-25T03:34:36.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.096+0000 7f4842829700 1 -- start start 2023-08-25T03:34:36.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.096+0000 7f4842829700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f483c159f10 0x7f483c15dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:36.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.096+0000 7f4842829700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f483c0cacf0 0x7f483c0cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:36.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.096+0000 7f4842829700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f483c06eaf0 con 0x7f483c0cedf0 2023-08-25T03:34:36.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.096+0000 7f4842829700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f483c06e850 con 0x7f483c0cacf0 2023-08-25T03:34:36.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.096+0000 7f4842829700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f483c15e020 con 0x7f483c159f10 2023-08-25T03:34:36.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.097+0000 7f4840dc6700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f483c0cacf0 0x7f483c0cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:36.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.097+0000 7f483b7fe700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f483c0cedf0 0x7f483c06ef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46888/0 (socket says 172.21.15.2:46888) 2023-08-25T03:34:36.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.097+0000 7f483b7fe700 1 -- 172.21.15.2:0/3664927075 learned_addr learned my addr 172.21.15.2:0/3664927075 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:36.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.097+0000 7f483bfff700 1 --2- 172.21.15.2:0/3664927075 >> v2:172.21.15.31:3300/0 conn(0x7f483c159f10 0x7f483c15dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:36.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.097+0000 7f4840dc6700 1 -- 172.21.15.2:0/3664927075 >> v2:172.21.15.31:3300/0 conn(0x7f483c159f10 msgr2=0x7f483c15dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:36.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.097+0000 7f4840dc6700 1 --2- 172.21.15.2:0/3664927075 >> v2:172.21.15.31:3300/0 conn(0x7f483c159f10 0x7f483c15dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:36.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.097+0000 7f4840dc6700 1 -- 172.21.15.2:0/3664927075 >> v1:172.21.15.2:6789/0 conn(0x7f483c0cedf0 legacy=0x7f483c06ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:36.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.097+0000 7f483affd700 1 -- 172.21.15.2:0/3664927075 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2220697307 0 0) 0x7f483c06eaf0 con 0x7f483c0cedf0 2023-08-25T03:34:36.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.097+0000 7f4840dc6700 1 -- 172.21.15.2:0/3664927075 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483c15e2c0 con 0x7f483c0cacf0 2023-08-25T03:34:36.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.097+0000 7f4840dc6700 1 --2- 172.21.15.2:0/3664927075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f483c0cacf0 0x7f483c0cb1a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f483c071fd0 tx=0x7f483400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b45b4a92d9c776 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:36.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.097+0000 7f483affd700 1 -- 172.21.15.2:0/3664927075 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483400e070 con 0x7f483c0cacf0 2023-08-25T03:34:36.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.097+0000 7f483affd700 1 -- 172.21.15.2:0/3664927075 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483400bb20 con 0x7f483c0cacf0 2023-08-25T03:34:36.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.097+0000 7f483affd700 1 -- 172.21.15.2:0/3664927075 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4834012430 con 0x7f483c0cacf0 2023-08-25T03:34:36.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.098+0000 7f4842829700 1 -- 172.21.15.2:0/3664927075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f483c0cacf0 msgr2=0x7f483c0cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:36.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.098+0000 7f4842829700 1 --2- 172.21.15.2:0/3664927075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f483c0cacf0 0x7f483c0cb1a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f483c071fd0 tx=0x7f483400b330 comp rx=0 tx=0).stop 2023-08-25T03:34:36.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.098+0000 7f4842829700 1 -- 172.21.15.2:0/3664927075 shutdown_connections 2023-08-25T03:34:36.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.098+0000 7f4842829700 1 --2- 172.21.15.2:0/3664927075 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f483c0cacf0 0x7f483c0cb1a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:36.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.098+0000 7f4842829700 1 --2- 172.21.15.2:0/3664927075 >> v2:172.21.15.31:3300/0 conn(0x7f483c159f10 0x7f483c15dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:36.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.098+0000 7f4842829700 1 -- 172.21.15.2:0/3664927075 >> 172.21.15.2:0/3664927075 conn(0x7f483c0cfd90 msgr2=0x7f483c0d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:36.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.098+0000 7f4842829700 1 -- 172.21.15.2:0/3664927075 shutdown_connections 2023-08-25T03:34:36.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.098+0000 7f4842829700 1 -- 172.21.15.2:0/3664927075 wait complete. 2023-08-25T03:34:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.099+0000 7f4842829700 1 Processor -- start 2023-08-25T03:34:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.099+0000 7f4842829700 1 -- start start 2023-08-25T03:34:36.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.100+0000 7f4842829700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f483c0cacf0 0x7f483c147a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.100+0000 7f4842829700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f483c0cedf0 0x7f483c0b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.100+0000 7f4842829700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f483c159f10 0x7f483c148af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.100+0000 7f4842829700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483c147fc0 con 0x7f483c0cacf0 2023-08-25T03:34:36.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.100+0000 7f4842829700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483c0b59a0 con 0x7f483c159f10 2023-08-25T03:34:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.100+0000 7f4842829700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f483c0b46a0 con 0x7f483c0cedf0 2023-08-25T03:34:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.100+0000 7f483bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f483c0cacf0 0x7f483c147a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.100+0000 7f483bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f483c0cacf0 0x7f483c147a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60226/0 (socket says 172.21.15.2:60226) 2023-08-25T03:34:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.100+0000 7f483bfff700 1 -- 172.21.15.2:0/2179732756 learned_addr learned my addr 172.21.15.2:0/2179732756 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.100+0000 7f4840dc6700 1 --2- 172.21.15.2:0/2179732756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f483c159f10 0x7f483c148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:36.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.100+0000 7f483b7fe700 1 --2- 172.21.15.2:0/2179732756 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f483c0cedf0 0x7f483c0b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.100+0000 7f483bfff700 1 -- 172.21.15.2:0/2179732756 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f483c0cedf0 msgr2=0x7f483c0b5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.100+0000 7f483bfff700 1 --2- 172.21.15.2:0/2179732756 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f483c0cedf0 0x7f483c0b5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.100+0000 7f483bfff700 1 -- 172.21.15.2:0/2179732756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f483c159f10 msgr2=0x7f483c148af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.101+0000 7f483bfff700 1 --2- 172.21.15.2:0/2179732756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f483c159f10 0x7f483c148af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.101+0000 7f483bfff700 1 -- 172.21.15.2:0/2179732756 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4834009a70 con 0x7f483c0cacf0 2023-08-25T03:34:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.101+0000 7f483bfff700 1 --2- 172.21.15.2:0/2179732756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f483c0cacf0 0x7f483c147a80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f482c00e870 tx=0x7f482c00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:36.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.101+0000 7f4838ff9700 1 -- 172.21.15.2:0/2179732756 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c009590 con 0x7f483c0cacf0 2023-08-25T03:34:36.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.101+0000 7f4838ff9700 1 -- 172.21.15.2:0/2179732756 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f482c013680 con 0x7f483c0cacf0 2023-08-25T03:34:36.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.101+0000 7f4842829700 1 -- 172.21.15.2:0/2179732756 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f483c0b4980 con 0x7f483c0cacf0 2023-08-25T03:34:36.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.101+0000 7f4838ff9700 1 -- 172.21.15.2:0/2179732756 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f482c0139f0 con 0x7f483c0cacf0 2023-08-25T03:34:36.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.101+0000 7f4842829700 1 -- 172.21.15.2:0/2179732756 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f483c0b4f40 con 0x7f483c0cacf0 2023-08-25T03:34:36.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.102+0000 7f4842829700 1 -- 172.21.15.2:0/2179732756 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f483c0b5130 con 0x7f483c0cacf0 2023-08-25T03:34:36.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.102+0000 7f4838ff9700 1 -- 172.21.15.2:0/2179732756 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f482c013b50 con 0x7f483c0cacf0 2023-08-25T03:34:36.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.103+0000 7f4838ff9700 1 --2- 172.21.15.2:0/2179732756 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f482806a0e0 0x7f482806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:36.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.103+0000 7f4838ff9700 1 -- 172.21.15.2:0/2179732756 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f482c088320 con 0x7f483c0cacf0 2023-08-25T03:34:36.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.103+0000 7f483b7fe700 1 --2- 172.21.15.2:0/2179732756 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f482806a0e0 0x7f482806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:36.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.103+0000 7f483b7fe700 1 --2- 172.21.15.2:0/2179732756 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f482806a0e0 0x7f482806c5a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4830007fd0 tx=0x7f4830007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:36.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.107+0000 7f4838ff9700 1 -- 172.21.15.2:0/2179732756 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f482c057530 con 0x7f483c0cacf0 2023-08-25T03:34:36.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.308+0000 7f4842829700 1 -- 172.21.15.2:0/2179732756 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f483c0746d0 con 0x7f483c0cacf0 2023-08-25T03:34:36.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.308+0000 7f4838ff9700 1 -- 172.21.15.2:0/2179732756 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f482c0570c0 con 0x7f483c0cacf0 2023-08-25T03:34:36.311 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-08-25T03:34:36.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.310+0000 7f4842829700 1 -- 172.21.15.2:0/2179732756 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f482806a0e0 msgr2=0x7f482806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:36.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.310+0000 7f4842829700 1 --2- 172.21.15.2:0/2179732756 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f482806a0e0 0x7f482806c5a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4830007fd0 tx=0x7f4830007de0 comp rx=0 tx=0).stop 2023-08-25T03:34:36.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.310+0000 7f4842829700 1 -- 172.21.15.2:0/2179732756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f483c0cacf0 msgr2=0x7f483c147a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:36.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.310+0000 7f4842829700 1 --2- 172.21.15.2:0/2179732756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f483c0cacf0 0x7f483c147a80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f482c00e870 tx=0x7f482c00eb80 comp rx=0 tx=0).stop 2023-08-25T03:34:36.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.310+0000 7f4842829700 1 -- 172.21.15.2:0/2179732756 shutdown_connections 2023-08-25T03:34:36.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.310+0000 7f4842829700 1 --2- 172.21.15.2:0/2179732756 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f482806a0e0 0x7f482806c5a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:36.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.310+0000 7f4842829700 1 --2- 172.21.15.2:0/2179732756 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f483c0cacf0 0x7f483c147a80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:36.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.310+0000 7f4842829700 1 --2- 172.21.15.2:0/2179732756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f483c159f10 0x7f483c148af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:36.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.310+0000 7f4842829700 1 --2- 172.21.15.2:0/2179732756 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f483c0cedf0 0x7f483c0b5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:36.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.310+0000 7f4842829700 1 -- 172.21.15.2:0/2179732756 >> 172.21.15.2:0/2179732756 conn(0x7f483c0cfd90 msgr2=0x7f483c0ccfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:36.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.310+0000 7f4842829700 1 -- 172.21.15.2:0/2179732756 shutdown_connections 2023-08-25T03:34:36.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.310+0000 7f4842829700 1 -- 172.21.15.2:0/2179732756 wait complete. 2023-08-25T03:34:36.324 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:34:36.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.448+0000 7f14bb8bc700 1 Processor -- start 2023-08-25T03:34:36.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.448+0000 7f14bb8bc700 1 -- start start 2023-08-25T03:34:36.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.449+0000 7f14bb8bc700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f14b406b1f0 0x7f14b406b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:36.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.449+0000 7f14bb8bc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f14b40b4480 0x7f14b415c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:36.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.449+0000 7f14bb8bc700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b40b3bb0 con 0x7f14b40b1c40 2023-08-25T03:34:36.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.449+0000 7f14bb8bc700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b406bb10 con 0x7f14b40b4480 2023-08-25T03:34:36.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.449+0000 7f14bb8bc700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b40c4760 con 0x7f14b406b1f0 2023-08-25T03:34:36.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.449+0000 7f14b8e57700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f14b40b1c40 0x7f14b40ce260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46900/0 (socket says 172.21.15.2:46900) 2023-08-25T03:34:36.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.449+0000 7f14b8e57700 1 -- 172.21.15.2:0/2988702415 learned_addr learned my addr 172.21.15.2:0/2988702415 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:36.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.449+0000 7f14b9e59700 1 --2- 172.21.15.2:0/2988702415 >> v2:172.21.15.31:3300/0 conn(0x7f14b40b4480 0x7f14b415c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:36.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.449+0000 7f14b9658700 1 --2- 172.21.15.2:0/2988702415 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f14b406b1f0 0x7f14b406b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:36.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.450+0000 7f14abfff700 1 -- 172.21.15.2:0/2988702415 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 15093429 0 0) 0x7f14b40b3bb0 con 0x7f14b40b1c40 2023-08-25T03:34:36.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.450+0000 7f14abfff700 1 -- 172.21.15.2:0/2988702415 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14a00035b0 con 0x7f14b40b1c40 2023-08-25T03:34:36.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.450+0000 7f14b9e59700 1 -- 172.21.15.2:0/2988702415 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f14b406b1f0 msgr2=0x7f14b406b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:36.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.450+0000 7f14b9e59700 1 --2- 172.21.15.2:0/2988702415 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f14b406b1f0 0x7f14b406b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:36.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.450+0000 7f14b9e59700 1 -- 172.21.15.2:0/2988702415 >> v1:172.21.15.2:6789/0 conn(0x7f14b40b1c40 legacy=0x7f14b40ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:36.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.450+0000 7f14b9e59700 1 -- 172.21.15.2:0/2988702415 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b40b3560 con 0x7f14b40b4480 2023-08-25T03:34:36.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.450+0000 7f14b9e59700 1 --2- 172.21.15.2:0/2988702415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14b40b4480 0x7f14b415c2a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009d90 tx=0x7f14ac00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9eb8a206dc4ad960 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:36.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.450+0000 7f14abfff700 1 -- 172.21.15.2:0/2988702415 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac00e070 con 0x7f14b40b4480 2023-08-25T03:34:36.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.450+0000 7f14abfff700 1 -- 172.21.15.2:0/2988702415 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ac00bb20 con 0x7f14b40b4480 2023-08-25T03:34:36.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.450+0000 7f14abfff700 1 -- 172.21.15.2:0/2988702415 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac00be90 con 0x7f14b40b4480 2023-08-25T03:34:36.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.451+0000 7f14bb8bc700 1 -- 172.21.15.2:0/2988702415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14b40b4480 msgr2=0x7f14b415c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:36.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.451+0000 7f14bb8bc700 1 --2- 172.21.15.2:0/2988702415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14b40b4480 0x7f14b415c2a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009d90 tx=0x7f14ac00b330 comp rx=0 tx=0).stop 2023-08-25T03:34:36.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.451+0000 7f14bb8bc700 1 -- 172.21.15.2:0/2988702415 shutdown_connections 2023-08-25T03:34:36.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.451+0000 7f14bb8bc700 1 --2- 172.21.15.2:0/2988702415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14b40b4480 0x7f14b415c2a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:36.463 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.451+0000 7f14bb8bc700 1 --2- 172.21.15.2:0/2988702415 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f14b406b1f0 0x7f14b406b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:36.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.451+0000 7f14bb8bc700 1 -- 172.21.15.2:0/2988702415 >> 172.21.15.2:0/2988702415 conn(0x7f14b40cfd10 msgr2=0x7f14b406ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:36.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.451+0000 7f14bb8bc700 1 -- 172.21.15.2:0/2988702415 shutdown_connections 2023-08-25T03:34:36.464 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.451+0000 7f14bb8bc700 1 -- 172.21.15.2:0/2988702415 wait complete. 2023-08-25T03:34:36.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.452+0000 7f14bb8bc700 1 Processor -- start 2023-08-25T03:34:36.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.452+0000 7f14bb8bc700 1 -- start start 2023-08-25T03:34:36.465 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.453+0000 7f14bb8bc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14b406b1f0 0x7f14b4154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:36.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.453+0000 7f14bb8bc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14b40b1c40 0x7f14b415ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:36.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.453+0000 7f14bb8bc700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f14b40b4480 0x7f14b415bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:36.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.453+0000 7f14bb8bc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b4154070 con 0x7f14b406b1f0 2023-08-25T03:34:36.466 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.453+0000 7f14bb8bc700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b4153d90 con 0x7f14b40b1c40 2023-08-25T03:34:36.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.453+0000 7f14bb8bc700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b41577f0 con 0x7f14b40b4480 2023-08-25T03:34:36.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.453+0000 7f14b9658700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14b406b1f0 0x7f14b4154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:36.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.454+0000 7f14b9658700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14b406b1f0 0x7f14b4154910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60240/0 (socket says 172.21.15.2:60240) 2023-08-25T03:34:36.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.454+0000 7f14b9658700 1 -- 172.21.15.2:0/303791104 learned_addr learned my addr 172.21.15.2:0/303791104 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:36.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.454+0000 7f14b8e57700 1 --2- 172.21.15.2:0/303791104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14b40b1c40 0x7f14b415ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:36.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.454+0000 7f14b9658700 1 -- 172.21.15.2:0/303791104 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f14b40b4480 msgr2=0x7f14b415bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:36.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.454+0000 7f14b9658700 1 --2- 172.21.15.2:0/303791104 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f14b40b4480 0x7f14b415bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:36.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.454+0000 7f14b9658700 1 -- 172.21.15.2:0/303791104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14b40b1c40 msgr2=0x7f14b415ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:36.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.454+0000 7f14b9658700 1 --2- 172.21.15.2:0/303791104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14b40b1c40 0x7f14b415ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:36.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.454+0000 7f14b9658700 1 -- 172.21.15.2:0/303791104 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac009a70 con 0x7f14b406b1f0 2023-08-25T03:34:36.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.454+0000 7f14b8e57700 1 --2- 172.21.15.2:0/303791104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14b40b1c40 0x7f14b415ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:36.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.454+0000 7f14b9658700 1 --2- 172.21.15.2:0/303791104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14b406b1f0 0x7f14b4154910 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f14a400e9b0 tx=0x7f14a400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:36.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.454+0000 7f14a9ffb700 1 -- 172.21.15.2:0/303791104 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a4009710 con 0x7f14b406b1f0 2023-08-25T03:34:36.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.454+0000 7f14a9ffb700 1 -- 172.21.15.2:0/303791104 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a400c850 con 0x7f14b406b1f0 2023-08-25T03:34:36.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.454+0000 7f14bb8bc700 1 -- 172.21.15.2:0/303791104 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b4157ad0 con 0x7f14b406b1f0 2023-08-25T03:34:36.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.454+0000 7f14a9ffb700 1 -- 172.21.15.2:0/303791104 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a400cbc0 con 0x7f14b406b1f0 2023-08-25T03:34:36.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.455+0000 7f14bb8bc700 1 -- 172.21.15.2:0/303791104 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b4154e50 con 0x7f14b406b1f0 2023-08-25T03:34:36.471 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.455+0000 7f14bb8bc700 1 -- 172.21.15.2:0/303791104 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b4157650 con 0x7f14b406b1f0 2023-08-25T03:34:36.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.456+0000 7f14a9ffb700 1 -- 172.21.15.2:0/303791104 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f14a400cd20 con 0x7f14b406b1f0 2023-08-25T03:34:36.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.456+0000 7f14a9ffb700 1 --2- 172.21.15.2:0/303791104 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f149806a0e0 0x7f149806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:36.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.456+0000 7f14a9ffb700 1 -- 172.21.15.2:0/303791104 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f14a4088700 con 0x7f14b406b1f0 2023-08-25T03:34:36.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.456+0000 7f14b8e57700 1 --2- 172.21.15.2:0/303791104 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f149806a0e0 0x7f149806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:36.473 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.457+0000 7f14b8e57700 1 --2- 172.21.15.2:0/303791104 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f149806a0e0 0x7f149806c5a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f14b4156370 tx=0x7f14b0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:36.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.461+0000 7f14a9ffb700 1 -- 172.21.15.2:0/303791104 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f14b4157650 con 0x7f14b406b1f0 2023-08-25T03:34:36.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.663+0000 7f14bb8bc700 1 -- 172.21.15.2:0/303791104 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14b4157650 con 0x7f14b406b1f0 2023-08-25T03:34:36.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.664+0000 7f14a9ffb700 1 -- 172.21.15.2:0/303791104 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18135 (secure 0 0 0) 0x7f14b4157650 con 0x7f14b406b1f0 2023-08-25T03:34:36.666 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:36.666 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":24,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:34:04.604059+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:34:00.869118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T03:34:01.987016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:34:36.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.666+0000 7f14bb8bc700 1 -- 172.21.15.2:0/303791104 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f149806a0e0 msgr2=0x7f149806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:36.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.666+0000 7f14bb8bc700 1 --2- 172.21.15.2:0/303791104 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f149806a0e0 0x7f149806c5a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f14b4156370 tx=0x7f14b0009040 comp rx=0 tx=0).stop 2023-08-25T03:34:36.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.666+0000 7f14bb8bc700 1 -- 172.21.15.2:0/303791104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14b406b1f0 msgr2=0x7f14b4154910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:36.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.666+0000 7f14bb8bc700 1 --2- 172.21.15.2:0/303791104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14b406b1f0 0x7f14b4154910 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f14a400e9b0 tx=0x7f14a400ed70 comp rx=0 tx=0).stop 2023-08-25T03:34:36.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.666+0000 7f14bb8bc700 1 -- 172.21.15.2:0/303791104 shutdown_connections 2023-08-25T03:34:36.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.666+0000 7f14bb8bc700 1 --2- 172.21.15.2:0/303791104 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f149806a0e0 0x7f149806c5a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:36.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.666+0000 7f14bb8bc700 1 --2- 172.21.15.2:0/303791104 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f14b406b1f0 0x7f14b4154910 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:36.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.666+0000 7f14bb8bc700 1 --2- 172.21.15.2:0/303791104 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f14b40b4480 0x7f14b415bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:36.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.666+0000 7f14bb8bc700 1 --2- 172.21.15.2:0/303791104 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14b40b1c40 0x7f14b415ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:36.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.666+0000 7f14bb8bc700 1 -- 172.21.15.2:0/303791104 >> 172.21.15.2:0/303791104 conn(0x7f14b40cfd10 msgr2=0x7f14b4070fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:36.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.666+0000 7f14bb8bc700 1 -- 172.21.15.2:0/303791104 shutdown_connections 2023-08-25T03:34:36.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.666+0000 7f14bb8bc700 1 -- 172.21.15.2:0/303791104 wait complete. 2023-08-25T03:34:36.680 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T03:33:49.560643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T03:33:56.265184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T03:34:00.869118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T03:34:01.987016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T03:34:36.681 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T03:34:36.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.808+0000 7f382bb97700 1 Processor -- start 2023-08-25T03:34:36.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.808+0000 7f382bb97700 1 -- start start 2023-08-25T03:34:36.812 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.809+0000 7f382bb97700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f38241607f0 0x7f3824070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:36.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.809+0000 7f382bb97700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f382406a1e0 0x7f382406a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:36.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.809+0000 7f382bb97700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3824160bd0 con 0x7f38240af180 2023-08-25T03:34:36.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.809+0000 7f382bb97700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f382406abd0 con 0x7f382406a1e0 2023-08-25T03:34:36.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.809+0000 7f382bb97700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38240713a0 con 0x7f38241607f0 2023-08-25T03:34:36.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.809+0000 7f382a134700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f382406a1e0 0x7f382406a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:36.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.809+0000 7f382a134700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f382406a1e0 0x7f382406a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60252/0 (socket says 172.21.15.2:60252) 2023-08-25T03:34:36.813 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.809+0000 7f382a134700 1 -- 172.21.15.2:0/1211334729 learned_addr learned my addr 172.21.15.2:0/1211334729 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:36.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.809+0000 7f3829933700 1 --2- 172.21.15.2:0/1211334729 >> v2:172.21.15.31:3300/0 conn(0x7f38241607f0 0x7f3824070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:36.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.809+0000 7f382a134700 1 -- 172.21.15.2:0/1211334729 >> v2:172.21.15.31:3300/0 conn(0x7f38241607f0 msgr2=0x7f3824070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:36.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.809+0000 7f382a134700 1 --2- 172.21.15.2:0/1211334729 >> v2:172.21.15.31:3300/0 conn(0x7f38241607f0 0x7f3824070e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:36.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.809+0000 7f382a134700 1 -- 172.21.15.2:0/1211334729 >> v1:172.21.15.31:6789/0 conn(0x7f38240af180 legacy=0x7f38240ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:36.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.809+0000 7f382a134700 1 -- 172.21.15.2:0/1211334729 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38240cdd00 con 0x7f382406a1e0 2023-08-25T03:34:36.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.809+0000 7f382a134700 1 --2- 172.21.15.2:0/1211334729 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f382406a1e0 0x7f382406a690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f382406df30 tx=0x7f381c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29664a36cf8c0504 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:36.814 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.809+0000 7f3828931700 1 -- 172.21.15.2:0/1211334729 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c00e070 con 0x7f382406a1e0 2023-08-25T03:34:36.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.809+0000 7f3828931700 1 -- 172.21.15.2:0/1211334729 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f381c003a70 con 0x7f382406a1e0 2023-08-25T03:34:36.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.810+0000 7f3828931700 1 -- 172.21.15.2:0/1211334729 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c003e20 con 0x7f382406a1e0 2023-08-25T03:34:36.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.810+0000 7f382bb97700 1 -- 172.21.15.2:0/1211334729 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f382406a1e0 msgr2=0x7f382406a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:36.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.810+0000 7f382bb97700 1 --2- 172.21.15.2:0/1211334729 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f382406a1e0 0x7f382406a690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f382406df30 tx=0x7f381c00b330 comp rx=0 tx=0).stop 2023-08-25T03:34:36.815 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.810+0000 7f382bb97700 1 -- 172.21.15.2:0/1211334729 shutdown_connections 2023-08-25T03:34:36.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.810+0000 7f382bb97700 1 --2- 172.21.15.2:0/1211334729 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f382406a1e0 0x7f382406a690 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:36.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.810+0000 7f382bb97700 1 --2- 172.21.15.2:0/1211334729 >> v2:172.21.15.31:3300/0 conn(0x7f38241607f0 0x7f3824070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:36.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.810+0000 7f382bb97700 1 -- 172.21.15.2:0/1211334729 >> 172.21.15.2:0/1211334729 conn(0x7f38240cfd90 msgr2=0x7f38240d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:36.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.810+0000 7f382bb97700 1 -- 172.21.15.2:0/1211334729 shutdown_connections 2023-08-25T03:34:36.816 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.810+0000 7f382bb97700 1 -- 172.21.15.2:0/1211334729 wait complete. 2023-08-25T03:34:36.819 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.811+0000 7f382bb97700 1 Processor -- start 2023-08-25T03:34:36.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.811+0000 7f382bb97700 1 -- start start 2023-08-25T03:34:36.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.812+0000 7f382bb97700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f382406a1e0 0x7f382415c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:36.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.812+0000 7f382bb97700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38240af180 0x7f38240c5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:36.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.812+0000 7f382bb97700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38241607f0 0x7f38240b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:36.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.812+0000 7f382bb97700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38240c6e60 con 0x7f38240af180 2023-08-25T03:34:36.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.812+0000 7f382bb97700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38240c7420 con 0x7f38241607f0 2023-08-25T03:34:36.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.812+0000 7f382bb97700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38240c6120 con 0x7f382406a1e0 2023-08-25T03:34:36.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.812+0000 7f3829132700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38240af180 0x7f38240c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:36.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.812+0000 7f3829132700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38240af180 0x7f38240c5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60256/0 (socket says 172.21.15.2:60256) 2023-08-25T03:34:36.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.812+0000 7f3829132700 1 -- 172.21.15.2:0/1986985658 learned_addr learned my addr 172.21.15.2:0/1986985658 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:36.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.812+0000 7f382a134700 1 --2- 172.21.15.2:0/1986985658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38241607f0 0x7f38240b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:36.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.812+0000 7f3829933700 1 --2- 172.21.15.2:0/1986985658 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f382406a1e0 0x7f382415c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:36.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.812+0000 7f3829132700 1 -- 172.21.15.2:0/1986985658 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f382406a1e0 msgr2=0x7f382415c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:36.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.813+0000 7f3829132700 1 --2- 172.21.15.2:0/1986985658 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f382406a1e0 0x7f382415c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:36.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.813+0000 7f3829132700 1 -- 172.21.15.2:0/1986985658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38241607f0 msgr2=0x7f38240b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:36.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.813+0000 7f3829132700 1 --2- 172.21.15.2:0/1986985658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38241607f0 0x7f38240b0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:36.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.813+0000 7f3829132700 1 -- 172.21.15.2:0/1986985658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381c009b30 con 0x7f38240af180 2023-08-25T03:34:36.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.813+0000 7f3829132700 1 --2- 172.21.15.2:0/1986985658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38240af180 0x7f38240c5660 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f381400c500 tx=0x7f381400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:36.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.813+0000 7f381a7fc700 1 -- 172.21.15.2:0/1986985658 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3814011790 con 0x7f38240af180 2023-08-25T03:34:36.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.813+0000 7f381a7fc700 1 -- 172.21.15.2:0/1986985658 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38140118f0 con 0x7f38240af180 2023-08-25T03:34:36.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.813+0000 7f382bb97700 1 -- 172.21.15.2:0/1986985658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38240c6400 con 0x7f38240af180 2023-08-25T03:34:36.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.813+0000 7f381a7fc700 1 -- 172.21.15.2:0/1986985658 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3814011ca0 con 0x7f38240af180 2023-08-25T03:34:36.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.813+0000 7f382bb97700 1 -- 172.21.15.2:0/1986985658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38240c4970 con 0x7f38240af180 2023-08-25T03:34:36.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.814+0000 7f382bb97700 1 -- 172.21.15.2:0/1986985658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38240c75b0 con 0x7f38240af180 2023-08-25T03:34:36.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.814+0000 7f381a7fc700 1 -- 172.21.15.2:0/1986985658 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f381401c460 con 0x7f38240af180 2023-08-25T03:34:36.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.815+0000 7f381a7fc700 1 --2- 172.21.15.2:0/1986985658 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f380806a0e0 0x7f380806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:36.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.815+0000 7f381a7fc700 1 -- 172.21.15.2:0/1986985658 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f38140881f0 con 0x7f38240af180 2023-08-25T03:34:36.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.815+0000 7f3829933700 1 --2- 172.21.15.2:0/1986985658 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f380806a0e0 0x7f380806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:36.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.815+0000 7f3829933700 1 --2- 172.21.15.2:0/1986985658 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f380806a0e0 0x7f380806c5a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f382000ab60 tx=0x7f3820008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:36.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:36.819+0000 7f381a7fc700 1 -- 172.21.15.2:0/1986985658 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3814053970 con 0x7f38240af180 2023-08-25T03:34:37.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.019+0000 7f382bb97700 1 -- 172.21.15.2:0/1986985658 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f38240746d0 con 0x7f38240af180 2023-08-25T03:34:37.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.019+0000 7f381a7fc700 1 -- 172.21.15.2:0/1986985658 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3814056f90 con 0x7f38240af180 2023-08-25T03:34:37.021 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-08-25T03:34:37.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.021+0000 7f382bb97700 1 -- 172.21.15.2:0/1986985658 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f380806a0e0 msgr2=0x7f380806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:37.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.021+0000 7f382bb97700 1 --2- 172.21.15.2:0/1986985658 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f380806a0e0 0x7f380806c5a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f382000ab60 tx=0x7f3820008040 comp rx=0 tx=0).stop 2023-08-25T03:34:37.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.021+0000 7f382bb97700 1 -- 172.21.15.2:0/1986985658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38240af180 msgr2=0x7f38240c5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:37.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.021+0000 7f382bb97700 1 --2- 172.21.15.2:0/1986985658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38240af180 0x7f38240c5660 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f381400c500 tx=0x7f381400c810 comp rx=0 tx=0).stop 2023-08-25T03:34:37.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.021+0000 7f382bb97700 1 -- 172.21.15.2:0/1986985658 shutdown_connections 2023-08-25T03:34:37.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.021+0000 7f382bb97700 1 --2- 172.21.15.2:0/1986985658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38241607f0 0x7f38240b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.021+0000 7f382bb97700 1 --2- 172.21.15.2:0/1986985658 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f382406a1e0 0x7f382415c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.021+0000 7f382bb97700 1 --2- 172.21.15.2:0/1986985658 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f380806a0e0 0x7f380806c5a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.021+0000 7f382bb97700 1 --2- 172.21.15.2:0/1986985658 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38240af180 0x7f38240c5660 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.021+0000 7f382bb97700 1 -- 172.21.15.2:0/1986985658 >> 172.21.15.2:0/1986985658 conn(0x7f38240cfd90 msgr2=0x7f38240cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:37.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.021+0000 7f382bb97700 1 -- 172.21.15.2:0/1986985658 shutdown_connections 2023-08-25T03:34:37.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.022+0000 7f382bb97700 1 -- 172.21.15.2:0/1986985658 wait complete. 2023-08-25T03:34:37.035 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T03:34:37.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.162+0000 7f441f7ea700 1 Processor -- start 2023-08-25T03:34:37.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.162+0000 7f441f7ea700 1 -- start start 2023-08-25T03:34:37.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.163+0000 7f441f7ea700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f44181607f0 0x7f4418070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:37.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.163+0000 7f441f7ea700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f44180af180 0x7f44180ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:37.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.163+0000 7f441f7ea700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4418160bd0 con 0x7f441806a1e0 2023-08-25T03:34:37.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.163+0000 7f441f7ea700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44180713a0 con 0x7f44180af180 2023-08-25T03:34:37.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.163+0000 7f441f7ea700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44180718a0 con 0x7f44181607f0 2023-08-25T03:34:37.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.163+0000 7f441cd85700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f44180af180 0x7f44180ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:37.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.163+0000 7f441dd87700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f441806a1e0 0x7f441806ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46930/0 (socket says 172.21.15.2:46930) 2023-08-25T03:34:37.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.163+0000 7f441dd87700 1 -- 172.21.15.2:0/737986400 learned_addr learned my addr 172.21.15.2:0/737986400 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:37.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.163+0000 7f441d586700 1 --2- 172.21.15.2:0/737986400 >> v2:172.21.15.31:3300/0 conn(0x7f44181607f0 0x7f4418070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:37.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.163+0000 7f441cd85700 1 -- 172.21.15.2:0/737986400 >> v2:172.21.15.31:3300/0 conn(0x7f44181607f0 msgr2=0x7f4418070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:37.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.163+0000 7f441cd85700 1 --2- 172.21.15.2:0/737986400 >> v2:172.21.15.31:3300/0 conn(0x7f44181607f0 0x7f4418070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.163+0000 7f441cd85700 1 -- 172.21.15.2:0/737986400 >> v1:172.21.15.2:6789/0 conn(0x7f441806a1e0 legacy=0x7f441806ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:37.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.163+0000 7f441cd85700 1 -- 172.21.15.2:0/737986400 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44180cdd00 con 0x7f44180af180 2023-08-25T03:34:37.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.163+0000 7f440ffff700 1 -- 172.21.15.2:0/737986400 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3223424853 0 0) 0x7f4418160bd0 con 0x7f441806a1e0 2023-08-25T03:34:37.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.163+0000 7f441d586700 1 --2- 172.21.15.2:0/737986400 >> v2:172.21.15.31:3300/0 conn(0x7f44181607f0 0x7f4418070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:37.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.163+0000 7f441cd85700 1 --2- 172.21.15.2:0/737986400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44180af180 0x7f44180ac8e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f441806df30 tx=0x7f4408009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07ac59c56a3116 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:37.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.163+0000 7f440ffff700 1 -- 172.21.15.2:0/737986400 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408011830 con 0x7f44180af180 2023-08-25T03:34:37.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.164+0000 7f440ffff700 1 -- 172.21.15.2:0/737986400 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4408011990 con 0x7f44180af180 2023-08-25T03:34:37.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.164+0000 7f440ffff700 1 -- 172.21.15.2:0/737986400 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408011d20 con 0x7f44180af180 2023-08-25T03:34:37.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.164+0000 7f441f7ea700 1 -- 172.21.15.2:0/737986400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44180af180 msgr2=0x7f44180ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:37.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.164+0000 7f441f7ea700 1 --2- 172.21.15.2:0/737986400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44180af180 0x7f44180ac8e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f441806df30 tx=0x7f4408009b20 comp rx=0 tx=0).stop 2023-08-25T03:34:37.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.164+0000 7f441f7ea700 1 -- 172.21.15.2:0/737986400 shutdown_connections 2023-08-25T03:34:37.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.164+0000 7f441f7ea700 1 --2- 172.21.15.2:0/737986400 >> v2:172.21.15.31:3300/0 conn(0x7f44181607f0 0x7f4418070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.164+0000 7f441f7ea700 1 --2- 172.21.15.2:0/737986400 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f44180af180 0x7f44180ac8e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.164+0000 7f441f7ea700 1 -- 172.21.15.2:0/737986400 >> 172.21.15.2:0/737986400 conn(0x7f44180cfd90 msgr2=0x7f44180d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:37.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.164+0000 7f441f7ea700 1 -- 172.21.15.2:0/737986400 shutdown_connections 2023-08-25T03:34:37.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.164+0000 7f441f7ea700 1 -- 172.21.15.2:0/737986400 wait complete. 2023-08-25T03:34:37.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.165+0000 7f441f7ea700 1 Processor -- start 2023-08-25T03:34:37.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.165+0000 7f441f7ea700 1 -- start start 2023-08-25T03:34:37.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.166+0000 7f441f7ea700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f441806a1e0 0x7f44180c9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:37.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.166+0000 7f441f7ea700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44180af180 0x7f441806eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:37.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.166+0000 7f441f7ea700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44181607f0 0x7f441806f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:37.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.166+0000 7f441f7ea700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44180c9ec0 con 0x7f441806a1e0 2023-08-25T03:34:37.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.166+0000 7f441f7ea700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4418157e10 con 0x7f44180af180 2023-08-25T03:34:37.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.166+0000 7f441f7ea700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44180c5120 con 0x7f44181607f0 2023-08-25T03:34:37.176 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.166+0000 7f441d586700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f441806a1e0 0x7f44180c9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:37.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.166+0000 7f441d586700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f441806a1e0 0x7f44180c9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60268/0 (socket says 172.21.15.2:60268) 2023-08-25T03:34:37.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.166+0000 7f441d586700 1 -- 172.21.15.2:0/3683605478 learned_addr learned my addr 172.21.15.2:0/3683605478 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:37.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.166+0000 7f441cd85700 1 --2- 172.21.15.2:0/3683605478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44180af180 0x7f441806eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:37.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.166+0000 7f441dd87700 1 --2- 172.21.15.2:0/3683605478 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44181607f0 0x7f441806f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:37.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.166+0000 7f441d586700 1 -- 172.21.15.2:0/3683605478 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44181607f0 msgr2=0x7f441806f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:37.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.166+0000 7f441d586700 1 --2- 172.21.15.2:0/3683605478 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44181607f0 0x7f441806f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.166+0000 7f441d586700 1 -- 172.21.15.2:0/3683605478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44180af180 msgr2=0x7f441806eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:37.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.166+0000 7f441d586700 1 --2- 172.21.15.2:0/3683605478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44180af180 0x7f441806eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.166+0000 7f441d586700 1 -- 172.21.15.2:0/3683605478 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44080094a0 con 0x7f441806a1e0 2023-08-25T03:34:37.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.167+0000 7f441dd87700 1 --2- 172.21.15.2:0/3683605478 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44181607f0 0x7f441806f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:37.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.167+0000 7f441d586700 1 --2- 172.21.15.2:0/3683605478 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f441806a1e0 0x7f44180c9980 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f441400e810 tx=0x7f441400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:37.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.167+0000 7f440dffb700 1 -- 172.21.15.2:0/3683605478 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441400c020 con 0x7f441806a1e0 2023-08-25T03:34:37.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.167+0000 7f440dffb700 1 -- 172.21.15.2:0/3683605478 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4414004250 con 0x7f441806a1e0 2023-08-25T03:34:37.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.167+0000 7f441f7ea700 1 -- 172.21.15.2:0/3683605478 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44180c5400 con 0x7f441806a1e0 2023-08-25T03:34:37.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.167+0000 7f440dffb700 1 -- 172.21.15.2:0/3683605478 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44140036a0 con 0x7f441806a1e0 2023-08-25T03:34:37.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.167+0000 7f441f7ea700 1 -- 172.21.15.2:0/3683605478 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44180c59c0 con 0x7f441806a1e0 2023-08-25T03:34:37.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.168+0000 7f441f7ea700 1 -- 172.21.15.2:0/3683605478 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44180746d0 con 0x7f441806a1e0 2023-08-25T03:34:37.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.168+0000 7f440dffb700 1 -- 172.21.15.2:0/3683605478 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4414003800 con 0x7f441806a1e0 2023-08-25T03:34:37.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.169+0000 7f440dffb700 1 --2- 172.21.15.2:0/3683605478 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f43fc06a110 0x7f43fc06c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:37.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.169+0000 7f440dffb700 1 -- 172.21.15.2:0/3683605478 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4414087f80 con 0x7f441806a1e0 2023-08-25T03:34:37.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.169+0000 7f441cd85700 1 --2- 172.21.15.2:0/3683605478 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f43fc06a110 0x7f43fc06c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:37.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.169+0000 7f441cd85700 1 --2- 172.21.15.2:0/3683605478 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f43fc06a110 0x7f43fc06c5d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f44080090b0 tx=0x7f4408008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:37.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.173+0000 7f440dffb700 1 -- 172.21.15.2:0/3683605478 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f44140535e0 con 0x7f441806a1e0 2023-08-25T03:34:37.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.370+0000 7f441f7ea700 1 -- 172.21.15.2:0/3683605478 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f44180ae800 con 0x7f441806a1e0 2023-08-25T03:34:37.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.370+0000 7f440dffb700 1 -- 172.21.15.2:0/3683605478 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4414056c00 con 0x7f441806a1e0 2023-08-25T03:34:37.373 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 8 2023-08-25T03:34:37.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.372+0000 7f441f7ea700 1 -- 172.21.15.2:0/3683605478 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f43fc06a110 msgr2=0x7f43fc06c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:37.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.372+0000 7f441f7ea700 1 --2- 172.21.15.2:0/3683605478 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f43fc06a110 0x7f43fc06c5d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f44080090b0 tx=0x7f4408008030 comp rx=0 tx=0).stop 2023-08-25T03:34:37.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.372+0000 7f441f7ea700 1 -- 172.21.15.2:0/3683605478 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f441806a1e0 msgr2=0x7f44180c9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:37.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.372+0000 7f441f7ea700 1 --2- 172.21.15.2:0/3683605478 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f441806a1e0 0x7f44180c9980 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f441400e810 tx=0x7f441400eb20 comp rx=0 tx=0).stop 2023-08-25T03:34:37.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.372+0000 7f441f7ea700 1 -- 172.21.15.2:0/3683605478 shutdown_connections 2023-08-25T03:34:37.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.372+0000 7f441f7ea700 1 --2- 172.21.15.2:0/3683605478 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f43fc06a110 0x7f43fc06c5d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.372+0000 7f441f7ea700 1 --2- 172.21.15.2:0/3683605478 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f441806a1e0 0x7f44180c9980 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.372+0000 7f441f7ea700 1 --2- 172.21.15.2:0/3683605478 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44181607f0 0x7f441806f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.376 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.372+0000 7f441f7ea700 1 --2- 172.21.15.2:0/3683605478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44180af180 0x7f441806eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.376 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.372+0000 7f441f7ea700 1 -- 172.21.15.2:0/3683605478 >> 172.21.15.2:0/3683605478 conn(0x7f44180cfd90 msgr2=0x7f44180ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:37.376 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.372+0000 7f441f7ea700 1 -- 172.21.15.2:0/3683605478 shutdown_connections 2023-08-25T03:34:37.376 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.373+0000 7f441f7ea700 1 -- 172.21.15.2:0/3683605478 wait complete. 2023-08-25T03:34:37.390 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T03:34:37.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.517+0000 7fe998388700 1 Processor -- start 2023-08-25T03:34:37.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.517+0000 7fe998388700 1 -- start start 2023-08-25T03:34:37.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.518+0000 7fe998388700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe9901607b0 0x7fe9900cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:37.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.518+0000 7fe998388700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe9900d2b20 0x7fe9900ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:37.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.518+0000 7fe998388700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe990160b90 con 0x7fe9900d1510 2023-08-25T03:34:37.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.518+0000 7fe998388700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9900d1a80 con 0x7fe9901607b0 2023-08-25T03:34:37.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.518+0000 7fe998388700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9900d1d60 con 0x7fe9900d2b20 2023-08-25T03:34:37.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.518+0000 7fe995923700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fe9900d1510 0x7fe99006c260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46942/0 (socket says 172.21.15.2:46942) 2023-08-25T03:34:37.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.518+0000 7fe996124700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe9901607b0 0x7fe9900cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:37.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.518+0000 7fe995923700 1 -- 172.21.15.2:0/1624901120 learned_addr learned my addr 172.21.15.2:0/1624901120 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:37.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.518+0000 7fe996925700 1 --2- 172.21.15.2:0/1624901120 >> v2:172.21.15.31:3300/0 conn(0x7fe9900d2b20 0x7fe9900ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:37.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.518+0000 7fe996124700 1 -- 172.21.15.2:0/1624901120 >> v2:172.21.15.31:3300/0 conn(0x7fe9900d2b20 msgr2=0x7fe9900ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:37.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.518+0000 7fe996124700 1 --2- 172.21.15.2:0/1624901120 >> v2:172.21.15.31:3300/0 conn(0x7fe9900d2b20 0x7fe9900ce300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.518+0000 7fe996124700 1 -- 172.21.15.2:0/1624901120 >> v1:172.21.15.2:6789/0 conn(0x7fe9900d1510 legacy=0x7fe99006c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:37.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.518+0000 7fe996124700 1 -- 172.21.15.2:0/1624901120 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9900d2040 con 0x7fe9901607b0 2023-08-25T03:34:37.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.518+0000 7fe996925700 1 --2- 172.21.15.2:0/1624901120 >> v2:172.21.15.31:3300/0 conn(0x7fe9900d2b20 0x7fe9900ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:37.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.518+0000 7fe996124700 1 --2- 172.21.15.2:0/1624901120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9901607b0 0x7fe9900cddc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe9900cd6d0 tx=0x7fe98c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc845645d66a2482 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:37.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.519+0000 7fe995122700 1 -- 172.21.15.2:0/1624901120 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c011830 con 0x7fe9901607b0 2023-08-25T03:34:37.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.519+0000 7fe995122700 1 -- 172.21.15.2:0/1624901120 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98c011990 con 0x7fe9901607b0 2023-08-25T03:34:37.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.519+0000 7fe995122700 1 -- 172.21.15.2:0/1624901120 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98c011d20 con 0x7fe9901607b0 2023-08-25T03:34:37.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.519+0000 7fe998388700 1 -- 172.21.15.2:0/1624901120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9901607b0 msgr2=0x7fe9900cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:37.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.519+0000 7fe998388700 1 --2- 172.21.15.2:0/1624901120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9901607b0 0x7fe9900cddc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe9900cd6d0 tx=0x7fe98c009b20 comp rx=0 tx=0).stop 2023-08-25T03:34:37.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.519+0000 7fe998388700 1 -- 172.21.15.2:0/1624901120 shutdown_connections 2023-08-25T03:34:37.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.519+0000 7fe998388700 1 --2- 172.21.15.2:0/1624901120 >> v2:172.21.15.31:3300/0 conn(0x7fe9900d2b20 0x7fe9900ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.519+0000 7fe998388700 1 --2- 172.21.15.2:0/1624901120 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9901607b0 0x7fe9900cddc0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.519+0000 7fe998388700 1 -- 172.21.15.2:0/1624901120 >> 172.21.15.2:0/1624901120 conn(0x7fe9900cfb70 msgr2=0x7fe9900cff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:37.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.519+0000 7fe998388700 1 -- 172.21.15.2:0/1624901120 shutdown_connections 2023-08-25T03:34:37.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.519+0000 7fe998388700 1 -- 172.21.15.2:0/1624901120 wait complete. 2023-08-25T03:34:37.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.520+0000 7fe998388700 1 Processor -- start 2023-08-25T03:34:37.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.520+0000 7fe998388700 1 -- start start 2023-08-25T03:34:37.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.521+0000 7fe998388700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9900d1510 0x7fe99014e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:37.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.521+0000 7fe998388700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9900d2b20 0x7fe9900cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:37.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.521+0000 7fe998388700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9901607b0 0x7fe990154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:37.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.521+0000 7fe998388700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99014ed10 con 0x7fe9900d2b20 2023-08-25T03:34:37.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.521+0000 7fe998388700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9900cb540 con 0x7fe9901607b0 2023-08-25T03:34:37.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.521+0000 7fe998388700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9900cb820 con 0x7fe9900d1510 2023-08-25T03:34:37.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.521+0000 7fe995923700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9900d2b20 0x7fe9900cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:37.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.521+0000 7fe995923700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9900d2b20 0x7fe9900cabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60284/0 (socket says 172.21.15.2:60284) 2023-08-25T03:34:37.539 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.521+0000 7fe995923700 1 -- 172.21.15.2:0/1232089206 learned_addr learned my addr 172.21.15.2:0/1232089206 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:37.539 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.521+0000 7fe996925700 1 --2- 172.21.15.2:0/1232089206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9901607b0 0x7fe990154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:37.539 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.521+0000 7fe996124700 1 --2- 172.21.15.2:0/1232089206 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9900d1510 0x7fe99014e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:37.539 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.522+0000 7fe995923700 1 -- 172.21.15.2:0/1232089206 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9900d1510 msgr2=0x7fe99014e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:37.539 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.522+0000 7fe995923700 1 --2- 172.21.15.2:0/1232089206 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9900d1510 0x7fe99014e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.522+0000 7fe995923700 1 -- 172.21.15.2:0/1232089206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9901607b0 msgr2=0x7fe990154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:37.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.522+0000 7fe995923700 1 --2- 172.21.15.2:0/1232089206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9901607b0 0x7fe990154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.522+0000 7fe995923700 1 -- 172.21.15.2:0/1232089206 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c0094a0 con 0x7fe9900d2b20 2023-08-25T03:34:37.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.522+0000 7fe996925700 1 --2- 172.21.15.2:0/1232089206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9901607b0 0x7fe990154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:37.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.522+0000 7fe995923700 1 --2- 172.21.15.2:0/1232089206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9900d2b20 0x7fe9900cabd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe98000c520 tx=0x7fe98000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:37.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.522+0000 7fe986ffd700 1 -- 172.21.15.2:0/1232089206 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980005d70 con 0x7fe9900d2b20 2023-08-25T03:34:37.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.522+0000 7fe986ffd700 1 -- 172.21.15.2:0/1232089206 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9800070d0 con 0x7fe9900d2b20 2023-08-25T03:34:37.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.522+0000 7fe986ffd700 1 -- 172.21.15.2:0/1232089206 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe980007440 con 0x7fe9900d2b20 2023-08-25T03:34:37.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.522+0000 7fe998388700 1 -- 172.21.15.2:0/1232089206 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9900cbb00 con 0x7fe9900d2b20 2023-08-25T03:34:37.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.522+0000 7fe998388700 1 -- 172.21.15.2:0/1232089206 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9900cc0c0 con 0x7fe9900d2b20 2023-08-25T03:34:37.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.523+0000 7fe998388700 1 -- 172.21.15.2:0/1232089206 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9900746d0 con 0x7fe9900d2b20 2023-08-25T03:34:37.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.523+0000 7fe986ffd700 1 -- 172.21.15.2:0/1232089206 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe9800075a0 con 0x7fe9900d2b20 2023-08-25T03:34:37.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.524+0000 7fe986ffd700 1 --2- 172.21.15.2:0/1232089206 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe97406a0e0 0x7fe97406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:37.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.524+0000 7fe986ffd700 1 -- 172.21.15.2:0/1232089206 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe980087920 con 0x7fe9900d2b20 2023-08-25T03:34:37.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.524+0000 7fe996124700 1 --2- 172.21.15.2:0/1232089206 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe97406a0e0 0x7fe97406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:37.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.524+0000 7fe996124700 1 --2- 172.21.15.2:0/1232089206 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe97406a0e0 0x7fe97406c5a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0090b0 tx=0x7fe98c01c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:37.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.529+0000 7fe986ffd700 1 -- 172.21.15.2:0/1232089206 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe9800530c0 con 0x7fe9900d2b20 2023-08-25T03:34:37.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.729+0000 7fe998388700 1 -- 172.21.15.2:0/1232089206 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe990157a20 con 0x7fe9900d2b20 2023-08-25T03:34:37.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.730+0000 7fe986ffd700 1 -- 172.21.15.2:0/1232089206 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe9800566e0 con 0x7fe9900d2b20 2023-08-25T03:34:37.732 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-08-25T03:34:37.734 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.732+0000 7fe998388700 1 -- 172.21.15.2:0/1232089206 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe97406a0e0 msgr2=0x7fe97406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:37.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.732+0000 7fe998388700 1 --2- 172.21.15.2:0/1232089206 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe97406a0e0 0x7fe97406c5a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe98c0090b0 tx=0x7fe98c01c040 comp rx=0 tx=0).stop 2023-08-25T03:34:37.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.732+0000 7fe998388700 1 -- 172.21.15.2:0/1232089206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9900d2b20 msgr2=0x7fe9900cabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:37.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.732+0000 7fe998388700 1 --2- 172.21.15.2:0/1232089206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9900d2b20 0x7fe9900cabd0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe98000c520 tx=0x7fe98000c8e0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.732+0000 7fe998388700 1 -- 172.21.15.2:0/1232089206 shutdown_connections 2023-08-25T03:34:37.735 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.732+0000 7fe998388700 1 --2- 172.21.15.2:0/1232089206 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9901607b0 0x7fe990154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.732+0000 7fe998388700 1 --2- 172.21.15.2:0/1232089206 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9900d1510 0x7fe99014e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.732+0000 7fe998388700 1 --2- 172.21.15.2:0/1232089206 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe97406a0e0 0x7fe97406c5a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.732+0000 7fe998388700 1 --2- 172.21.15.2:0/1232089206 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe9900d2b20 0x7fe9900cabd0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.732+0000 7fe998388700 1 -- 172.21.15.2:0/1232089206 >> 172.21.15.2:0/1232089206 conn(0x7fe9900cfb70 msgr2=0x7fe99015f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:37.736 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.732+0000 7fe998388700 1 -- 172.21.15.2:0/1232089206 shutdown_connections 2023-08-25T03:34:37.737 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.732+0000 7fe998388700 1 -- 172.21.15.2:0/1232089206 wait complete. 2023-08-25T03:34:37.746 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T03:34:37.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.873+0000 7fec84592700 1 Processor -- start 2023-08-25T03:34:37.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.873+0000 7fec84592700 1 -- start start 2023-08-25T03:34:37.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.874+0000 7fec84592700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fec7c159f10 0x7fec7c15dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:37.884 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.874+0000 7fec84592700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fec7c0cedf0 0x7fec7c06ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:37.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.874+0000 7fec84592700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fec7c06f4c0 0x7fec7c0cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:37.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.874+0000 7fec84592700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec7c06e850 con 0x7fec7c159f10 2023-08-25T03:34:37.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.874+0000 7fec84592700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec7c06eaf0 con 0x7fec7c0cedf0 2023-08-25T03:34:37.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.874+0000 7fec84592700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c15e020 con 0x7fec7c06f4c0 2023-08-25T03:34:37.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.874+0000 7fec8232e700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fec7c159f10 0x7fec7c15dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:37.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.874+0000 7fec8232e700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fec7c159f10 0x7fec7c15dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60290/0 (socket says 172.21.15.2:60290) 2023-08-25T03:34:37.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.874+0000 7fec8232e700 1 -- 172.21.15.2:0/1431884916 learned_addr learned my addr 172.21.15.2:0/1431884916 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:37.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.874+0000 7fec81b2d700 1 --2- 172.21.15.2:0/1431884916 >> v2:172.21.15.31:3300/0 conn(0x7fec7c0cedf0 0x7fec7c06ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:37.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.874+0000 7fec82b2f700 1 --2- 172.21.15.2:0/1431884916 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fec7c06f4c0 0x7fec7c0cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:37.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.874+0000 7fec8232e700 1 -- 172.21.15.2:0/1431884916 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fec7c06f4c0 msgr2=0x7fec7c0cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:37.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.874+0000 7fec8232e700 1 --2- 172.21.15.2:0/1431884916 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fec7c06f4c0 0x7fec7c0cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.874+0000 7fec8232e700 1 -- 172.21.15.2:0/1431884916 >> v2:172.21.15.31:3300/0 conn(0x7fec7c0cedf0 msgr2=0x7fec7c06ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:37.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.874+0000 7fec8232e700 1 --2- 172.21.15.2:0/1431884916 >> v2:172.21.15.31:3300/0 conn(0x7fec7c0cedf0 0x7fec7c06ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.874+0000 7fec8232e700 1 -- 172.21.15.2:0/1431884916 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7c15e2c0 con 0x7fec7c159f10 2023-08-25T03:34:37.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.874+0000 7fec81b2d700 1 --2- 172.21.15.2:0/1431884916 >> v2:172.21.15.31:3300/0 conn(0x7fec7c0cedf0 0x7fec7c06ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:37.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.874+0000 7fec8232e700 1 --2- 172.21.15.2:0/1431884916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec7c159f10 0x7fec7c15dae0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fec7c072300 tx=0x7fec7800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ff8153b30be068 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:37.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.875+0000 7fec8132c700 1 -- 172.21.15.2:0/1431884916 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7800dd30 con 0x7fec7c159f10 2023-08-25T03:34:37.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.875+0000 7fec8132c700 1 -- 172.21.15.2:0/1431884916 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7800de90 con 0x7fec7c159f10 2023-08-25T03:34:37.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.875+0000 7fec8132c700 1 -- 172.21.15.2:0/1431884916 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec78010690 con 0x7fec7c159f10 2023-08-25T03:34:37.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.875+0000 7fec84592700 1 -- 172.21.15.2:0/1431884916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec7c159f10 msgr2=0x7fec7c15dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:37.889 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.875+0000 7fec84592700 1 --2- 172.21.15.2:0/1431884916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec7c159f10 0x7fec7c15dae0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fec7c072300 tx=0x7fec7800be70 comp rx=0 tx=0).stop 2023-08-25T03:34:37.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.875+0000 7fec84592700 1 -- 172.21.15.2:0/1431884916 shutdown_connections 2023-08-25T03:34:37.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.875+0000 7fec84592700 1 --2- 172.21.15.2:0/1431884916 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fec7c06f4c0 0x7fec7c0cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.875+0000 7fec84592700 1 --2- 172.21.15.2:0/1431884916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec7c159f10 0x7fec7c15dae0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.875+0000 7fec84592700 1 --2- 172.21.15.2:0/1431884916 >> v2:172.21.15.31:3300/0 conn(0x7fec7c0cedf0 0x7fec7c06ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.875+0000 7fec84592700 1 -- 172.21.15.2:0/1431884916 >> 172.21.15.2:0/1431884916 conn(0x7fec7c0cfd90 msgr2=0x7fec7c0d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:37.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.875+0000 7fec84592700 1 -- 172.21.15.2:0/1431884916 shutdown_connections 2023-08-25T03:34:37.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.875+0000 7fec84592700 1 -- 172.21.15.2:0/1431884916 wait complete. 2023-08-25T03:34:37.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.877+0000 7fec84592700 1 Processor -- start 2023-08-25T03:34:37.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.877+0000 7fec84592700 1 -- start start 2023-08-25T03:34:37.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.877+0000 7fec84592700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec7c06f4c0 0x7fec7c14d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:37.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.877+0000 7fec84592700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec7c0cedf0 0x7fec7c0c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:37.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.877+0000 7fec84592700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fec7c159f10 0x7fec7c06a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:37.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.877+0000 7fec84592700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c15a630 con 0x7fec7c0cedf0 2023-08-25T03:34:37.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.877+0000 7fec84592700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c0c5d80 con 0x7fec7c06f4c0 2023-08-25T03:34:37.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.877+0000 7fec84592700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c0c6060 con 0x7fec7c159f10 2023-08-25T03:34:37.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.877+0000 7fec81b2d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec7c0cedf0 0x7fec7c0c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:37.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.877+0000 7fec81b2d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec7c0cedf0 0x7fec7c0c52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60298/0 (socket says 172.21.15.2:60298) 2023-08-25T03:34:37.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.877+0000 7fec81b2d700 1 -- 172.21.15.2:0/1550297616 learned_addr learned my addr 172.21.15.2:0/1550297616 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:37.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.878+0000 7fec8232e700 1 --2- 172.21.15.2:0/1550297616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec7c06f4c0 0x7fec7c14d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:37.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.878+0000 7fec81b2d700 1 -- 172.21.15.2:0/1550297616 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fec7c159f10 msgr2=0x7fec7c06a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:37.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.878+0000 7fec82b2f700 1 --2- 172.21.15.2:0/1550297616 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fec7c159f10 0x7fec7c06a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:37.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.878+0000 7fec81b2d700 1 --2- 172.21.15.2:0/1550297616 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fec7c159f10 0x7fec7c06a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.878+0000 7fec81b2d700 1 -- 172.21.15.2:0/1550297616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec7c06f4c0 msgr2=0x7fec7c14d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:37.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.878+0000 7fec81b2d700 1 --2- 172.21.15.2:0/1550297616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec7c06f4c0 0x7fec7c14d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:37.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.878+0000 7fec81b2d700 1 -- 172.21.15.2:0/1550297616 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7800b7f0 con 0x7fec7c0cedf0 2023-08-25T03:34:37.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.878+0000 7fec8232e700 1 --2- 172.21.15.2:0/1550297616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec7c06f4c0 0x7fec7c14d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:37.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.878+0000 7fec81b2d700 1 --2- 172.21.15.2:0/1550297616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec7c0cedf0 0x7fec7c0c52c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00e810 tx=0x7fec6c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:37.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.878+0000 7fec72ffd700 1 -- 172.21.15.2:0/1550297616 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c004230 con 0x7fec7c0cedf0 2023-08-25T03:34:37.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.878+0000 7fec72ffd700 1 -- 172.21.15.2:0/1550297616 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec6c004390 con 0x7fec7c0cedf0 2023-08-25T03:34:37.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.878+0000 7fec84592700 1 -- 172.21.15.2:0/1550297616 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec7c0c6340 con 0x7fec7c0cedf0 2023-08-25T03:34:37.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.878+0000 7fec72ffd700 1 -- 172.21.15.2:0/1550297616 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c0036a0 con 0x7fec7c0cedf0 2023-08-25T03:34:37.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.878+0000 7fec84592700 1 -- 172.21.15.2:0/1550297616 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7c14d860 con 0x7fec7c0cedf0 2023-08-25T03:34:37.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.879+0000 7fec84592700 1 -- 172.21.15.2:0/1550297616 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec7c0746d0 con 0x7fec7c0cedf0 2023-08-25T03:34:37.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.880+0000 7fec72ffd700 1 -- 172.21.15.2:0/1550297616 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fec6c003800 con 0x7fec7c0cedf0 2023-08-25T03:34:37.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.880+0000 7fec72ffd700 1 --2- 172.21.15.2:0/1550297616 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fec6006a0e0 0x7fec6006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:37.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.880+0000 7fec72ffd700 1 -- 172.21.15.2:0/1550297616 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fec6c0884e0 con 0x7fec7c0cedf0 2023-08-25T03:34:37.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.880+0000 7fec8232e700 1 --2- 172.21.15.2:0/1550297616 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fec6006a0e0 0x7fec6006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:37.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.881+0000 7fec8232e700 1 --2- 172.21.15.2:0/1550297616 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fec6006a0e0 0x7fec6006c5a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fec7800beb0 tx=0x7fec78012000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:37.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:37.884+0000 7fec72ffd700 1 -- 172.21.15.2:0/1550297616 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fec6c053bb0 con 0x7fec7c0cedf0 2023-08-25T03:34:38.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.084+0000 7fec84592700 1 -- 172.21.15.2:0/1550297616 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fec7c06dbe0 con 0x7fec7c0cedf0 2023-08-25T03:34:38.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.085+0000 7fec72ffd700 1 -- 172.21.15.2:0/1550297616 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fec6c0094c0 con 0x7fec7c0cedf0 2023-08-25T03:34:38.087 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-08-25T03:34:38.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.087+0000 7fec84592700 1 -- 172.21.15.2:0/1550297616 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fec6006a0e0 msgr2=0x7fec6006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:38.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.087+0000 7fec84592700 1 --2- 172.21.15.2:0/1550297616 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fec6006a0e0 0x7fec6006c5a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fec7800beb0 tx=0x7fec78012000 comp rx=0 tx=0).stop 2023-08-25T03:34:38.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.087+0000 7fec84592700 1 -- 172.21.15.2:0/1550297616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec7c0cedf0 msgr2=0x7fec7c0c52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:38.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.087+0000 7fec84592700 1 --2- 172.21.15.2:0/1550297616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec7c0cedf0 0x7fec7c0c52c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00e810 tx=0x7fec6c00ebd0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.087+0000 7fec84592700 1 -- 172.21.15.2:0/1550297616 shutdown_connections 2023-08-25T03:34:38.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.087+0000 7fec84592700 1 --2- 172.21.15.2:0/1550297616 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fec7c159f10 0x7fec7c06a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.087+0000 7fec84592700 1 --2- 172.21.15.2:0/1550297616 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec7c06f4c0 0x7fec7c14d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.087+0000 7fec84592700 1 --2- 172.21.15.2:0/1550297616 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fec6006a0e0 0x7fec6006c5a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.087+0000 7fec84592700 1 --2- 172.21.15.2:0/1550297616 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fec7c0cedf0 0x7fec7c0c52c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.087+0000 7fec84592700 1 -- 172.21.15.2:0/1550297616 >> 172.21.15.2:0/1550297616 conn(0x7fec7c0cfd90 msgr2=0x7fec7c070e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:38.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.087+0000 7fec84592700 1 -- 172.21.15.2:0/1550297616 shutdown_connections 2023-08-25T03:34:38.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.087+0000 7fec84592700 1 -- 172.21.15.2:0/1550297616 wait complete. 2023-08-25T03:34:38.101 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:34:38.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.224+0000 7fad50455700 1 Processor -- start 2023-08-25T03:34:38.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.224+0000 7fad50455700 1 -- start start 2023-08-25T03:34:38.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.225+0000 7fad50455700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fad4815a030 0x7fad480cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:38.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.225+0000 7fad50455700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fad480cc1f0 0x7fad480ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:38.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.225+0000 7fad50455700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad48071dc0 con 0x7fad4815a410 2023-08-25T03:34:38.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.225+0000 7fad50455700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad480720a0 con 0x7fad4815a030 2023-08-25T03:34:38.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.225+0000 7fad50455700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad480cb4b0 con 0x7fad480cc1f0 2023-08-25T03:34:38.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.226+0000 7fad4d9f0700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fad480cc1f0 0x7fad480ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:38.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.226+0000 7fad4e1f1700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fad4815a030 0x7fad480cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:38.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.226+0000 7fad4e9f2700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fad4815a410 0x7fad480ccd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.2:54998/0 (socket says 172.21.15.2:54998) 2023-08-25T03:34:38.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.226+0000 7fad4d9f0700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fad480cc1f0 0x7fad480ce170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.2:60548/0 (socket says 172.21.15.2:60548) 2023-08-25T03:34:38.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.226+0000 7fad4e9f2700 1 -- 172.21.15.2:0/3435386934 learned_addr learned my addr 172.21.15.2:0/3435386934 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:38.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.226+0000 7fad4e1f1700 1 -- 172.21.15.2:0/3435386934 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fad480cc1f0 msgr2=0x7fad480ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:38.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.226+0000 7fad4e1f1700 1 --2- 172.21.15.2:0/3435386934 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fad480cc1f0 0x7fad480ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.226+0000 7fad4e1f1700 1 -- 172.21.15.2:0/3435386934 >> v1:172.21.15.31:6789/0 conn(0x7fad4815a410 legacy=0x7fad480ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:38.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.226+0000 7fad4e1f1700 1 -- 172.21.15.2:0/3435386934 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad480cb790 con 0x7fad4815a030 2023-08-25T03:34:38.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.226+0000 7fad4e1f1700 1 --2- 172.21.15.2:0/3435386934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad4815a030 0x7fad480cbcb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fad3c009840 tx=0x7fad3c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78a5f523cc6c0c68 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:38.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.226+0000 7fad4d1ef700 1 -- 172.21.15.2:0/3435386934 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c011830 con 0x7fad4815a030 2023-08-25T03:34:38.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.226+0000 7fad4d1ef700 1 -- 172.21.15.2:0/3435386934 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad3c011990 con 0x7fad4815a030 2023-08-25T03:34:38.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.226+0000 7fad4d1ef700 1 -- 172.21.15.2:0/3435386934 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3c011d20 con 0x7fad4815a030 2023-08-25T03:34:38.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.227+0000 7fad50455700 1 -- 172.21.15.2:0/3435386934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad4815a030 msgr2=0x7fad480cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:38.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.227+0000 7fad50455700 1 --2- 172.21.15.2:0/3435386934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad4815a030 0x7fad480cbcb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fad3c009840 tx=0x7fad3c009b50 comp rx=0 tx=0).stop 2023-08-25T03:34:38.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.227+0000 7fad50455700 1 -- 172.21.15.2:0/3435386934 shutdown_connections 2023-08-25T03:34:38.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.227+0000 7fad50455700 1 --2- 172.21.15.2:0/3435386934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad4815a030 0x7fad480cbcb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.227+0000 7fad50455700 1 --2- 172.21.15.2:0/3435386934 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fad480cc1f0 0x7fad480ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.227+0000 7fad50455700 1 -- 172.21.15.2:0/3435386934 >> 172.21.15.2:0/3435386934 conn(0x7fad480b1cf0 msgr2=0x7fad480b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:38.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.227+0000 7fad50455700 1 -- 172.21.15.2:0/3435386934 shutdown_connections 2023-08-25T03:34:38.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.227+0000 7fad50455700 1 -- 172.21.15.2:0/3435386934 wait complete. 2023-08-25T03:34:38.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.228+0000 7fad50455700 1 Processor -- start 2023-08-25T03:34:38.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.228+0000 7fad50455700 1 -- start start 2023-08-25T03:34:38.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.229+0000 7fad50455700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad480cc1f0 0x7fad480af060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:38.236 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.229+0000 7fad50455700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fad4815a030 0x7fad48156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:38.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.229+0000 7fad50455700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fad4815a410 0x7fad481f3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:38.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.229+0000 7fad50455700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4815ba80 con 0x7fad4815a030 2023-08-25T03:34:38.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.229+0000 7fad50455700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad4815bd60 con 0x7fad480cc1f0 2023-08-25T03:34:38.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.229+0000 7fad50455700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad480ad270 con 0x7fad4815a410 2023-08-25T03:34:38.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.229+0000 7fad4d9f0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fad4815a030 0x7fad48156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:38.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.229+0000 7fad4d9f0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fad4815a030 0x7fad48156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60312/0 (socket says 172.21.15.2:60312) 2023-08-25T03:34:38.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.229+0000 7fad4d9f0700 1 -- 172.21.15.2:0/3331182734 learned_addr learned my addr 172.21.15.2:0/3331182734 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:38.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.229+0000 7fad4e1f1700 1 --2- 172.21.15.2:0/3331182734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad480cc1f0 0x7fad480af060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:38.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.229+0000 7fad4e9f2700 1 --2- 172.21.15.2:0/3331182734 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fad4815a410 0x7fad481f3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:38.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.229+0000 7fad4d9f0700 1 -- 172.21.15.2:0/3331182734 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fad4815a410 msgr2=0x7fad481f3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:38.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.229+0000 7fad4d9f0700 1 --2- 172.21.15.2:0/3331182734 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fad4815a410 0x7fad481f3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.229+0000 7fad4d9f0700 1 -- 172.21.15.2:0/3331182734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad480cc1f0 msgr2=0x7fad480af060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:38.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.229+0000 7fad4d9f0700 1 --2- 172.21.15.2:0/3331182734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad480cc1f0 0x7fad480af060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.229+0000 7fad4d9f0700 1 -- 172.21.15.2:0/3331182734 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3c0094a0 con 0x7fad4815a030 2023-08-25T03:34:38.239 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.230+0000 7fad4d9f0700 1 --2- 172.21.15.2:0/3331182734 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fad4815a030 0x7fad48156f70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fad4400e8d0 tx=0x7fad4400ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:38.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.230+0000 7fad3affd700 1 -- 172.21.15.2:0/3331182734 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad44009590 con 0x7fad4815a030 2023-08-25T03:34:38.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.230+0000 7fad3affd700 1 -- 172.21.15.2:0/3331182734 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad44013680 con 0x7fad4815a030 2023-08-25T03:34:38.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.230+0000 7fad50455700 1 -- 172.21.15.2:0/3331182734 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad481f3560 con 0x7fad4815a030 2023-08-25T03:34:38.240 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.230+0000 7fad3affd700 1 -- 172.21.15.2:0/3331182734 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad440139f0 con 0x7fad4815a030 2023-08-25T03:34:38.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.230+0000 7fad50455700 1 -- 172.21.15.2:0/3331182734 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad480ad7c0 con 0x7fad4815a030 2023-08-25T03:34:38.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.231+0000 7fad50455700 1 -- 172.21.15.2:0/3331182734 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad48066f70 con 0x7fad4815a030 2023-08-25T03:34:38.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.232+0000 7fad3affd700 1 -- 172.21.15.2:0/3331182734 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fad44013b50 con 0x7fad4815a030 2023-08-25T03:34:38.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.232+0000 7fad3affd700 1 --2- 172.21.15.2:0/3331182734 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fad2c06a0e0 0x7fad2c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:38.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.232+0000 7fad3affd700 1 -- 172.21.15.2:0/3331182734 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fad44088a30 con 0x7fad4815a030 2023-08-25T03:34:38.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.232+0000 7fad4e1f1700 1 --2- 172.21.15.2:0/3331182734 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fad2c06a0e0 0x7fad2c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:38.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.233+0000 7fad4e1f1700 1 --2- 172.21.15.2:0/3331182734 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fad2c06a0e0 0x7fad2c06c5a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fad3c000c00 tx=0x7fad3c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:38.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.236+0000 7fad3affd700 1 -- 172.21.15.2:0/3331182734 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fad44054290 con 0x7fad4815a030 2023-08-25T03:34:38.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.442+0000 7fad50455700 1 -- 172.21.15.2:0/3331182734 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad4815bef0 con 0x7fad4815a030 2023-08-25T03:34:38.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.443+0000 7fad3affd700 1 -- 172.21.15.2:0/3331182734 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18135 (secure 0 0 0) 0x7fad440578b0 con 0x7fad4815a030 2023-08-25T03:34:38.445 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:38.446 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":24,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:34:04.604059+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:34:00.869118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T03:34:01.987016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:34:38.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.446+0000 7fad50455700 1 -- 172.21.15.2:0/3331182734 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fad2c06a0e0 msgr2=0x7fad2c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:38.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.446+0000 7fad50455700 1 --2- 172.21.15.2:0/3331182734 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fad2c06a0e0 0x7fad2c06c5a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fad3c000c00 tx=0x7fad3c008030 comp rx=0 tx=0).stop 2023-08-25T03:34:38.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.446+0000 7fad50455700 1 -- 172.21.15.2:0/3331182734 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fad4815a030 msgr2=0x7fad48156f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:38.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.446+0000 7fad50455700 1 --2- 172.21.15.2:0/3331182734 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fad4815a030 0x7fad48156f70 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fad4400e8d0 tx=0x7fad4400ebe0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.446+0000 7fad50455700 1 -- 172.21.15.2:0/3331182734 shutdown_connections 2023-08-25T03:34:38.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.446+0000 7fad50455700 1 --2- 172.21.15.2:0/3331182734 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fad4815a410 0x7fad481f3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.446+0000 7fad50455700 1 --2- 172.21.15.2:0/3331182734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad480cc1f0 0x7fad480af060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.446+0000 7fad50455700 1 --2- 172.21.15.2:0/3331182734 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fad2c06a0e0 0x7fad2c06c5a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.446+0000 7fad50455700 1 --2- 172.21.15.2:0/3331182734 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fad4815a030 0x7fad48156f70 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.446+0000 7fad50455700 1 -- 172.21.15.2:0/3331182734 >> 172.21.15.2:0/3331182734 conn(0x7fad480b1cf0 msgr2=0x7fad480b20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:38.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.446+0000 7fad50455700 1 -- 172.21.15.2:0/3331182734 shutdown_connections 2023-08-25T03:34:38.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.446+0000 7fad50455700 1 -- 172.21.15.2:0/3331182734 wait complete. 2023-08-25T03:34:38.460 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T03:33:49.560643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T03:33:56.265184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T03:34:00.869118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T03:34:01.987016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T03:34:38.462 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T03:34:38.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.589+0000 7f943f1cf700 1 Processor -- start 2023-08-25T03:34:38.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.589+0000 7f943f1cf700 1 -- start start 2023-08-25T03:34:38.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.589+0000 7f943f1cf700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f94380af180 0x7f9438070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:38.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.589+0000 7f943f1cf700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f943806a1e0 0x7f943806a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:38.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.589+0000 7f943f1cf700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9438160bd0 con 0x7f94381607f0 2023-08-25T03:34:38.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.589+0000 7f943f1cf700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f943806abd0 con 0x7f94380af180 2023-08-25T03:34:38.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.589+0000 7f943f1cf700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94380713a0 con 0x7f943806a1e0 2023-08-25T03:34:38.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.590+0000 7f9437fff700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f94380af180 0x7f9438070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:38.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.590+0000 7f943cf6b700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f94381607f0 0x7f94380ac8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:46984/0 (socket says 172.21.15.2:46984) 2023-08-25T03:34:38.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.590+0000 7f943cf6b700 1 -- 172.21.15.2:0/3904164110 learned_addr learned my addr 172.21.15.2:0/3904164110 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:38.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.590+0000 7f943d76c700 1 --2- 172.21.15.2:0/3904164110 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f943806a1e0 0x7f943806a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:38.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.590+0000 7f9437fff700 1 -- 172.21.15.2:0/3904164110 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f943806a1e0 msgr2=0x7f943806a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:38.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.590+0000 7f9437fff700 1 --2- 172.21.15.2:0/3904164110 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f943806a1e0 0x7f943806a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.590+0000 7f9437fff700 1 -- 172.21.15.2:0/3904164110 >> v1:172.21.15.2:6789/0 conn(0x7f94381607f0 legacy=0x7f94380ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:38.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.590+0000 7f9437fff700 1 -- 172.21.15.2:0/3904164110 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94380cdd00 con 0x7f94380af180 2023-08-25T03:34:38.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.590+0000 7f94377fe700 1 -- 172.21.15.2:0/3904164110 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 953774948 0 0) 0x7f9438160bd0 con 0x7f94381607f0 2023-08-25T03:34:38.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.590+0000 7f9437fff700 1 --2- 172.21.15.2:0/3904164110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94380af180 0x7f9438070e60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9424009840 tx=0x7f9424009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c02b21fb1ede9929 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:38.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.590+0000 7f94377fe700 1 -- 172.21.15.2:0/3904164110 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9424011830 con 0x7f94380af180 2023-08-25T03:34:38.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.590+0000 7f94377fe700 1 -- 172.21.15.2:0/3904164110 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9424011990 con 0x7f94380af180 2023-08-25T03:34:38.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.590+0000 7f94377fe700 1 -- 172.21.15.2:0/3904164110 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9424011d20 con 0x7f94380af180 2023-08-25T03:34:38.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.591+0000 7f943f1cf700 1 -- 172.21.15.2:0/3904164110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94380af180 msgr2=0x7f9438070e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:38.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.591+0000 7f943f1cf700 1 --2- 172.21.15.2:0/3904164110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94380af180 0x7f9438070e60 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9424009840 tx=0x7f9424009b50 comp rx=0 tx=0).stop 2023-08-25T03:34:38.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.591+0000 7f943f1cf700 1 -- 172.21.15.2:0/3904164110 shutdown_connections 2023-08-25T03:34:38.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.591+0000 7f943f1cf700 1 --2- 172.21.15.2:0/3904164110 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f943806a1e0 0x7f943806a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.591+0000 7f943f1cf700 1 --2- 172.21.15.2:0/3904164110 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f94380af180 0x7f9438070e60 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.591+0000 7f943f1cf700 1 -- 172.21.15.2:0/3904164110 >> 172.21.15.2:0/3904164110 conn(0x7f94380cfd90 msgr2=0x7f94380d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:38.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.591+0000 7f943f1cf700 1 -- 172.21.15.2:0/3904164110 shutdown_connections 2023-08-25T03:34:38.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.591+0000 7f943f1cf700 1 -- 172.21.15.2:0/3904164110 wait complete. 2023-08-25T03:34:38.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.592+0000 7f943f1cf700 1 Processor -- start 2023-08-25T03:34:38.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.593+0000 7f943f1cf700 1 -- start start 2023-08-25T03:34:38.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.594+0000 7f943f1cf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f943806a1e0 0x7f94381592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:38.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.594+0000 7f943f1cf700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94380af180 0x7f943815bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:38.602 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.594+0000 7f943f1cf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94381607f0 0x7f943815c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:38.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.594+0000 7f943f1cf700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94380c7830 con 0x7f943806a1e0 2023-08-25T03:34:38.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.594+0000 7f943f1cf700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94380c6bd0 con 0x7f94381607f0 2023-08-25T03:34:38.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.594+0000 7f943cf6b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f943806a1e0 0x7f94381592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:38.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.594+0000 7f943f1cf700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94380c6f00 con 0x7f94380af180 2023-08-25T03:34:38.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.594+0000 7f943cf6b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f943806a1e0 0x7f94381592a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60324/0 (socket says 172.21.15.2:60324) 2023-08-25T03:34:38.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.594+0000 7f943cf6b700 1 -- 172.21.15.2:0/464745233 learned_addr learned my addr 172.21.15.2:0/464745233 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:38.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.594+0000 7f943cf6b700 1 -- 172.21.15.2:0/464745233 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94380af180 msgr2=0x7f943815bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:38.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.594+0000 7f943cf6b700 1 --2- 172.21.15.2:0/464745233 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94380af180 0x7f943815bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.594+0000 7f943cf6b700 1 -- 172.21.15.2:0/464745233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94381607f0 msgr2=0x7f943815c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:38.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.594+0000 7f943cf6b700 1 --2- 172.21.15.2:0/464745233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94381607f0 0x7f943815c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.594+0000 7f943cf6b700 1 -- 172.21.15.2:0/464745233 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94240094a0 con 0x7f943806a1e0 2023-08-25T03:34:38.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.594+0000 7f943cf6b700 1 --2- 172.21.15.2:0/464745233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f943806a1e0 0x7f94381592a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f942c00c550 tx=0x7f942c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:38.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.594+0000 7f94357fa700 1 -- 172.21.15.2:0/464745233 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c004c80 con 0x7f943806a1e0 2023-08-25T03:34:38.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.594+0000 7f94357fa700 1 -- 172.21.15.2:0/464745233 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f942c004e00 con 0x7f943806a1e0 2023-08-25T03:34:38.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.594+0000 7f943f1cf700 1 -- 172.21.15.2:0/464745233 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94380c71e0 con 0x7f943806a1e0 2023-08-25T03:34:38.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.594+0000 7f94357fa700 1 -- 172.21.15.2:0/464745233 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f942c007310 con 0x7f943806a1e0 2023-08-25T03:34:38.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.594+0000 7f943f1cf700 1 -- 172.21.15.2:0/464745233 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94381597e0 con 0x7f943806a1e0 2023-08-25T03:34:38.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.595+0000 7f943f1cf700 1 -- 172.21.15.2:0/464745233 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94380746d0 con 0x7f943806a1e0 2023-08-25T03:34:38.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.596+0000 7f94357fa700 1 -- 172.21.15.2:0/464745233 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f942c007470 con 0x7f943806a1e0 2023-08-25T03:34:38.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.596+0000 7f94357fa700 1 --2- 172.21.15.2:0/464745233 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f941c06a0e0 0x7f941c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:38.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.596+0000 7f94357fa700 1 -- 172.21.15.2:0/464745233 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f942c088340 con 0x7f943806a1e0 2023-08-25T03:34:38.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.596+0000 7f9437fff700 1 --2- 172.21.15.2:0/464745233 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f941c06a0e0 0x7f941c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:38.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.597+0000 7f9437fff700 1 --2- 172.21.15.2:0/464745233 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f941c06a0e0 0x7f941c06c5a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f94380cc390 tx=0x7f9424010000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:38.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.601+0000 7f94357fa700 1 -- 172.21.15.2:0/464745233 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f942c0539b0 con 0x7f943806a1e0 2023-08-25T03:34:38.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.798+0000 7f943f1cf700 1 -- 172.21.15.2:0/464745233 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f94380ce260 con 0x7f943806a1e0 2023-08-25T03:34:38.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.798+0000 7f94357fa700 1 -- 172.21.15.2:0/464745233 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f942c056fd0 con 0x7f943806a1e0 2023-08-25T03:34:38.801 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-08-25T03:34:38.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.800+0000 7f943f1cf700 1 -- 172.21.15.2:0/464745233 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f941c06a0e0 msgr2=0x7f941c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:38.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.800+0000 7f943f1cf700 1 --2- 172.21.15.2:0/464745233 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f941c06a0e0 0x7f941c06c5a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f94380cc390 tx=0x7f9424010000 comp rx=0 tx=0).stop 2023-08-25T03:34:38.803 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.800+0000 7f943f1cf700 1 -- 172.21.15.2:0/464745233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f943806a1e0 msgr2=0x7f94381592a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:38.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.800+0000 7f943f1cf700 1 --2- 172.21.15.2:0/464745233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f943806a1e0 0x7f94381592a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f942c00c550 tx=0x7f942c00c910 comp rx=0 tx=0).stop 2023-08-25T03:34:38.804 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.800+0000 7f943f1cf700 1 -- 172.21.15.2:0/464745233 shutdown_connections 2023-08-25T03:34:38.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.800+0000 7f943f1cf700 1 --2- 172.21.15.2:0/464745233 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f941c06a0e0 0x7f941c06c5a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.800+0000 7f943f1cf700 1 --2- 172.21.15.2:0/464745233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f943806a1e0 0x7f94381592a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.805 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.800+0000 7f943f1cf700 1 --2- 172.21.15.2:0/464745233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94381607f0 0x7f943815c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.800+0000 7f943f1cf700 1 --2- 172.21.15.2:0/464745233 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94380af180 0x7f943815bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.800+0000 7f943f1cf700 1 -- 172.21.15.2:0/464745233 >> 172.21.15.2:0/464745233 conn(0x7f94380cfd90 msgr2=0x7f94380d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:38.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.800+0000 7f943f1cf700 1 -- 172.21.15.2:0/464745233 shutdown_connections 2023-08-25T03:34:38.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.800+0000 7f943f1cf700 1 -- 172.21.15.2:0/464745233 wait complete. 2023-08-25T03:34:38.814 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T03:34:38.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.942+0000 7f0f88667700 1 Processor -- start 2023-08-25T03:34:38.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.942+0000 7f0f88667700 1 -- start start 2023-08-25T03:34:38.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.943+0000 7f0f88667700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0f801607f0 0x7f0f80070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:38.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.943+0000 7f0f88667700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f0f800af180 0x7f0f800ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:38.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.943+0000 7f0f88667700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0f8006a1e0 0x7f0f8006a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:38.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.943+0000 7f0f88667700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f800713a0 con 0x7f0f800af180 2023-08-25T03:34:38.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.943+0000 7f0f88667700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f800718a0 con 0x7f0f8006a1e0 2023-08-25T03:34:38.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.943+0000 7f0f88667700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8006ab70 con 0x7f0f801607f0 2023-08-25T03:34:38.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.944+0000 7f0f86c04700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0f8006a1e0 0x7f0f8006a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:38.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.944+0000 7f0f85c02700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f0f800af180 0x7f0f800ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:38.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.944+0000 7f0f85c02700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f0f800af180 0x7f0f800ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60334/0 (socket says 172.21.15.2:60334) 2023-08-25T03:34:38.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.944+0000 7f0f85c02700 1 -- 172.21.15.2:0/734056262 learned_addr learned my addr 172.21.15.2:0/734056262 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:38.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.944+0000 7f0f86403700 1 --2- 172.21.15.2:0/734056262 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0f801607f0 0x7f0f80070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:38.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.944+0000 7f0f85c02700 1 -- 172.21.15.2:0/734056262 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0f801607f0 msgr2=0x7f0f80070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:38.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.944+0000 7f0f85c02700 1 --2- 172.21.15.2:0/734056262 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0f801607f0 0x7f0f80070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.944+0000 7f0f85c02700 1 -- 172.21.15.2:0/734056262 >> v2:172.21.15.31:3300/0 conn(0x7f0f8006a1e0 msgr2=0x7f0f8006a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:38.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.944+0000 7f0f85c02700 1 --2- 172.21.15.2:0/734056262 >> v2:172.21.15.31:3300/0 conn(0x7f0f8006a1e0 0x7f0f8006a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.944+0000 7f0f85c02700 1 -- 172.21.15.2:0/734056262 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f80160bd0 con 0x7f0f800af180 2023-08-25T03:34:38.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.944+0000 7f0f85c02700 1 --2- 172.21.15.2:0/734056262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f800af180 0x7f0f800ac8e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0f8006b070 tx=0x7f0f7c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92545d72f8373de2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:38.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.944+0000 7f0f85401700 1 -- 172.21.15.2:0/734056262 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c011830 con 0x7f0f800af180 2023-08-25T03:34:38.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.945+0000 7f0f85401700 1 -- 172.21.15.2:0/734056262 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7c011990 con 0x7f0f800af180 2023-08-25T03:34:38.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.945+0000 7f0f85401700 1 -- 172.21.15.2:0/734056262 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c011d20 con 0x7f0f800af180 2023-08-25T03:34:38.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.945+0000 7f0f88667700 1 -- 172.21.15.2:0/734056262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f800af180 msgr2=0x7f0f800ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:38.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.945+0000 7f0f88667700 1 --2- 172.21.15.2:0/734056262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f800af180 0x7f0f800ac8e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0f8006b070 tx=0x7f0f7c009bd0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.945+0000 7f0f88667700 1 -- 172.21.15.2:0/734056262 shutdown_connections 2023-08-25T03:34:38.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.945+0000 7f0f88667700 1 --2- 172.21.15.2:0/734056262 >> v2:172.21.15.31:3300/0 conn(0x7f0f8006a1e0 0x7f0f8006a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.945+0000 7f0f88667700 1 --2- 172.21.15.2:0/734056262 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0f801607f0 0x7f0f80070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.945+0000 7f0f88667700 1 --2- 172.21.15.2:0/734056262 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f800af180 0x7f0f800ac8e0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.945+0000 7f0f88667700 1 -- 172.21.15.2:0/734056262 >> 172.21.15.2:0/734056262 conn(0x7f0f800cfd90 msgr2=0x7f0f800d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:38.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.946+0000 7f0f88667700 1 -- 172.21.15.2:0/734056262 shutdown_connections 2023-08-25T03:34:38.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.946+0000 7f0f88667700 1 -- 172.21.15.2:0/734056262 wait complete. 2023-08-25T03:34:38.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.947+0000 7f0f88667700 1 Processor -- start 2023-08-25T03:34:38.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.947+0000 7f0f88667700 1 -- start start 2023-08-25T03:34:38.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.947+0000 7f0f88667700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0f8006a1e0 0x7f0f800c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:38.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.948+0000 7f0f88667700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f800af180 0x7f0f801f6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:38.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.948+0000 7f0f88667700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f801607f0 0x7f0f8015c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:38.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.948+0000 7f0f88667700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f800c9ae0 con 0x7f0f800af180 2023-08-25T03:34:38.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.948+0000 7f0f88667700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f801f65d0 con 0x7f0f801607f0 2023-08-25T03:34:38.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.948+0000 7f0f88667700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8015c930 con 0x7f0f8006a1e0 2023-08-25T03:34:38.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.948+0000 7f0f85c02700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f800af180 0x7f0f801f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:38.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.948+0000 7f0f85c02700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f800af180 0x7f0f801f6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60340/0 (socket says 172.21.15.2:60340) 2023-08-25T03:34:38.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.948+0000 7f0f85c02700 1 -- 172.21.15.2:0/3869203985 learned_addr learned my addr 172.21.15.2:0/3869203985 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:38.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.948+0000 7f0f86c04700 1 --2- 172.21.15.2:0/3869203985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f801607f0 0x7f0f8015c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:38.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.948+0000 7f0f86403700 1 --2- 172.21.15.2:0/3869203985 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0f8006a1e0 0x7f0f800c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:38.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.948+0000 7f0f85c02700 1 -- 172.21.15.2:0/3869203985 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0f8006a1e0 msgr2=0x7f0f800c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:38.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.948+0000 7f0f85c02700 1 --2- 172.21.15.2:0/3869203985 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0f8006a1e0 0x7f0f800c95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.948+0000 7f0f85c02700 1 -- 172.21.15.2:0/3869203985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f801607f0 msgr2=0x7f0f8015c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:38.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.948+0000 7f0f85c02700 1 --2- 172.21.15.2:0/3869203985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f801607f0 0x7f0f8015c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:38.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.948+0000 7f0f85c02700 1 -- 172.21.15.2:0/3869203985 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f7c009550 con 0x7f0f800af180 2023-08-25T03:34:38.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.948+0000 7f0f86c04700 1 --2- 172.21.15.2:0/3869203985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f801607f0 0x7f0f8015c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:38.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.948+0000 7f0f85c02700 1 --2- 172.21.15.2:0/3869203985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f800af180 0x7f0f801f6090 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009c10 tx=0x7f0f7c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:38.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.948+0000 7f0f76ffd700 1 -- 172.21.15.2:0/3869203985 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c004300 con 0x7f0f800af180 2023-08-25T03:34:38.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.948+0000 7f0f76ffd700 1 -- 172.21.15.2:0/3869203985 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f7c01f070 con 0x7f0f800af180 2023-08-25T03:34:38.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.948+0000 7f0f88667700 1 -- 172.21.15.2:0/3869203985 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f801f6800 con 0x7f0f800af180 2023-08-25T03:34:38.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.948+0000 7f0f76ffd700 1 -- 172.21.15.2:0/3869203985 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c0036a0 con 0x7f0f800af180 2023-08-25T03:34:38.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.948+0000 7f0f88667700 1 -- 172.21.15.2:0/3869203985 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f801f6dc0 con 0x7f0f800af180 2023-08-25T03:34:38.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.949+0000 7f0f88667700 1 -- 172.21.15.2:0/3869203985 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f800746d0 con 0x7f0f800af180 2023-08-25T03:34:38.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.950+0000 7f0f76ffd700 1 -- 172.21.15.2:0/3869203985 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0f7c021030 con 0x7f0f800af180 2023-08-25T03:34:38.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.950+0000 7f0f76ffd700 1 --2- 172.21.15.2:0/3869203985 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f0f6406a0e0 0x7f0f6406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:38.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.950+0000 7f0f76ffd700 1 -- 172.21.15.2:0/3869203985 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0f7c089700 con 0x7f0f800af180 2023-08-25T03:34:38.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.950+0000 7f0f86403700 1 --2- 172.21.15.2:0/3869203985 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f0f6406a0e0 0x7f0f6406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:38.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.951+0000 7f0f86403700 1 --2- 172.21.15.2:0/3869203985 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f0f6406a0e0 0x7f0f6406c5a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009a10 tx=0x7f0f70008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:38.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:38.955+0000 7f0f76ffd700 1 -- 172.21.15.2:0/3869203985 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0f7c054e80 con 0x7f0f800af180 2023-08-25T03:34:39.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.152+0000 7f0f88667700 1 -- 172.21.15.2:0/3869203985 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0f800ae800 con 0x7f0f800af180 2023-08-25T03:34:39.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.152+0000 7f0f76ffd700 1 -- 172.21.15.2:0/3869203985 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0f7c0584a0 con 0x7f0f800af180 2023-08-25T03:34:39.155 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 8 2023-08-25T03:34:39.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.154+0000 7f0f88667700 1 -- 172.21.15.2:0/3869203985 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f0f6406a0e0 msgr2=0x7f0f6406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:39.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.154+0000 7f0f88667700 1 --2- 172.21.15.2:0/3869203985 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f0f6406a0e0 0x7f0f6406c5a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0f70009a10 tx=0x7f0f70008040 comp rx=0 tx=0).stop 2023-08-25T03:34:39.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.154+0000 7f0f88667700 1 -- 172.21.15.2:0/3869203985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f800af180 msgr2=0x7f0f801f6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:39.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.154+0000 7f0f88667700 1 --2- 172.21.15.2:0/3869203985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f800af180 0x7f0f801f6090 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009c10 tx=0x7f0f7c004750 comp rx=0 tx=0).stop 2023-08-25T03:34:39.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.154+0000 7f0f88667700 1 -- 172.21.15.2:0/3869203985 shutdown_connections 2023-08-25T03:34:39.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.154+0000 7f0f88667700 1 --2- 172.21.15.2:0/3869203985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f801607f0 0x7f0f8015c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:39.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.154+0000 7f0f88667700 1 --2- 172.21.15.2:0/3869203985 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0f8006a1e0 0x7f0f800c95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:39.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.154+0000 7f0f88667700 1 --2- 172.21.15.2:0/3869203985 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f0f6406a0e0 0x7f0f6406c5a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:39.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.154+0000 7f0f88667700 1 --2- 172.21.15.2:0/3869203985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0f800af180 0x7f0f801f6090 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:39.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.154+0000 7f0f88667700 1 -- 172.21.15.2:0/3869203985 >> 172.21.15.2:0/3869203985 conn(0x7f0f800cfd90 msgr2=0x7f0f800af9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:39.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.154+0000 7f0f88667700 1 -- 172.21.15.2:0/3869203985 shutdown_connections 2023-08-25T03:34:39.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.154+0000 7f0f88667700 1 -- 172.21.15.2:0/3869203985 wait complete. 2023-08-25T03:34:39.170 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T03:34:39.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.297+0000 7f1097e52700 1 Processor -- start 2023-08-25T03:34:39.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.297+0000 7f1097e52700 1 -- start start 2023-08-25T03:34:39.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.298+0000 7f1097e52700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f10901607f0 0x7f1090070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:39.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.298+0000 7f1097e52700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10900af180 0x7f10900ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:39.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.298+0000 7f1097e52700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1090160bd0 con 0x7f109006a1e0 2023-08-25T03:34:39.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.298+0000 7f1097e52700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10900713a0 con 0x7f10900af180 2023-08-25T03:34:39.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.298+0000 7f1097e52700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10900718a0 con 0x7f10901607f0 2023-08-25T03:34:39.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.298+0000 7f10963ef700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f109006a1e0 0x7f109006ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47014/0 (socket says 172.21.15.2:47014) 2023-08-25T03:34:39.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.298+0000 7f10963ef700 1 -- 172.21.15.2:0/3176124513 learned_addr learned my addr 172.21.15.2:0/3176124513 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:39.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.298+0000 7f10953ed700 1 --2- 172.21.15.2:0/3176124513 >> v2:172.21.15.31:3300/0 conn(0x7f10900af180 0x7f10900ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:39.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.298+0000 7f1095bee700 1 --2- 172.21.15.2:0/3176124513 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f10901607f0 0x7f1090070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:39.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.298+0000 7f1094bec700 1 -- 172.21.15.2:0/3176124513 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1090054228 0 0) 0x7f1090160bd0 con 0x7f109006a1e0 2023-08-25T03:34:39.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.299+0000 7f1094bec700 1 -- 172.21.15.2:0/3176124513 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f107c0035b0 con 0x7f109006a1e0 2023-08-25T03:34:39.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.299+0000 7f1094bec700 1 -- 172.21.15.2:0/3176124513 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 255971157 0 0) 0x7f107c0035b0 con 0x7f109006a1e0 2023-08-25T03:34:39.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.299+0000 7f1094bec700 1 -- 172.21.15.2:0/3176124513 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f10901607f0 msgr2=0x7f1090070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:39.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.299+0000 7f1094bec700 1 --2- 172.21.15.2:0/3176124513 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f10901607f0 0x7f1090070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:39.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.299+0000 7f1094bec700 1 -- 172.21.15.2:0/3176124513 >> v2:172.21.15.31:3300/0 conn(0x7f10900af180 msgr2=0x7f10900ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:39.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.299+0000 7f1094bec700 1 --2- 172.21.15.2:0/3176124513 >> v2:172.21.15.31:3300/0 conn(0x7f10900af180 0x7f10900ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:39.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.299+0000 7f1094bec700 1 -- 172.21.15.2:0/3176124513 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10900cdd00 con 0x7f109006a1e0 2023-08-25T03:34:39.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.299+0000 7f1094bec700 1 -- 172.21.15.2:0/3176124513 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f1088001a40 con 0x7f109006a1e0 2023-08-25T03:34:39.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.299+0000 7f1094bec700 1 -- 172.21.15.2:0/3176124513 >> v1:172.21.15.2:6789/0 conn(0x7f109006a1e0 legacy=0x7f109006ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:39.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.299+0000 7f1094bec700 1 --2- 172.21.15.2:0/3176124513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f107c003a00 0x7f107c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:39.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.299+0000 7f1094bec700 1 --2- 172.21.15.2:0/3176124513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f107c004850 0x7f107c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:39.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.299+0000 7f1094bec700 1 --2- 172.21.15.2:0/3176124513 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f10901607f0 0x7f107c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:39.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.299+0000 7f1094bec700 1 -- 172.21.15.2:0/3176124513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10900718a0 con 0x7f107c004850 2023-08-25T03:34:39.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.299+0000 7f1094bec700 1 -- 172.21.15.2:0/3176124513 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107c00ba10 con 0x7f107c003a00 2023-08-25T03:34:39.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.299+0000 7f1094bec700 1 -- 172.21.15.2:0/3176124513 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107c00bb50 con 0x7f10901607f0 2023-08-25T03:34:39.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.299+0000 7f10953ed700 1 --2- 172.21.15.2:0/3176124513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f107c004850 0x7f107c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:39.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.299+0000 7f1095bee700 1 --2- 172.21.15.2:0/3176124513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f107c003a00 0x7f107c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:39.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.299+0000 7f10963ef700 1 --2- 172.21.15.2:0/3176124513 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f10901607f0 0x7f107c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:39.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.299+0000 7f10953ed700 1 -- 172.21.15.2:0/3176124513 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f10901607f0 msgr2=0x7f107c005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:39.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.299+0000 7f10953ed700 1 --2- 172.21.15.2:0/3176124513 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f10901607f0 0x7f107c005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:39.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.299+0000 7f10953ed700 1 -- 172.21.15.2:0/3176124513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f107c003a00 msgr2=0x7f107c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:39.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.299+0000 7f10953ed700 1 --2- 172.21.15.2:0/3176124513 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f107c003a00 0x7f107c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:39.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.299+0000 7f10953ed700 1 -- 172.21.15.2:0/3176124513 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107c0035b0 con 0x7f107c004850 2023-08-25T03:34:39.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.300+0000 7f10953ed700 1 --2- 172.21.15.2:0/3176124513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f107c004850 0x7f107c004cd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1084009b60 tx=0x7f107c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c7413141f516044 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:39.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.300+0000 7f1094bec700 1 -- 172.21.15.2:0/3176124513 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108400e6c0 con 0x7f107c004850 2023-08-25T03:34:39.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.300+0000 7f1094bec700 1 -- 172.21.15.2:0/3176124513 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f108400e820 con 0x7f107c004850 2023-08-25T03:34:39.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.300+0000 7f1094bec700 1 -- 172.21.15.2:0/3176124513 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108400eb90 con 0x7f107c004850 2023-08-25T03:34:39.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.300+0000 7f1097e52700 1 -- 172.21.15.2:0/3176124513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f107c004850 msgr2=0x7f107c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:39.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.300+0000 7f1097e52700 1 --2- 172.21.15.2:0/3176124513 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f107c004850 0x7f107c004cd0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1084009b60 tx=0x7f107c005880 comp rx=0 tx=0).stop 2023-08-25T03:34:39.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.300+0000 7f10963ef700 1 -- 172.21.15.2:0/3176124513 reap_dead start 2023-08-25T03:34:39.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.300+0000 7f1097e52700 1 -- 172.21.15.2:0/3176124513 shutdown_connections 2023-08-25T03:34:39.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.300+0000 7f1097e52700 1 -- 172.21.15.2:0/3176124513 >> 172.21.15.2:0/3176124513 conn(0x7f10900cfd90 msgr2=0x7f10900d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:39.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.300+0000 7f1097e52700 1 -- 172.21.15.2:0/3176124513 shutdown_connections 2023-08-25T03:34:39.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.300+0000 7f1097e52700 1 -- 172.21.15.2:0/3176124513 wait complete. 2023-08-25T03:34:39.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.302+0000 7f1097e52700 1 Processor -- start 2023-08-25T03:34:39.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.302+0000 7f1097e52700 1 -- start start 2023-08-25T03:34:39.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.302+0000 7f1097e52700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f107c004850 0x7f10900b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:39.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.302+0000 7f1097e52700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10900b2f80 0x7f10901f2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:39.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.302+0000 7f1097e52700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10901f3290 0x7f10901f3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:39.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.302+0000 7f1097e52700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10901f3c80 con 0x7f10900b2f80 2023-08-25T03:34:39.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.302+0000 7f1097e52700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10900b3550 con 0x7f10901f3290 2023-08-25T03:34:39.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.302+0000 7f1097e52700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10900c9700 con 0x7f107c004850 2023-08-25T03:34:39.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.302+0000 7f10953ed700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10900b2f80 0x7f10901f2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:39.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.302+0000 7f10953ed700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10900b2f80 0x7f10901f2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60358/0 (socket says 172.21.15.2:60358) 2023-08-25T03:34:39.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.302+0000 7f10953ed700 1 -- 172.21.15.2:0/1304900691 learned_addr learned my addr 172.21.15.2:0/1304900691 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:39.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.303+0000 7f1095bee700 1 --2- 172.21.15.2:0/1304900691 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f107c004850 0x7f10900b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:39.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.303+0000 7f10963ef700 1 --2- 172.21.15.2:0/1304900691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10901f3290 0x7f10901f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:39.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.303+0000 7f10953ed700 1 -- 172.21.15.2:0/1304900691 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f107c004850 msgr2=0x7f10900b5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:39.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.303+0000 7f10953ed700 1 --2- 172.21.15.2:0/1304900691 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f107c004850 0x7f10900b5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:39.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.303+0000 7f10953ed700 1 -- 172.21.15.2:0/1304900691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10901f3290 msgr2=0x7f10901f3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:39.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.303+0000 7f10953ed700 1 --2- 172.21.15.2:0/1304900691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10901f3290 0x7f10901f3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:39.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.303+0000 7f10953ed700 1 -- 172.21.15.2:0/1304900691 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10900c99e0 con 0x7f10900b2f80 2023-08-25T03:34:39.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.303+0000 7f10953ed700 1 --2- 172.21.15.2:0/1304900691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10900b2f80 0x7f10901f2d50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f10840090b0 tx=0x7f10840047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:39.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.303+0000 7f10827fc700 1 -- 172.21.15.2:0/1304900691 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f108400e6c0 con 0x7f10900b2f80 2023-08-25T03:34:39.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.303+0000 7f10827fc700 1 -- 172.21.15.2:0/1304900691 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1084003ad0 con 0x7f10900b2f80 2023-08-25T03:34:39.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.303+0000 7f1097e52700 1 -- 172.21.15.2:0/1304900691 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10900c9cc0 con 0x7f10900b2f80 2023-08-25T03:34:39.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.303+0000 7f10827fc700 1 -- 172.21.15.2:0/1304900691 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1084003e80 con 0x7f10900b2f80 2023-08-25T03:34:39.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.303+0000 7f1097e52700 1 -- 172.21.15.2:0/1304900691 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109015d3d0 con 0x7f10900b2f80 2023-08-25T03:34:39.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.304+0000 7f1097e52700 1 -- 172.21.15.2:0/1304900691 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10900746d0 con 0x7f10900b2f80 2023-08-25T03:34:39.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.305+0000 7f10827fc700 1 -- 172.21.15.2:0/1304900691 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1084013470 con 0x7f10900b2f80 2023-08-25T03:34:39.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.305+0000 7f10827fc700 1 --2- 172.21.15.2:0/1304900691 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f107406a0e0 0x7f107406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:39.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.305+0000 7f10827fc700 1 -- 172.21.15.2:0/1304900691 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f1084089e50 con 0x7f10900b2f80 2023-08-25T03:34:39.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.305+0000 7f1095bee700 1 --2- 172.21.15.2:0/1304900691 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f107406a0e0 0x7f107406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:39.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.305+0000 7f1095bee700 1 --2- 172.21.15.2:0/1304900691 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f107406a0e0 0x7f107406c5a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f108c009b80 tx=0x7f108c009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:39.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.309+0000 7f10827fc700 1 -- 172.21.15.2:0/1304900691 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f10840555d0 con 0x7f10900b2f80 2023-08-25T03:34:39.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.515+0000 7f1097e52700 1 -- 172.21.15.2:0/1304900691 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f10900b5b40 con 0x7f10900b2f80 2023-08-25T03:34:39.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.515+0000 7f10827fc700 1 -- 172.21.15.2:0/1304900691 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1084017020 con 0x7f10900b2f80 2023-08-25T03:34:39.517 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-08-25T03:34:39.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.517+0000 7f1097e52700 1 -- 172.21.15.2:0/1304900691 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f107406a0e0 msgr2=0x7f107406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.517+0000 7f1097e52700 1 --2- 172.21.15.2:0/1304900691 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f107406a0e0 0x7f107406c5a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f108c009b80 tx=0x7f108c009210 comp rx=0 tx=0).stop 2023-08-25T03:34:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.517+0000 7f1097e52700 1 -- 172.21.15.2:0/1304900691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10900b2f80 msgr2=0x7f10901f2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.517+0000 7f1097e52700 1 --2- 172.21.15.2:0/1304900691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10900b2f80 0x7f10901f2d50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f10840090b0 tx=0x7f10840047c0 comp rx=0 tx=0).stop 2023-08-25T03:34:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.517+0000 7f1097e52700 1 -- 172.21.15.2:0/1304900691 shutdown_connections 2023-08-25T03:34:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.517+0000 7f1097e52700 1 --2- 172.21.15.2:0/1304900691 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10901f3290 0x7f10901f3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:39.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.517+0000 7f1097e52700 1 --2- 172.21.15.2:0/1304900691 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f107c004850 0x7f10900b5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.517+0000 7f1097e52700 1 --2- 172.21.15.2:0/1304900691 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f107406a0e0 0x7f107406c5a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:39.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.517+0000 7f1097e52700 1 --2- 172.21.15.2:0/1304900691 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f10900b2f80 0x7f10901f2d50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:39.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.517+0000 7f1097e52700 1 -- 172.21.15.2:0/1304900691 >> 172.21.15.2:0/1304900691 conn(0x7f10900cfd90 msgr2=0x7f10900b04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:39.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.517+0000 7f1097e52700 1 -- 172.21.15.2:0/1304900691 shutdown_connections 2023-08-25T03:34:39.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.518+0000 7f1097e52700 1 -- 172.21.15.2:0/1304900691 wait complete. 2023-08-25T03:34:39.532 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T03:34:39.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.659+0000 7f5b0cbf2700 1 Processor -- start 2023-08-25T03:34:39.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.659+0000 7f5b0cbf2700 1 -- start start 2023-08-25T03:34:39.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.660+0000 7f5b0cbf2700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5b0806a450 0x7f5b080d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:39.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.660+0000 7f5b0cbf2700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5b08071330 0x7f5b0806d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:39.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.660+0000 7f5b0cbf2700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b080cc470 con 0x7f5b0806a830 2023-08-25T03:34:39.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.660+0000 7f5b0cbf2700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b080cbe90 con 0x7f5b0806a450 2023-08-25T03:34:39.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.660+0000 7f5b0cbf2700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b08071740 con 0x7f5b08071330 2023-08-25T03:34:39.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.660+0000 7f5b0659c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5b0806a450 0x7f5b080d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:39.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.660+0000 7f5b0659c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5b0806a450 0x7f5b080d08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60364/0 (socket says 172.21.15.2:60364) 2023-08-25T03:34:39.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.660+0000 7f5b0659c700 1 -- 172.21.15.2:0/3942263961 learned_addr learned my addr 172.21.15.2:0/3942263961 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:39.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.660+0000 7f5b06d9d700 1 --2- 172.21.15.2:0/3942263961 >> v2:172.21.15.31:3300/0 conn(0x7f5b08071330 0x7f5b0806d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:39.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.660+0000 7f5b0659c700 1 -- 172.21.15.2:0/3942263961 >> v2:172.21.15.31:3300/0 conn(0x7f5b08071330 msgr2=0x7f5b0806d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:39.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.660+0000 7f5b0659c700 1 --2- 172.21.15.2:0/3942263961 >> v2:172.21.15.31:3300/0 conn(0x7f5b08071330 0x7f5b0806d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:39.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.660+0000 7f5b0659c700 1 -- 172.21.15.2:0/3942263961 >> v1:172.21.15.31:6789/0 conn(0x7f5b0806a830 legacy=0x7f5b0806cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:39.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.660+0000 7f5b0659c700 1 -- 172.21.15.2:0/3942263961 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b080ca3e0 con 0x7f5b0806a450 2023-08-25T03:34:39.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.660+0000 7f5b06d9d700 1 --2- 172.21.15.2:0/3942263961 >> v2:172.21.15.31:3300/0 conn(0x7f5b08071330 0x7f5b0806d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:39.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.661+0000 7f5b0659c700 1 --2- 172.21.15.2:0/3942263961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b0806a450 0x7f5b080d08b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5b0806e940 tx=0x7f5afc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a14d438b2a0ed12 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:39.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.661+0000 7f5b0559a700 1 -- 172.21.15.2:0/3942263961 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc004230 con 0x7f5b0806a450 2023-08-25T03:34:39.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.661+0000 7f5b0559a700 1 -- 172.21.15.2:0/3942263961 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5afc004390 con 0x7f5b0806a450 2023-08-25T03:34:39.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.661+0000 7f5b0559a700 1 -- 172.21.15.2:0/3942263961 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc0036a0 con 0x7f5b0806a450 2023-08-25T03:34:39.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.661+0000 7f5b0cbf2700 1 -- 172.21.15.2:0/3942263961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b0806a450 msgr2=0x7f5b080d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:39.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.661+0000 7f5b0cbf2700 1 --2- 172.21.15.2:0/3942263961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b0806a450 0x7f5b080d08b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5b0806e940 tx=0x7f5afc009bd0 comp rx=0 tx=0).stop 2023-08-25T03:34:39.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.661+0000 7f5b0cbf2700 1 -- 172.21.15.2:0/3942263961 shutdown_connections 2023-08-25T03:34:39.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.661+0000 7f5b0cbf2700 1 --2- 172.21.15.2:0/3942263961 >> v2:172.21.15.31:3300/0 conn(0x7f5b08071330 0x7f5b0806d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:39.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.661+0000 7f5b0cbf2700 1 --2- 172.21.15.2:0/3942263961 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b0806a450 0x7f5b080d08b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:39.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.661+0000 7f5b0cbf2700 1 -- 172.21.15.2:0/3942263961 >> 172.21.15.2:0/3942263961 conn(0x7f5b080cfd90 msgr2=0x7f5b080d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:39.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.661+0000 7f5b0cbf2700 1 -- 172.21.15.2:0/3942263961 shutdown_connections 2023-08-25T03:34:39.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.661+0000 7f5b0cbf2700 1 -- 172.21.15.2:0/3942263961 wait complete. 2023-08-25T03:34:39.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.663+0000 7f5b0cbf2700 1 Processor -- start 2023-08-25T03:34:39.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.663+0000 7f5b0cbf2700 1 -- start start 2023-08-25T03:34:39.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.664+0000 7f5b0cbf2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b0806a450 0x7f5b08158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:39.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.664+0000 7f5b0cbf2700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5b0806a830 0x7f5b08153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:39.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.664+0000 7f5b0cbf2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b08071330 0x7f5b080c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:39.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.664+0000 7f5b0cbf2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b080ce950 con 0x7f5b0806a450 2023-08-25T03:34:39.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.664+0000 7f5b0cbf2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b080ce670 con 0x7f5b08071330 2023-08-25T03:34:39.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.664+0000 7f5b0cbf2700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b08158470 con 0x7f5b0806a830 2023-08-25T03:34:39.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.664+0000 7f5b0659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b0806a450 0x7f5b08158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:39.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.664+0000 7f5b0659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b0806a450 0x7f5b08158f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60370/0 (socket says 172.21.15.2:60370) 2023-08-25T03:34:39.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.664+0000 7f5b0659c700 1 -- 172.21.15.2:0/2351909218 learned_addr learned my addr 172.21.15.2:0/2351909218 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:39.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.664+0000 7f5b05d9b700 1 --2- 172.21.15.2:0/2351909218 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5b0806a830 0x7f5b08153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:39.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.664+0000 7f5b06d9d700 1 --2- 172.21.15.2:0/2351909218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b08071330 0x7f5b080c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:39.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.665+0000 7f5b0659c700 1 -- 172.21.15.2:0/2351909218 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5b0806a830 msgr2=0x7f5b08153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:39.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.665+0000 7f5b0659c700 1 --2- 172.21.15.2:0/2351909218 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5b0806a830 0x7f5b08153e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:39.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.665+0000 7f5b0659c700 1 -- 172.21.15.2:0/2351909218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b08071330 msgr2=0x7f5b080c3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:39.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.665+0000 7f5b0659c700 1 --2- 172.21.15.2:0/2351909218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b08071330 0x7f5b080c3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:39.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.665+0000 7f5b0659c700 1 -- 172.21.15.2:0/2351909218 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5afc009550 con 0x7f5b0806a450 2023-08-25T03:34:39.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.665+0000 7f5b0659c700 1 --2- 172.21.15.2:0/2351909218 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b0806a450 0x7f5b08158f50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5b0806e000 tx=0x7f5afc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:39.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.665+0000 7f5af6ffd700 1 -- 172.21.15.2:0/2351909218 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc003c90 con 0x7f5b0806a450 2023-08-25T03:34:39.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.665+0000 7f5af6ffd700 1 -- 172.21.15.2:0/2351909218 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5afc01f070 con 0x7f5b0806a450 2023-08-25T03:34:39.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.665+0000 7f5b0cbf2700 1 -- 172.21.15.2:0/2351909218 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b08158750 con 0x7f5b0806a450 2023-08-25T03:34:39.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.665+0000 7f5af6ffd700 1 -- 172.21.15.2:0/2351909218 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc011870 con 0x7f5b0806a450 2023-08-25T03:34:39.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.665+0000 7f5b0cbf2700 1 -- 172.21.15.2:0/2351909218 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b080c4360 con 0x7f5b0806a450 2023-08-25T03:34:39.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.666+0000 7f5b0cbf2700 1 -- 172.21.15.2:0/2351909218 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b081582d0 con 0x7f5b0806a450 2023-08-25T03:34:39.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.667+0000 7f5af6ffd700 1 -- 172.21.15.2:0/2351909218 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5afc021030 con 0x7f5b0806a450 2023-08-25T03:34:39.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.667+0000 7f5af6ffd700 1 --2- 172.21.15.2:0/2351909218 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5ae806a0e0 0x7f5ae806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:39.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.667+0000 7f5af6ffd700 1 -- 172.21.15.2:0/2351909218 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5afc089970 con 0x7f5b0806a450 2023-08-25T03:34:39.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.667+0000 7f5b05d9b700 1 --2- 172.21.15.2:0/2351909218 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5ae806a0e0 0x7f5ae806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:39.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.668+0000 7f5b05d9b700 1 --2- 172.21.15.2:0/2351909218 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5ae806a0e0 0x7f5ae806c5a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5b080c4b20 tx=0x7f5af8007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:39.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.671+0000 7f5af6ffd700 1 -- 172.21.15.2:0/2351909218 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5afc0551d0 con 0x7f5b0806a450 2023-08-25T03:34:39.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.869+0000 7f5b0cbf2700 1 -- 172.21.15.2:0/2351909218 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5b081582d0 con 0x7f5b0806a450 2023-08-25T03:34:39.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.869+0000 7f5af6ffd700 1 -- 172.21.15.2:0/2351909218 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5b081582d0 con 0x7f5b0806a450 2023-08-25T03:34:39.872 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-08-25T03:34:39.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.872+0000 7f5b0cbf2700 1 -- 172.21.15.2:0/2351909218 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5ae806a0e0 msgr2=0x7f5ae806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:39.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.872+0000 7f5b0cbf2700 1 --2- 172.21.15.2:0/2351909218 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5ae806a0e0 0x7f5ae806c5a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5b080c4b20 tx=0x7f5af8007ec0 comp rx=0 tx=0).stop 2023-08-25T03:34:39.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.872+0000 7f5b0cbf2700 1 -- 172.21.15.2:0/2351909218 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b0806a450 msgr2=0x7f5b08158f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:39.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.872+0000 7f5b0cbf2700 1 --2- 172.21.15.2:0/2351909218 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b0806a450 0x7f5b08158f50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5b0806e000 tx=0x7f5afc004750 comp rx=0 tx=0).stop 2023-08-25T03:34:39.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.872+0000 7f5b0cbf2700 1 -- 172.21.15.2:0/2351909218 shutdown_connections 2023-08-25T03:34:39.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.872+0000 7f5b0cbf2700 1 --2- 172.21.15.2:0/2351909218 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5ae806a0e0 0x7f5ae806c5a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:39.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.872+0000 7f5b0cbf2700 1 --2- 172.21.15.2:0/2351909218 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5b0806a450 0x7f5b08158f50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:39.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.872+0000 7f5b0cbf2700 1 --2- 172.21.15.2:0/2351909218 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5b08071330 0x7f5b080c3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:39.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.872+0000 7f5b0cbf2700 1 --2- 172.21.15.2:0/2351909218 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5b0806a830 0x7f5b08153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:39.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.872+0000 7f5b0cbf2700 1 -- 172.21.15.2:0/2351909218 >> 172.21.15.2:0/2351909218 conn(0x7f5b080cfd90 msgr2=0x7f5b080c5b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:39.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.872+0000 7f5b0cbf2700 1 -- 172.21.15.2:0/2351909218 shutdown_connections 2023-08-25T03:34:39.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:39.872+0000 7f5b0cbf2700 1 -- 172.21.15.2:0/2351909218 wait complete. 2023-08-25T03:34:39.910 INFO:tasks.cephfs_test_runner:Starting test: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-08-25T03:34:39.911 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values' 2023-08-25T03:34:40.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.035+0000 7f3324086700 1 Processor -- start 2023-08-25T03:34:40.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.035+0000 7f3324086700 1 -- start start 2023-08-25T03:34:40.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.036+0000 7f3324086700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f331c15e890 0x7f331c071430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:40.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.036+0000 7f3324086700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f331c0caea0 0x7f331c0cb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:40.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.036+0000 7f3324086700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f331c0720b0 con 0x7f331c0b19a0 2023-08-25T03:34:40.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.036+0000 7f3324086700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f331c071dd0 con 0x7f331c0caea0 2023-08-25T03:34:40.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.036+0000 7f3324086700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f331c15ddb0 con 0x7f331c15e890 2023-08-25T03:34:40.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.036+0000 7f3321e22700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f331c0b19a0 0x7f331c0b1d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47044/0 (socket says 172.21.15.2:47044) 2023-08-25T03:34:40.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.036+0000 7f3321e22700 1 -- 172.21.15.2:0/3810517245 learned_addr learned my addr 172.21.15.2:0/3810517245 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:40.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.036+0000 7f3322623700 1 --2- 172.21.15.2:0/3810517245 >> v2:172.21.15.2:3300/0 conn(0x7f331c0caea0 0x7f331c0cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:40.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.036+0000 7f3321621700 1 --2- 172.21.15.2:0/3810517245 >> v2:172.21.15.31:3300/0 conn(0x7f331c15e890 0x7f331c071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:40.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.036+0000 7f3320e20700 1 -- 172.21.15.2:0/3810517245 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1439585622 0 0) 0x7f331c0720b0 con 0x7f331c0b19a0 2023-08-25T03:34:40.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.036+0000 7f3320e20700 1 -- 172.21.15.2:0/3810517245 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33080035b0 con 0x7f331c0b19a0 2023-08-25T03:34:40.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.036+0000 7f3322623700 1 -- 172.21.15.2:0/3810517245 >> v2:172.21.15.31:3300/0 conn(0x7f331c15e890 msgr2=0x7f331c071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:40.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.036+0000 7f3322623700 1 --2- 172.21.15.2:0/3810517245 >> v2:172.21.15.31:3300/0 conn(0x7f331c15e890 0x7f331c071430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:40.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.036+0000 7f3322623700 1 -- 172.21.15.2:0/3810517245 >> v1:172.21.15.2:6789/0 conn(0x7f331c0b19a0 legacy=0x7f331c0b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:40.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.036+0000 7f3322623700 1 -- 172.21.15.2:0/3810517245 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331c15e090 con 0x7f331c0caea0 2023-08-25T03:34:40.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.036+0000 7f3321621700 1 --2- 172.21.15.2:0/3810517245 >> v2:172.21.15.31:3300/0 conn(0x7f331c15e890 0x7f331c071430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:40.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.036+0000 7f3322623700 1 --2- 172.21.15.2:0/3810517245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f331c0caea0 0x7f331c0cb330 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f331c06dce0 tx=0x7f331400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2183d51226bbc0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:40.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.037+0000 7f3320e20700 1 -- 172.21.15.2:0/3810517245 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f331400e070 con 0x7f331c0caea0 2023-08-25T03:34:40.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.037+0000 7f3320e20700 1 -- 172.21.15.2:0/3810517245 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f331400bb20 con 0x7f331c0caea0 2023-08-25T03:34:40.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.037+0000 7f3320e20700 1 -- 172.21.15.2:0/3810517245 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3314012430 con 0x7f331c0caea0 2023-08-25T03:34:40.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.037+0000 7f3324086700 1 -- 172.21.15.2:0/3810517245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f331c0caea0 msgr2=0x7f331c0cb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:40.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.037+0000 7f3324086700 1 --2- 172.21.15.2:0/3810517245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f331c0caea0 0x7f331c0cb330 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f331c06dce0 tx=0x7f331400b330 comp rx=0 tx=0).stop 2023-08-25T03:34:40.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.037+0000 7f3324086700 1 -- 172.21.15.2:0/3810517245 shutdown_connections 2023-08-25T03:34:40.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.037+0000 7f3324086700 1 --2- 172.21.15.2:0/3810517245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f331c0caea0 0x7f331c0cb330 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:40.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.037+0000 7f3324086700 1 --2- 172.21.15.2:0/3810517245 >> v2:172.21.15.31:3300/0 conn(0x7f331c15e890 0x7f331c071430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:40.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.037+0000 7f3324086700 1 -- 172.21.15.2:0/3810517245 >> 172.21.15.2:0/3810517245 conn(0x7f331c0cfea0 msgr2=0x7f331c0d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:40.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.037+0000 7f3324086700 1 -- 172.21.15.2:0/3810517245 shutdown_connections 2023-08-25T03:34:40.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.037+0000 7f3324086700 1 -- 172.21.15.2:0/3810517245 wait complete. 2023-08-25T03:34:40.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.038+0000 7f3324086700 1 Processor -- start 2023-08-25T03:34:40.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.038+0000 7f3324086700 1 -- start start 2023-08-25T03:34:40.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.039+0000 7f3324086700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f331c0b19a0 0x7f331c154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:40.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.039+0000 7f3324086700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f331c0caea0 0x7f331c15cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:40.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.039+0000 7f3324086700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f331c15e890 0x7f331c15d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:40.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.039+0000 7f3324086700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331c0c8060 con 0x7f331c0b19a0 2023-08-25T03:34:40.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.039+0000 7f3324086700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331c0c5340 con 0x7f331c15e890 2023-08-25T03:34:40.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.039+0000 7f3324086700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f331c0c5620 con 0x7f331c0caea0 2023-08-25T03:34:40.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.039+0000 7f3321e22700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f331c0b19a0 0x7f331c154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:40.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.039+0000 7f3321e22700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f331c0b19a0 0x7f331c154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60384/0 (socket says 172.21.15.2:60384) 2023-08-25T03:34:40.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.039+0000 7f3321e22700 1 -- 172.21.15.2:0/797015800 learned_addr learned my addr 172.21.15.2:0/797015800 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:40.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.039+0000 7f3322623700 1 --2- 172.21.15.2:0/797015800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f331c15e890 0x7f331c15d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:40.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.039+0000 7f3321621700 1 --2- 172.21.15.2:0/797015800 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f331c0caea0 0x7f331c15cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:40.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.040+0000 7f3321e22700 1 -- 172.21.15.2:0/797015800 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f331c0caea0 msgr2=0x7f331c15cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:40.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.040+0000 7f3321e22700 1 --2- 172.21.15.2:0/797015800 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f331c0caea0 0x7f331c15cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:40.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.040+0000 7f3321e22700 1 -- 172.21.15.2:0/797015800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f331c15e890 msgr2=0x7f331c15d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:40.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.040+0000 7f3321e22700 1 --2- 172.21.15.2:0/797015800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f331c15e890 0x7f331c15d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:40.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.040+0000 7f3321e22700 1 -- 172.21.15.2:0/797015800 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3314009a70 con 0x7f331c0b19a0 2023-08-25T03:34:40.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.040+0000 7f3321e22700 1 --2- 172.21.15.2:0/797015800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f331c0b19a0 0x7f331c154960 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3318001eb0 tx=0x7f331800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:40.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.040+0000 7f33127fc700 1 -- 172.21.15.2:0/797015800 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3318005d70 con 0x7f331c0b19a0 2023-08-25T03:34:40.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.040+0000 7f33127fc700 1 -- 172.21.15.2:0/797015800 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33180070d0 con 0x7f331c0b19a0 2023-08-25T03:34:40.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.040+0000 7f33127fc700 1 -- 172.21.15.2:0/797015800 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3318007460 con 0x7f331c0b19a0 2023-08-25T03:34:40.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.040+0000 7f3324086700 1 -- 172.21.15.2:0/797015800 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f331c0c82b0 con 0x7f331c0b19a0 2023-08-25T03:34:40.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.040+0000 7f3324086700 1 -- 172.21.15.2:0/797015800 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f331c15acb0 con 0x7f331c0b19a0 2023-08-25T03:34:40.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.041+0000 7f3324086700 1 -- 172.21.15.2:0/797015800 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f331c066f70 con 0x7f331c0b19a0 2023-08-25T03:34:40.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.042+0000 7f33127fc700 1 -- 172.21.15.2:0/797015800 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f33180075c0 con 0x7f331c0b19a0 2023-08-25T03:34:40.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.042+0000 7f33127fc700 1 --2- 172.21.15.2:0/797015800 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f330006a0b0 0x7f330006c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:40.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.042+0000 7f33127fc700 1 -- 172.21.15.2:0/797015800 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f33180879b0 con 0x7f331c0b19a0 2023-08-25T03:34:40.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.042+0000 7f3321621700 1 --2- 172.21.15.2:0/797015800 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f330006a0b0 0x7f330006c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:40.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.042+0000 7f3321621700 1 --2- 172.21.15.2:0/797015800 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f330006a0b0 0x7f330006c570 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f331c154ea0 tx=0x7f330c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:40.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.047+0000 7f33127fc700 1 -- 172.21.15.2:0/797015800 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f33180530a0 con 0x7f331c0b19a0 2023-08-25T03:34:40.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.336+0000 7f3324086700 1 -- 172.21.15.2:0/797015800 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values"]} v 0) v1 -- 0x7f331c15ae40 con 0x7f331c0b19a0 2023-08-25T03:34:40.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.440+0000 7f33127fc700 1 -- 172.21.15.2:0/797015800 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values"]}]=0 v60) v1 ==== 163+0+0 (secure 0 0 0) 0x7f33180566c0 con 0x7f331c0b19a0 2023-08-25T03:34:40.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.442+0000 7f3324086700 1 -- 172.21.15.2:0/797015800 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f330006a0b0 msgr2=0x7f330006c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:40.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.442+0000 7f3324086700 1 --2- 172.21.15.2:0/797015800 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f330006a0b0 0x7f330006c570 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f331c154ea0 tx=0x7f330c008040 comp rx=0 tx=0).stop 2023-08-25T03:34:40.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.442+0000 7f3324086700 1 -- 172.21.15.2:0/797015800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f331c0b19a0 msgr2=0x7f331c154960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:40.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.442+0000 7f3324086700 1 --2- 172.21.15.2:0/797015800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f331c0b19a0 0x7f331c154960 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f3318001eb0 tx=0x7f331800c7b0 comp rx=0 tx=0).stop 2023-08-25T03:34:40.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.442+0000 7f3324086700 1 -- 172.21.15.2:0/797015800 shutdown_connections 2023-08-25T03:34:40.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.442+0000 7f3324086700 1 --2- 172.21.15.2:0/797015800 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f330006a0b0 0x7f330006c570 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:40.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.442+0000 7f3324086700 1 --2- 172.21.15.2:0/797015800 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f331c0b19a0 0x7f331c154960 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:40.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.442+0000 7f3324086700 1 --2- 172.21.15.2:0/797015800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f331c15e890 0x7f331c15d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:40.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.442+0000 7f3324086700 1 --2- 172.21.15.2:0/797015800 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f331c0caea0 0x7f331c15cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:40.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.442+0000 7f3324086700 1 -- 172.21.15.2:0/797015800 >> 172.21.15.2:0/797015800 conn(0x7f331c0cfea0 msgr2=0x7f331c0b11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:40.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.442+0000 7f3324086700 1 -- 172.21.15.2:0/797015800 shutdown_connections 2023-08-25T03:34:40.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.442+0000 7f3324086700 1 -- 172.21.15.2:0/797015800 wait complete. 2023-08-25T03:34:40.456 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:34:40.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.579+0000 7f86d5b13700 1 Processor -- start 2023-08-25T03:34:40.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.579+0000 7f86d5b13700 1 -- start start 2023-08-25T03:34:40.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.580+0000 7f86d5b13700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f86d0157b40 0x7f86d0161ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:40.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.580+0000 7f86d5b13700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86d00b3a40 con 0x7f86d015da90 2023-08-25T03:34:40.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.580+0000 7f86d5b13700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86d0162020 con 0x7f86d00af660 2023-08-25T03:34:40.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.580+0000 7f86d5b13700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d015eb00 con 0x7f86d0157b40 2023-08-25T03:34:40.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.580+0000 7f86ceffd700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f86d015da90 0x7f86d00aef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47060/0 (socket says 172.21.15.2:47060) 2023-08-25T03:34:40.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.580+0000 7f86ceffd700 1 -- 172.21.15.2:0/2100846259 learned_addr learned my addr 172.21.15.2:0/2100846259 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:40.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.580+0000 7f86cf7fe700 1 --2- 172.21.15.2:0/2100846259 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f86d0157b40 0x7f86d0161ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:40.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.580+0000 7f86ce7fc700 1 -- 172.21.15.2:0/2100846259 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 281890173 0 0) 0x7f86d00b3a40 con 0x7f86d015da90 2023-08-25T03:34:40.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.580+0000 7f86ce7fc700 1 -- 172.21.15.2:0/2100846259 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86b80035b0 con 0x7f86d015da90 2023-08-25T03:34:40.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.580+0000 7f86ce7fc700 1 -- 172.21.15.2:0/2100846259 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330881017 0 0) 0x7f86d0162020 con 0x7f86d00af660 2023-08-25T03:34:40.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.580+0000 7f86ce7fc700 1 -- 172.21.15.2:0/2100846259 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86d00b3a40 con 0x7f86d00af660 2023-08-25T03:34:40.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.580+0000 7f86ce7fc700 1 -- 172.21.15.2:0/2100846259 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 890069519 0 0) 0x7f86b80035b0 con 0x7f86d015da90 2023-08-25T03:34:40.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.580+0000 7f86ce7fc700 1 -- 172.21.15.2:0/2100846259 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f86d0157b40 msgr2=0x7f86d0161ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:40.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.580+0000 7f86ce7fc700 1 --2- 172.21.15.2:0/2100846259 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f86d0157b40 0x7f86d0161ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:40.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.580+0000 7f86ce7fc700 1 -- 172.21.15.2:0/2100846259 >> v1:172.21.15.31:6789/0 conn(0x7f86d00af660 legacy=0x7f86d006d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:40.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.580+0000 7f86ce7fc700 1 -- 172.21.15.2:0/2100846259 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d006a490 con 0x7f86d015da90 2023-08-25T03:34:40.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.580+0000 7f86ce7fc700 1 -- 172.21.15.2:0/2100846259 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f86c4001690 con 0x7f86d015da90 2023-08-25T03:34:40.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.580+0000 7f86ce7fc700 1 -- 172.21.15.2:0/2100846259 >> v1:172.21.15.2:6789/0 conn(0x7f86d015da90 legacy=0x7f86d00aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:40.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.580+0000 7f86ce7fc700 1 --2- 172.21.15.2:0/2100846259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86b8003bc0 0x7f86b8003fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:40.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.580+0000 7f86ce7fc700 1 --2- 172.21.15.2:0/2100846259 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f86d0157b40 0x7f86b8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:40.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.580+0000 7f86ce7fc700 1 --2- 172.21.15.2:0/2100846259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86b8004d90 0x7f86b8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:40.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.580+0000 7f86ce7fc700 1 -- 172.21.15.2:0/2100846259 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d015eb00 con 0x7f86b8003bc0 2023-08-25T03:34:40.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.580+0000 7f86ce7fc700 1 -- 172.21.15.2:0/2100846259 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86b800bac0 con 0x7f86b8004d90 2023-08-25T03:34:40.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.580+0000 7f86ce7fc700 1 -- 172.21.15.2:0/2100846259 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86b800bc80 con 0x7f86d0157b40 2023-08-25T03:34:40.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.581+0000 7f86cf7fe700 1 --2- 172.21.15.2:0/2100846259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86b8003bc0 0x7f86b8003fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:40.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.581+0000 7f86ceffd700 1 --2- 172.21.15.2:0/2100846259 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f86d0157b40 0x7f86b8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:40.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.581+0000 7f86cffff700 1 --2- 172.21.15.2:0/2100846259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86b8004d90 0x7f86b8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:40.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.581+0000 7f86cf7fe700 1 -- 172.21.15.2:0/2100846259 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f86d0157b40 msgr2=0x7f86b8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:40.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.581+0000 7f86cf7fe700 1 --2- 172.21.15.2:0/2100846259 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f86d0157b40 0x7f86b8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:40.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.581+0000 7f86cf7fe700 1 -- 172.21.15.2:0/2100846259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86b8004d90 msgr2=0x7f86b8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:40.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.581+0000 7f86cf7fe700 1 --2- 172.21.15.2:0/2100846259 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86b8004d90 0x7f86b8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:40.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.581+0000 7f86cf7fe700 1 -- 172.21.15.2:0/2100846259 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b80035b0 con 0x7f86b8003bc0 2023-08-25T03:34:40.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.581+0000 7f86ceffd700 1 --2- 172.21.15.2:0/2100846259 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f86d0157b40 0x7f86b8004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:40.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.581+0000 7f86cf7fe700 1 --2- 172.21.15.2:0/2100846259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86b8003bc0 0x7f86b8003fc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009b60 tx=0x7f86b8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c59254eafdc42814 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:40.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.581+0000 7f86ce7fc700 1 -- 172.21.15.2:0/2100846259 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c000e6c0 con 0x7f86b8003bc0 2023-08-25T03:34:40.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.581+0000 7f86ce7fc700 1 -- 172.21.15.2:0/2100846259 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c000e820 con 0x7f86b8003bc0 2023-08-25T03:34:40.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.581+0000 7f86ce7fc700 1 -- 172.21.15.2:0/2100846259 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c000eb90 con 0x7f86b8003bc0 2023-08-25T03:34:40.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.582+0000 7f86d5b13700 1 -- 172.21.15.2:0/2100846259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86b8003bc0 msgr2=0x7f86b8003fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:40.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.582+0000 7f86d5b13700 1 --2- 172.21.15.2:0/2100846259 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86b8003bc0 0x7f86b8003fc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009b60 tx=0x7f86b8005880 comp rx=0 tx=0).stop 2023-08-25T03:34:40.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.582+0000 7f86cffff700 1 -- 172.21.15.2:0/2100846259 reap_dead start 2023-08-25T03:34:40.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.582+0000 7f86d5b13700 1 -- 172.21.15.2:0/2100846259 shutdown_connections 2023-08-25T03:34:40.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.582+0000 7f86d5b13700 1 -- 172.21.15.2:0/2100846259 >> 172.21.15.2:0/2100846259 conn(0x7f86d00cfea0 msgr2=0x7f86d00d02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:40.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.582+0000 7f86d5b13700 1 -- 172.21.15.2:0/2100846259 shutdown_connections 2023-08-25T03:34:40.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.582+0000 7f86d5b13700 1 -- 172.21.15.2:0/2100846259 wait complete. 2023-08-25T03:34:40.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.583+0000 7f86d5b13700 1 Processor -- start 2023-08-25T03:34:40.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.584+0000 7f86d5b13700 1 -- start start 2023-08-25T03:34:40.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.584+0000 7f86d5b13700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86d00ce440 0x7f86d00cf280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:40.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.584+0000 7f86d5b13700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86d00ce820 0x7f86d01f3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:40.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.584+0000 7f86d5b13700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f86d015ff70 0x7f86d0160400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:40.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.585+0000 7f86d5b13700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d0160940 con 0x7f86d00ce440 2023-08-25T03:34:40.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.585+0000 7f86d5b13700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d01f3ab0 con 0x7f86d00ce820 2023-08-25T03:34:40.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.585+0000 7f86d5b13700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d01f6080 con 0x7f86d015ff70 2023-08-25T03:34:40.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.585+0000 7f86cf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86d00ce440 0x7f86d00cf280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:40.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.585+0000 7f86cf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86d00ce440 0x7f86d00cf280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60404/0 (socket says 172.21.15.2:60404) 2023-08-25T03:34:40.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.585+0000 7f86cf7fe700 1 -- 172.21.15.2:0/690779560 learned_addr learned my addr 172.21.15.2:0/690779560 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.585+0000 7f86ceffd700 1 --2- 172.21.15.2:0/690779560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86d00ce820 0x7f86d01f3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.585+0000 7f86cffff700 1 --2- 172.21.15.2:0/690779560 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f86d015ff70 0x7f86d0160400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.585+0000 7f86cf7fe700 1 -- 172.21.15.2:0/690779560 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f86d015ff70 msgr2=0x7f86d0160400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.585+0000 7f86cf7fe700 1 --2- 172.21.15.2:0/690779560 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f86d015ff70 0x7f86d0160400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.585+0000 7f86cf7fe700 1 -- 172.21.15.2:0/690779560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86d00ce820 msgr2=0x7f86d01f3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.585+0000 7f86cf7fe700 1 --2- 172.21.15.2:0/690779560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86d00ce820 0x7f86d01f3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.585+0000 7f86cf7fe700 1 -- 172.21.15.2:0/690779560 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d01f6c00 con 0x7f86d00ce440 2023-08-25T03:34:40.594 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.585+0000 7f86cf7fe700 1 --2- 172.21.15.2:0/690779560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86d00ce440 0x7f86d00cf280 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f86b8005830 tx=0x7f86c00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:40.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.585+0000 7f86b7fff700 1 -- 172.21.15.2:0/690779560 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c000e6c0 con 0x7f86d00ce440 2023-08-25T03:34:40.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.585+0000 7f86b7fff700 1 -- 172.21.15.2:0/690779560 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c00038c0 con 0x7f86d00ce440 2023-08-25T03:34:40.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.585+0000 7f86d5b13700 1 -- 172.21.15.2:0/690779560 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d01f62d0 con 0x7f86d00ce440 2023-08-25T03:34:40.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.585+0000 7f86b7fff700 1 -- 172.21.15.2:0/690779560 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c0003c70 con 0x7f86d00ce440 2023-08-25T03:34:40.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.585+0000 7f86d5b13700 1 -- 172.21.15.2:0/690779560 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d01f6890 con 0x7f86d00ce440 2023-08-25T03:34:40.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.586+0000 7f86d5b13700 1 -- 172.21.15.2:0/690779560 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86d0066f70 con 0x7f86d00ce440 2023-08-25T03:34:40.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.587+0000 7f86b7fff700 1 -- 172.21.15.2:0/690779560 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f86c0013470 con 0x7f86d00ce440 2023-08-25T03:34:40.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.587+0000 7f86b7fff700 1 --2- 172.21.15.2:0/690779560 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f86bc06a0e0 0x7f86bc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:40.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.587+0000 7f86b7fff700 1 -- 172.21.15.2:0/690779560 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f86c0089d00 con 0x7f86d00ce440 2023-08-25T03:34:40.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.587+0000 7f86ceffd700 1 --2- 172.21.15.2:0/690779560 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f86bc06a0e0 0x7f86bc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:40.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.587+0000 7f86ceffd700 1 --2- 172.21.15.2:0/690779560 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f86bc06a0e0 0x7f86bc06c5a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f86d00ceee0 tx=0x7f86c4007040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:40.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.591+0000 7f86b7fff700 1 -- 172.21.15.2:0/690779560 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f86c0055480 con 0x7f86d00ce440 2023-08-25T03:34:40.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.793+0000 7f86d5b13700 1 -- 172.21.15.2:0/690779560 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86d006aa00 con 0x7f86d00ce440 2023-08-25T03:34:40.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.794+0000 7f86b7fff700 1 -- 172.21.15.2:0/690779560 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18135 (secure 0 0 0) 0x7f86c0017090 con 0x7f86d00ce440 2023-08-25T03:34:40.796 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:40.796 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":24,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:34:04.604059+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:34:00.869118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T03:34:01.987016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:34:40.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.796+0000 7f86d5b13700 1 -- 172.21.15.2:0/690779560 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f86bc06a0e0 msgr2=0x7f86bc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:40.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.796+0000 7f86d5b13700 1 --2- 172.21.15.2:0/690779560 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f86bc06a0e0 0x7f86bc06c5a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f86d00ceee0 tx=0x7f86c4007040 comp rx=0 tx=0).stop 2023-08-25T03:34:40.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.796+0000 7f86d5b13700 1 -- 172.21.15.2:0/690779560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86d00ce440 msgr2=0x7f86d00cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:40.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.796+0000 7f86d5b13700 1 --2- 172.21.15.2:0/690779560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86d00ce440 0x7f86d00cf280 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f86b8005830 tx=0x7f86c00047c0 comp rx=0 tx=0).stop 2023-08-25T03:34:40.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.796+0000 7f86d5b13700 1 -- 172.21.15.2:0/690779560 shutdown_connections 2023-08-25T03:34:40.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.796+0000 7f86d5b13700 1 --2- 172.21.15.2:0/690779560 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f86bc06a0e0 0x7f86bc06c5a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:40.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.796+0000 7f86d5b13700 1 --2- 172.21.15.2:0/690779560 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f86d00ce440 0x7f86d00cf280 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:40.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.796+0000 7f86d5b13700 1 --2- 172.21.15.2:0/690779560 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f86d015ff70 0x7f86d0160400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:40.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.796+0000 7f86d5b13700 1 --2- 172.21.15.2:0/690779560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f86d00ce820 0x7f86d01f3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:40.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.796+0000 7f86d5b13700 1 -- 172.21.15.2:0/690779560 >> 172.21.15.2:0/690779560 conn(0x7f86d00cfea0 msgr2=0x7f86d00aee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:40.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.796+0000 7f86d5b13700 1 -- 172.21.15.2:0/690779560 shutdown_connections 2023-08-25T03:34:40.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.797+0000 7f86d5b13700 1 -- 172.21.15.2:0/690779560 wait complete. 2023-08-25T03:34:40.810 INFO:tasks.cephfs.cephfs_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T03:33:49.560643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T03:33:56.265184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T03:34:00.869118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T03:34:01.987016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T03:34:40.811 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T03:34:40.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.937+0000 7feae926f700 1 Processor -- start 2023-08-25T03:34:40.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.937+0000 7feae926f700 1 -- start start 2023-08-25T03:34:40.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.938+0000 7feae926f700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feae406a8f0 0x7feae4070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:40.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.938+0000 7feae926f700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feae4160bd0 con 0x7feae41607f0 2023-08-25T03:34:40.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.938+0000 7feae926f700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feae40cdd00 con 0x7feae40af180 2023-08-25T03:34:40.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.938+0000 7feae926f700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae40713a0 con 0x7feae406a8f0 2023-08-25T03:34:40.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.938+0000 7feae2ffd700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7feae41607f0 0x7feae40ac8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47078/0 (socket says 172.21.15.2:47078) 2023-08-25T03:34:40.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.938+0000 7feae2ffd700 1 -- 172.21.15.2:0/3668582248 learned_addr learned my addr 172.21.15.2:0/3668582248 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:40.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.939+0000 7feae37fe700 1 --2- 172.21.15.2:0/3668582248 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feae406a8f0 0x7feae4070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:40.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.939+0000 7feae1ffb700 1 -- 172.21.15.2:0/3668582248 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2381887838 0 0) 0x7feae4160bd0 con 0x7feae41607f0 2023-08-25T03:34:40.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.939+0000 7feae1ffb700 1 -- 172.21.15.2:0/3668582248 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feacc0035b0 con 0x7feae41607f0 2023-08-25T03:34:40.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.939+0000 7feae1ffb700 1 -- 172.21.15.2:0/3668582248 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2749088962 0 0) 0x7feae40cdd00 con 0x7feae40af180 2023-08-25T03:34:40.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.939+0000 7feae1ffb700 1 -- 172.21.15.2:0/3668582248 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feae4160bd0 con 0x7feae40af180 2023-08-25T03:34:40.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.939+0000 7feae37fe700 1 -- 172.21.15.2:0/3668582248 >> v1:172.21.15.31:6789/0 conn(0x7feae40af180 legacy=0x7feae406a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:40.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.939+0000 7feae37fe700 1 -- 172.21.15.2:0/3668582248 >> v1:172.21.15.2:6789/0 conn(0x7feae41607f0 legacy=0x7feae40ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:40.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.939+0000 7feae37fe700 1 -- 172.21.15.2:0/3668582248 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae40cda00 con 0x7feae406a8f0 2023-08-25T03:34:40.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.939+0000 7feae37fe700 1 --2- 172.21.15.2:0/3668582248 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feae406a8f0 0x7feae4070e60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feadc009dc0 tx=0x7feadc00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0f7b7fd8ac53e34 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:40.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.939+0000 7feae1ffb700 1 -- 172.21.15.2:0/3668582248 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc00e070 con 0x7feae406a8f0 2023-08-25T03:34:40.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.939+0000 7feae1ffb700 1 -- 172.21.15.2:0/3668582248 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc00bb20 con 0x7feae406a8f0 2023-08-25T03:34:40.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.939+0000 7feae1ffb700 1 -- 172.21.15.2:0/3668582248 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc012430 con 0x7feae406a8f0 2023-08-25T03:34:40.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.940+0000 7feae926f700 1 -- 172.21.15.2:0/3668582248 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feae406a8f0 msgr2=0x7feae4070e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:40.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.940+0000 7feae926f700 1 --2- 172.21.15.2:0/3668582248 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feae406a8f0 0x7feae4070e60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7feadc009dc0 tx=0x7feadc00b330 comp rx=0 tx=0).stop 2023-08-25T03:34:40.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.940+0000 7feae926f700 1 -- 172.21.15.2:0/3668582248 shutdown_connections 2023-08-25T03:34:40.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.940+0000 7feae926f700 1 --2- 172.21.15.2:0/3668582248 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feae406a8f0 0x7feae4070e60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:40.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.940+0000 7feae926f700 1 -- 172.21.15.2:0/3668582248 >> 172.21.15.2:0/3668582248 conn(0x7feae40cfd90 msgr2=0x7feae40d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:40.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.940+0000 7feae926f700 1 -- 172.21.15.2:0/3668582248 shutdown_connections 2023-08-25T03:34:40.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.940+0000 7feae926f700 1 -- 172.21.15.2:0/3668582248 wait complete. 2023-08-25T03:34:40.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.941+0000 7feae926f700 1 Processor -- start 2023-08-25T03:34:40.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.941+0000 7feae926f700 1 -- start start 2023-08-25T03:34:40.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.942+0000 7feae926f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feae406a8f0 0x7feae415a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:40.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.942+0000 7feae926f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feae40af180 0x7feae40cac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:40.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.942+0000 7feae926f700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feae41607f0 0x7feae40b0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:40.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.942+0000 7feae926f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae40cc060 con 0x7feae406a8f0 2023-08-25T03:34:40.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.942+0000 7feae926f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae40c62a0 con 0x7feae40af180 2023-08-25T03:34:40.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.942+0000 7feae926f700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae40c5200 con 0x7feae41607f0 2023-08-25T03:34:40.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.942+0000 7feae2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feae406a8f0 0x7feae415a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:40.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.942+0000 7feae2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feae406a8f0 0x7feae415a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60418/0 (socket says 172.21.15.2:60418) 2023-08-25T03:34:40.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.942+0000 7feae2ffd700 1 -- 172.21.15.2:0/402354153 learned_addr learned my addr 172.21.15.2:0/402354153 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:40.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.942+0000 7feae37fe700 1 --2- 172.21.15.2:0/402354153 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feae41607f0 0x7feae40b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:40.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.942+0000 7feae27fc700 1 --2- 172.21.15.2:0/402354153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feae40af180 0x7feae40cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:40.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.943+0000 7feae2ffd700 1 -- 172.21.15.2:0/402354153 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feae41607f0 msgr2=0x7feae40b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:40.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.943+0000 7feae2ffd700 1 --2- 172.21.15.2:0/402354153 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feae41607f0 0x7feae40b0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:40.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.943+0000 7feae2ffd700 1 -- 172.21.15.2:0/402354153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feae40af180 msgr2=0x7feae40cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:40.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.943+0000 7feae2ffd700 1 --2- 172.21.15.2:0/402354153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feae40af180 0x7feae40cac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:40.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.943+0000 7feae2ffd700 1 -- 172.21.15.2:0/402354153 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feadc009a70 con 0x7feae406a8f0 2023-08-25T03:34:40.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.943+0000 7feae2ffd700 1 --2- 172.21.15.2:0/402354153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feae406a8f0 0x7feae415a1a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fead8001eb0 tx=0x7fead800c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:40.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.943+0000 7feacb7fe700 1 -- 172.21.15.2:0/402354153 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead8005d70 con 0x7feae406a8f0 2023-08-25T03:34:40.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.943+0000 7feacb7fe700 1 -- 172.21.15.2:0/402354153 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead80070d0 con 0x7feae406a8f0 2023-08-25T03:34:40.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.943+0000 7feacb7fe700 1 -- 172.21.15.2:0/402354153 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead8007440 con 0x7feae406a8f0 2023-08-25T03:34:40.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.943+0000 7feae926f700 1 -- 172.21.15.2:0/402354153 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae40c54e0 con 0x7feae406a8f0 2023-08-25T03:34:40.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.943+0000 7feae926f700 1 -- 172.21.15.2:0/402354153 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae40c5aa0 con 0x7feae406a8f0 2023-08-25T03:34:40.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.944+0000 7feae926f700 1 -- 172.21.15.2:0/402354153 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae40746d0 con 0x7feae406a8f0 2023-08-25T03:34:40.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.945+0000 7feacb7fe700 1 -- 172.21.15.2:0/402354153 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fead80075a0 con 0x7feae406a8f0 2023-08-25T03:34:40.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.945+0000 7feacb7fe700 1 --2- 172.21.15.2:0/402354153 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fead006a0e0 0x7fead006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:40.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.945+0000 7feacb7fe700 1 -- 172.21.15.2:0/402354153 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fead8087750 con 0x7feae406a8f0 2023-08-25T03:34:40.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.945+0000 7feae27fc700 1 --2- 172.21.15.2:0/402354153 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fead006a0e0 0x7fead006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:40.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.945+0000 7feae27fc700 1 --2- 172.21.15.2:0/402354153 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fead006a0e0 0x7fead006c5a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7feae40c9670 tx=0x7fead4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:40.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:40.949+0000 7feacb7fe700 1 -- 172.21.15.2:0/402354153 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fead8052f80 con 0x7feae406a8f0 2023-08-25T03:34:41.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.150+0000 7feae926f700 1 -- 172.21.15.2:0/402354153 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feae40ae800 con 0x7feae406a8f0 2023-08-25T03:34:41.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.151+0000 7feacb7fe700 1 -- 172.21.15.2:0/402354153 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fead80565a0 con 0x7feae406a8f0 2023-08-25T03:34:41.153 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-08-25T03:34:41.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.153+0000 7feae926f700 1 -- 172.21.15.2:0/402354153 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fead006a0e0 msgr2=0x7fead006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:41.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.153+0000 7feae926f700 1 --2- 172.21.15.2:0/402354153 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fead006a0e0 0x7fead006c5a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7feae40c9670 tx=0x7fead4009040 comp rx=0 tx=0).stop 2023-08-25T03:34:41.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.153+0000 7feae926f700 1 -- 172.21.15.2:0/402354153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feae406a8f0 msgr2=0x7feae415a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:41.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.153+0000 7feae926f700 1 --2- 172.21.15.2:0/402354153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feae406a8f0 0x7feae415a1a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fead8001eb0 tx=0x7fead800c7b0 comp rx=0 tx=0).stop 2023-08-25T03:34:41.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.153+0000 7feae926f700 1 -- 172.21.15.2:0/402354153 shutdown_connections 2023-08-25T03:34:41.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.153+0000 7feae926f700 1 --2- 172.21.15.2:0/402354153 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fead006a0e0 0x7fead006c5a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:41.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.153+0000 7feae926f700 1 --2- 172.21.15.2:0/402354153 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feae406a8f0 0x7feae415a1a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:41.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.153+0000 7feae926f700 1 --2- 172.21.15.2:0/402354153 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feae41607f0 0x7feae40b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:41.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.153+0000 7feae926f700 1 --2- 172.21.15.2:0/402354153 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feae40af180 0x7feae40cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:41.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.153+0000 7feae926f700 1 -- 172.21.15.2:0/402354153 >> 172.21.15.2:0/402354153 conn(0x7feae40cfd90 msgr2=0x7feae40cccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:41.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.153+0000 7feae926f700 1 -- 172.21.15.2:0/402354153 shutdown_connections 2023-08-25T03:34:41.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.153+0000 7feae926f700 1 -- 172.21.15.2:0/402354153 wait complete. 2023-08-25T03:34:41.167 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T03:34:41.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.293+0000 7fbe0180f700 1 Processor -- start 2023-08-25T03:34:41.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.294+0000 7fbe0180f700 1 -- start start 2023-08-25T03:34:41.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.294+0000 7fbe0180f700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbdfc1607b0 0x7fbdfc0cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:41.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.294+0000 7fbe0180f700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fbdfc0d2b20 0x7fbdfc0ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:41.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.294+0000 7fbe0180f700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdfc160b90 con 0x7fbdfc0d1510 2023-08-25T03:34:41.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.294+0000 7fbe0180f700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdfc0d1a80 con 0x7fbdfc0d2b20 2023-08-25T03:34:41.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.294+0000 7fbe0180f700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc0d1d60 con 0x7fbdfc1607b0 2023-08-25T03:34:41.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.294+0000 7fbdfb7fe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fbdfc0d2b20 0x7fbdfc0ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:41.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.294+0000 7fbdfb7fe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fbdfc0d2b20 0x7fbdfc0ce300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60426/0 (socket says 172.21.15.2:60426) 2023-08-25T03:34:41.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.294+0000 7fbdfb7fe700 1 -- 172.21.15.2:0/2540000052 learned_addr learned my addr 172.21.15.2:0/2540000052 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:41.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.295+0000 7fbdfaffd700 1 --2- 172.21.15.2:0/2540000052 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbdfc1607b0 0x7fbdfc0cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:41.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.295+0000 7fbdfb7fe700 1 -- 172.21.15.2:0/2540000052 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbdfc1607b0 msgr2=0x7fbdfc0cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:41.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.295+0000 7fbdfb7fe700 1 --2- 172.21.15.2:0/2540000052 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbdfc1607b0 0x7fbdfc0cddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:41.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.295+0000 7fbdfb7fe700 1 -- 172.21.15.2:0/2540000052 >> v1:172.21.15.31:6789/0 conn(0x7fbdfc0d1510 legacy=0x7fbdfc06c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:41.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.295+0000 7fbdfb7fe700 1 -- 172.21.15.2:0/2540000052 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdfc0d2040 con 0x7fbdfc0d2b20 2023-08-25T03:34:41.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.295+0000 7fbdfb7fe700 1 --2- 172.21.15.2:0/2540000052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdfc0d2b20 0x7fbdfc0ce300 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4009e50 tx=0x7fbdf400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92968729e6aff20c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:41.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.295+0000 7fbdf9ffb700 1 -- 172.21.15.2:0/2540000052 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf400e070 con 0x7fbdfc0d2b20 2023-08-25T03:34:41.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.295+0000 7fbdf9ffb700 1 -- 172.21.15.2:0/2540000052 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf400b910 con 0x7fbdfc0d2b20 2023-08-25T03:34:41.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.295+0000 7fbdf9ffb700 1 -- 172.21.15.2:0/2540000052 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf400bcc0 con 0x7fbdfc0d2b20 2023-08-25T03:34:41.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.296+0000 7fbe0180f700 1 -- 172.21.15.2:0/2540000052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdfc0d2b20 msgr2=0x7fbdfc0ce300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:41.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.296+0000 7fbe0180f700 1 --2- 172.21.15.2:0/2540000052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdfc0d2b20 0x7fbdfc0ce300 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4009e50 tx=0x7fbdf400b330 comp rx=0 tx=0).stop 2023-08-25T03:34:41.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.296+0000 7fbe0180f700 1 -- 172.21.15.2:0/2540000052 shutdown_connections 2023-08-25T03:34:41.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.296+0000 7fbe0180f700 1 --2- 172.21.15.2:0/2540000052 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdfc0d2b20 0x7fbdfc0ce300 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:41.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.296+0000 7fbe0180f700 1 --2- 172.21.15.2:0/2540000052 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbdfc1607b0 0x7fbdfc0cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:41.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.296+0000 7fbe0180f700 1 -- 172.21.15.2:0/2540000052 >> 172.21.15.2:0/2540000052 conn(0x7fbdfc0cfb70 msgr2=0x7fbdfc0cff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:41.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.296+0000 7fbe0180f700 1 -- 172.21.15.2:0/2540000052 shutdown_connections 2023-08-25T03:34:41.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.296+0000 7fbe0180f700 1 -- 172.21.15.2:0/2540000052 wait complete. 2023-08-25T03:34:41.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.297+0000 7fbe0180f700 1 Processor -- start 2023-08-25T03:34:41.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.297+0000 7fbe0180f700 1 -- start start 2023-08-25T03:34:41.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.298+0000 7fbe0180f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdfc0d1510 0x7fbdfc15eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:41.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.298+0000 7fbe0180f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdfc0d2b20 0x7fbdfc0c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:41.305 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.298+0000 7fbe0180f700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbdfc1607b0 0x7fbdfc1f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:41.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.298+0000 7fbe0180f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc0c89d0 con 0x7fbdfc0d2b20 2023-08-25T03:34:41.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.298+0000 7fbe0180f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc0c7dc0 con 0x7fbdfc0d1510 2023-08-25T03:34:41.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.298+0000 7fbe0180f700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdfc0c80a0 con 0x7fbdfc1607b0 2023-08-25T03:34:41.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.298+0000 7fbdfa7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdfc0d2b20 0x7fbdfc0c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:41.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.298+0000 7fbdfa7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdfc0d2b20 0x7fbdfc0c8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60434/0 (socket says 172.21.15.2:60434) 2023-08-25T03:34:41.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.298+0000 7fbdfa7fc700 1 -- 172.21.15.2:0/2862126092 learned_addr learned my addr 172.21.15.2:0/2862126092 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:41.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.298+0000 7fbdfaffd700 1 --2- 172.21.15.2:0/2862126092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdfc0d1510 0x7fbdfc15eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:41.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.298+0000 7fbdfb7fe700 1 --2- 172.21.15.2:0/2862126092 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbdfc1607b0 0x7fbdfc1f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:41.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.298+0000 7fbdfa7fc700 1 -- 172.21.15.2:0/2862126092 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbdfc1607b0 msgr2=0x7fbdfc1f38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:41.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.298+0000 7fbdfa7fc700 1 --2- 172.21.15.2:0/2862126092 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbdfc1607b0 0x7fbdfc1f38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:41.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.298+0000 7fbdfa7fc700 1 -- 172.21.15.2:0/2862126092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdfc0d1510 msgr2=0x7fbdfc15eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:41.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.298+0000 7fbdfa7fc700 1 --2- 172.21.15.2:0/2862126092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdfc0d1510 0x7fbdfc15eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:41.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.298+0000 7fbdfa7fc700 1 -- 172.21.15.2:0/2862126092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdf4009b30 con 0x7fbdfc0d2b20 2023-08-25T03:34:41.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.298+0000 7fbdfa7fc700 1 --2- 172.21.15.2:0/2862126092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdfc0d2b20 0x7fbdfc0c8e60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbdec00c4e0 tx=0x7fbdec00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:41.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.299+0000 7fbde3fff700 1 -- 172.21.15.2:0/2862126092 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec0052d0 con 0x7fbdfc0d2b20 2023-08-25T03:34:41.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.299+0000 7fbde3fff700 1 -- 172.21.15.2:0/2862126092 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdec005430 con 0x7fbdfc0d2b20 2023-08-25T03:34:41.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.299+0000 7fbe0180f700 1 -- 172.21.15.2:0/2862126092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdfc0c8380 con 0x7fbdfc0d2b20 2023-08-25T03:34:41.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.299+0000 7fbde3fff700 1 -- 172.21.15.2:0/2862126092 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdec0038c0 con 0x7fbdfc0d2b20 2023-08-25T03:34:41.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.299+0000 7fbe0180f700 1 -- 172.21.15.2:0/2862126092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdfc15f410 con 0x7fbdfc0d2b20 2023-08-25T03:34:41.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.300+0000 7fbe0180f700 1 -- 172.21.15.2:0/2862126092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdfc15f600 con 0x7fbdfc0d2b20 2023-08-25T03:34:41.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.300+0000 7fbde3fff700 1 -- 172.21.15.2:0/2862126092 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbdec003a20 con 0x7fbdfc0d2b20 2023-08-25T03:34:41.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.300+0000 7fbde3fff700 1 --2- 172.21.15.2:0/2862126092 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fbde806a0e0 0x7fbde806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:41.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.301+0000 7fbde3fff700 1 -- 172.21.15.2:0/2862126092 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fbdec0878d0 con 0x7fbdfc0d2b20 2023-08-25T03:34:41.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.301+0000 7fbdfaffd700 1 --2- 172.21.15.2:0/2862126092 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fbde806a0e0 0x7fbde806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:41.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.301+0000 7fbdfaffd700 1 --2- 172.21.15.2:0/2862126092 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fbde806a0e0 0x7fbde806c5a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0b82e0 tx=0x7fbdf0009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:41.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.305+0000 7fbde3fff700 1 -- 172.21.15.2:0/2862126092 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbdec052fc0 con 0x7fbdfc0d2b20 2023-08-25T03:34:41.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.507+0000 7fbe0180f700 1 -- 172.21.15.2:0/2862126092 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbdfc0746d0 con 0x7fbdfc0d2b20 2023-08-25T03:34:41.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.507+0000 7fbde3fff700 1 -- 172.21.15.2:0/2862126092 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v24) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbdec017020 con 0x7fbdfc0d2b20 2023-08-25T03:34:41.510 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 8 2023-08-25T03:34:41.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.510+0000 7fbe0180f700 1 -- 172.21.15.2:0/2862126092 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fbde806a0e0 msgr2=0x7fbde806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:41.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.510+0000 7fbe0180f700 1 --2- 172.21.15.2:0/2862126092 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fbde806a0e0 0x7fbde806c5a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc0b82e0 tx=0x7fbdf0009210 comp rx=0 tx=0).stop 2023-08-25T03:34:41.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.510+0000 7fbe0180f700 1 -- 172.21.15.2:0/2862126092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdfc0d2b20 msgr2=0x7fbdfc0c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:41.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.510+0000 7fbe0180f700 1 --2- 172.21.15.2:0/2862126092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdfc0d2b20 0x7fbdfc0c8e60 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbdec00c4e0 tx=0x7fbdec00c7f0 comp rx=0 tx=0).stop 2023-08-25T03:34:41.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.510+0000 7fbe0180f700 1 -- 172.21.15.2:0/2862126092 shutdown_connections 2023-08-25T03:34:41.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.510+0000 7fbe0180f700 1 --2- 172.21.15.2:0/2862126092 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbdfc1607b0 0x7fbdfc1f38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:41.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.510+0000 7fbe0180f700 1 --2- 172.21.15.2:0/2862126092 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbdfc0d1510 0x7fbdfc15eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:41.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.510+0000 7fbe0180f700 1 --2- 172.21.15.2:0/2862126092 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fbde806a0e0 0x7fbde806c5a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:41.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.510+0000 7fbe0180f700 1 --2- 172.21.15.2:0/2862126092 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbdfc0d2b20 0x7fbdfc0c8e60 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:41.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.510+0000 7fbe0180f700 1 -- 172.21.15.2:0/2862126092 >> 172.21.15.2:0/2862126092 conn(0x7fbdfc0cfb70 msgr2=0x7fbdfc153410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:41.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.510+0000 7fbe0180f700 1 -- 172.21.15.2:0/2862126092 shutdown_connections 2023-08-25T03:34:41.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.510+0000 7fbe0180f700 1 -- 172.21.15.2:0/2862126092 wait complete. 2023-08-25T03:34:41.524 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T03:34:41.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.650+0000 7fd9f1380700 1 Processor -- start 2023-08-25T03:34:41.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.650+0000 7fd9f1380700 1 -- start start 2023-08-25T03:34:41.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.651+0000 7fd9f1380700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd9ec1607b0 0x7fd9ec0cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:41.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.651+0000 7fd9f1380700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd9ec0d1510 0x7fd9ec0ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:41.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.651+0000 7fd9f1380700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9ec160b90 con 0x7fd9ec0d2b20 2023-08-25T03:34:41.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.651+0000 7fd9f1380700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9ec0d1a80 con 0x7fd9ec0d1510 2023-08-25T03:34:41.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.651+0000 7fd9f1380700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec0d1d60 con 0x7fd9ec1607b0 2023-08-25T03:34:41.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.651+0000 7fd9eb7fe700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fd9ec0d2b20 0x7fd9ec06c260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47108/0 (socket says 172.21.15.2:47108) 2023-08-25T03:34:41.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.651+0000 7fd9eb7fe700 1 -- 172.21.15.2:0/382764571 learned_addr learned my addr 172.21.15.2:0/382764571 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:41.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.651+0000 7fd9ea7fc700 1 --2- 172.21.15.2:0/382764571 >> v2:172.21.15.31:3300/0 conn(0x7fd9ec0d1510 0x7fd9ec0ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:41.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.651+0000 7fd9eaffd700 1 --2- 172.21.15.2:0/382764571 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd9ec1607b0 0x7fd9ec0cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:41.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.651+0000 7fd9e9ffb700 1 -- 172.21.15.2:0/382764571 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1517281651 0 0) 0x7fd9ec160b90 con 0x7fd9ec0d2b20 2023-08-25T03:34:41.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.652+0000 7fd9e9ffb700 1 -- 172.21.15.2:0/382764571 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9d40035b0 con 0x7fd9ec0d2b20 2023-08-25T03:34:41.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.652+0000 7fd9ea7fc700 1 -- 172.21.15.2:0/382764571 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd9ec1607b0 msgr2=0x7fd9ec0cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:41.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.652+0000 7fd9ea7fc700 1 --2- 172.21.15.2:0/382764571 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd9ec1607b0 0x7fd9ec0cddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:41.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.652+0000 7fd9ea7fc700 1 -- 172.21.15.2:0/382764571 >> v1:172.21.15.2:6789/0 conn(0x7fd9ec0d2b20 legacy=0x7fd9ec06c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:41.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.652+0000 7fd9e9ffb700 1 -- 172.21.15.2:0/382764571 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3377404592 0 0) 0x7fd9d40035b0 con 0x7fd9ec0d2b20 2023-08-25T03:34:41.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.652+0000 7fd9ea7fc700 1 -- 172.21.15.2:0/382764571 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ec0d2040 con 0x7fd9ec0d1510 2023-08-25T03:34:41.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.652+0000 7fd9eaffd700 1 --2- 172.21.15.2:0/382764571 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd9ec1607b0 0x7fd9ec0cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:34:41.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.652+0000 7fd9ea7fc700 1 --2- 172.21.15.2:0/382764571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9ec0d1510 0x7fd9ec0ce300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc009840 tx=0x7fd9dc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f717c5ee9745b6e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:41.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.652+0000 7fd9e9ffb700 1 -- 172.21.15.2:0/382764571 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e40019f0 con 0x7fd9ec0d1510 2023-08-25T03:34:41.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.652+0000 7fd9e9ffb700 1 -- 172.21.15.2:0/382764571 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9dc004030 con 0x7fd9ec0d1510 2023-08-25T03:34:41.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.652+0000 7fd9e9ffb700 1 -- 172.21.15.2:0/382764571 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc0043c0 con 0x7fd9ec0d1510 2023-08-25T03:34:41.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.653+0000 7fd9f1380700 1 -- 172.21.15.2:0/382764571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9ec0d1510 msgr2=0x7fd9ec0ce300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:41.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.653+0000 7fd9f1380700 1 --2- 172.21.15.2:0/382764571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9ec0d1510 0x7fd9ec0ce300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc009840 tx=0x7fd9dc009b50 comp rx=0 tx=0).stop 2023-08-25T03:34:41.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.653+0000 7fd9f1380700 1 -- 172.21.15.2:0/382764571 shutdown_connections 2023-08-25T03:34:41.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.653+0000 7fd9f1380700 1 --2- 172.21.15.2:0/382764571 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd9ec1607b0 0x7fd9ec0cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:41.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.653+0000 7fd9f1380700 1 --2- 172.21.15.2:0/382764571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9ec0d1510 0x7fd9ec0ce300 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:41.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.653+0000 7fd9f1380700 1 -- 172.21.15.2:0/382764571 >> 172.21.15.2:0/382764571 conn(0x7fd9ec0cfb70 msgr2=0x7fd9ec0cff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:41.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.653+0000 7fd9f1380700 1 -- 172.21.15.2:0/382764571 shutdown_connections 2023-08-25T03:34:41.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.653+0000 7fd9f1380700 1 -- 172.21.15.2:0/382764571 wait complete. 2023-08-25T03:34:41.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.654+0000 7fd9f1380700 1 Processor -- start 2023-08-25T03:34:41.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.654+0000 7fd9f1380700 1 -- start start 2023-08-25T03:34:41.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.655+0000 7fd9f1380700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9ec0d1510 0x7fd9ec0b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:41.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.655+0000 7fd9f1380700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd9ec0d2b20 0x7fd9ec0b3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:41.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.655+0000 7fd9f1380700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9ec1607b0 0x7fd9ec1f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:41.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.655+0000 7fd9f1380700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec15b860 con 0x7fd9ec0d1510 2023-08-25T03:34:41.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.655+0000 7fd9f1380700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec15bb40 con 0x7fd9ec1607b0 2023-08-25T03:34:41.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.655+0000 7fd9f1380700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec15be20 con 0x7fd9ec0d2b20 2023-08-25T03:34:41.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.655+0000 7fd9eaffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9ec0d1510 0x7fd9ec0b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:41.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.655+0000 7fd9eaffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9ec0d1510 0x7fd9ec0b54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60444/0 (socket says 172.21.15.2:60444) 2023-08-25T03:34:41.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.655+0000 7fd9eaffd700 1 -- 172.21.15.2:0/3042606718 learned_addr learned my addr 172.21.15.2:0/3042606718 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:41.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.655+0000 7fd9eb7fe700 1 --2- 172.21.15.2:0/3042606718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9ec1607b0 0x7fd9ec1f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:41.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.655+0000 7fd9ea7fc700 1 --2- 172.21.15.2:0/3042606718 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd9ec0d2b20 0x7fd9ec0b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:41.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.655+0000 7fd9eaffd700 1 -- 172.21.15.2:0/3042606718 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd9ec0d2b20 msgr2=0x7fd9ec0b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:41.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.655+0000 7fd9eaffd700 1 --2- 172.21.15.2:0/3042606718 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd9ec0d2b20 0x7fd9ec0b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:41.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.655+0000 7fd9eaffd700 1 -- 172.21.15.2:0/3042606718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9ec1607b0 msgr2=0x7fd9ec1f1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:41.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.655+0000 7fd9eaffd700 1 --2- 172.21.15.2:0/3042606718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9ec1607b0 0x7fd9ec1f1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:41.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.655+0000 7fd9eaffd700 1 -- 172.21.15.2:0/3042606718 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e00094a0 con 0x7fd9ec0d1510 2023-08-25T03:34:41.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.656+0000 7fd9eaffd700 1 --2- 172.21.15.2:0/3042606718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9ec0d1510 0x7fd9ec0b54e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd9e000ebe0 tx=0x7fd9e000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:41.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.656+0000 7fd9d37fe700 1 -- 172.21.15.2:0/3042606718 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0009760 con 0x7fd9ec0d1510 2023-08-25T03:34:41.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.656+0000 7fd9d37fe700 1 -- 172.21.15.2:0/3042606718 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e000c850 con 0x7fd9ec0d1510 2023-08-25T03:34:41.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.656+0000 7fd9f1380700 1 -- 172.21.15.2:0/3042606718 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9dc0094a0 con 0x7fd9ec0d1510 2023-08-25T03:34:41.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.656+0000 7fd9d37fe700 1 -- 172.21.15.2:0/3042606718 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e000cbe0 con 0x7fd9ec0d1510 2023-08-25T03:34:41.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.656+0000 7fd9f1380700 1 -- 172.21.15.2:0/3042606718 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9ec15c3e0 con 0x7fd9ec0d1510 2023-08-25T03:34:41.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.657+0000 7fd9f1380700 1 -- 172.21.15.2:0/3042606718 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9ec0746d0 con 0x7fd9ec0d1510 2023-08-25T03:34:41.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.657+0000 7fd9d37fe700 1 -- 172.21.15.2:0/3042606718 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd9e000cd40 con 0x7fd9ec0d1510 2023-08-25T03:34:41.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.658+0000 7fd9d37fe700 1 --2- 172.21.15.2:0/3042606718 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd9d806a0e0 0x7fd9d806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:41.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.658+0000 7fd9d37fe700 1 -- 172.21.15.2:0/3042606718 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fd9e0088550 con 0x7fd9ec0d1510 2023-08-25T03:34:41.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.658+0000 7fd9ea7fc700 1 --2- 172.21.15.2:0/3042606718 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd9d806a0e0 0x7fd9d806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:41.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.658+0000 7fd9ea7fc700 1 --2- 172.21.15.2:0/3042606718 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd9d806a0e0 0x7fd9d806c5a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc0090b0 tx=0x7fd9dc008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:41.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.662+0000 7fd9d37fe700 1 -- 172.21.15.2:0/3042606718 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd9e0053c40 con 0x7fd9ec0d1510 2023-08-25T03:34:41.861 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.860+0000 7fd9f1380700 1 -- 172.21.15.2:0/3042606718 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd9ec0b8090 con 0x7fd9ec0d1510 2023-08-25T03:34:41.862 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.860+0000 7fd9d37fe700 1 -- 172.21.15.2:0/3042606718 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd9e0057260 con 0x7fd9ec0d1510 2023-08-25T03:34:41.862 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-08-25T03:34:41.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.862+0000 7fd9f1380700 1 -- 172.21.15.2:0/3042606718 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd9d806a0e0 msgr2=0x7fd9d806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:41.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.862+0000 7fd9f1380700 1 --2- 172.21.15.2:0/3042606718 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd9d806a0e0 0x7fd9d806c5a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc0090b0 tx=0x7fd9dc008030 comp rx=0 tx=0).stop 2023-08-25T03:34:41.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.862+0000 7fd9f1380700 1 -- 172.21.15.2:0/3042606718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9ec0d1510 msgr2=0x7fd9ec0b54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:41.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.862+0000 7fd9f1380700 1 --2- 172.21.15.2:0/3042606718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9ec0d1510 0x7fd9ec0b54e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fd9e000ebe0 tx=0x7fd9e000eef0 comp rx=0 tx=0).stop 2023-08-25T03:34:41.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.862+0000 7fd9f1380700 1 -- 172.21.15.2:0/3042606718 shutdown_connections 2023-08-25T03:34:41.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.862+0000 7fd9f1380700 1 --2- 172.21.15.2:0/3042606718 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd9d806a0e0 0x7fd9d806c5a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:41.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.862+0000 7fd9f1380700 1 --2- 172.21.15.2:0/3042606718 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd9ec0d1510 0x7fd9ec0b54e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:41.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.862+0000 7fd9f1380700 1 --2- 172.21.15.2:0/3042606718 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9ec1607b0 0x7fd9ec1f1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:41.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.862+0000 7fd9f1380700 1 --2- 172.21.15.2:0/3042606718 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd9ec0d2b20 0x7fd9ec0b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:41.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.862+0000 7fd9f1380700 1 -- 172.21.15.2:0/3042606718 >> 172.21.15.2:0/3042606718 conn(0x7fd9ec0cfb70 msgr2=0x7fd9ec0d39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:41.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.863+0000 7fd9f1380700 1 -- 172.21.15.2:0/3042606718 shutdown_connections 2023-08-25T03:34:41.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:41.863+0000 7fd9f1380700 1 -- 172.21.15.2:0/3042606718 wait complete. 2023-08-25T03:34:41.877 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T03:34:42.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.004+0000 7fa7c0ba0700 1 Processor -- start 2023-08-25T03:34:42.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.004+0000 7fa7c0ba0700 1 -- start start 2023-08-25T03:34:42.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.005+0000 7fa7c0ba0700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa7bc15b170 0x7fa7bc0cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:42.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.005+0000 7fa7c0ba0700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7bc0cf6f0 con 0x7fa7bc0cdf00 2023-08-25T03:34:42.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.005+0000 7fa7c0ba0700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7bc0d1d50 con 0x7fa7bc0cc640 2023-08-25T03:34:42.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.005+0000 7fa7c0ba0700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7bc15b550 con 0x7fa7bc15b170 2023-08-25T03:34:42.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.005+0000 7fa7bad9d700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fa7bc0cdf00 0x7fa7bc06d9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47120/0 (socket says 172.21.15.2:47120) 2023-08-25T03:34:42.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.005+0000 7fa7bad9d700 1 -- 172.21.15.2:0/3317552636 learned_addr learned my addr 172.21.15.2:0/3317552636 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:42.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.005+0000 7fa7ba59c700 1 --2- 172.21.15.2:0/3317552636 >> v2:172.21.15.31:3300/0 conn(0x7fa7bc15b170 0x7fa7bc0cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:42.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.005+0000 7fa7b959a700 1 -- 172.21.15.2:0/3317552636 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 584984852 0 0) 0x7fa7bc0cf6f0 con 0x7fa7bc0cdf00 2023-08-25T03:34:42.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.005+0000 7fa7b959a700 1 -- 172.21.15.2:0/3317552636 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7a40035b0 con 0x7fa7bc0cdf00 2023-08-25T03:34:42.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.005+0000 7fa7b959a700 1 -- 172.21.15.2:0/3317552636 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 85354156 0 0) 0x7fa7a40035b0 con 0x7fa7bc0cdf00 2023-08-25T03:34:42.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.005+0000 7fa7b959a700 1 -- 172.21.15.2:0/3317552636 >> v2:172.21.15.31:3300/0 conn(0x7fa7bc15b170 msgr2=0x7fa7bc0cfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:42.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.005+0000 7fa7b959a700 1 --2- 172.21.15.2:0/3317552636 >> v2:172.21.15.31:3300/0 conn(0x7fa7bc15b170 0x7fa7bc0cfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:42.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.006+0000 7fa7b959a700 1 -- 172.21.15.2:0/3317552636 >> v1:172.21.15.31:6789/0 conn(0x7fa7bc0cc640 legacy=0x7fa7bc0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:42.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.006+0000 7fa7b959a700 1 -- 172.21.15.2:0/3317552636 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc0ceae0 con 0x7fa7bc0cdf00 2023-08-25T03:34:42.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.006+0000 7fa7b959a700 1 -- 172.21.15.2:0/3317552636 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7fa7b40019f0 con 0x7fa7bc0cdf00 2023-08-25T03:34:42.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.006+0000 7fa7b959a700 1 -- 172.21.15.2:0/3317552636 >> v1:172.21.15.2:6789/0 conn(0x7fa7bc0cdf00 legacy=0x7fa7bc06d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:42.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.006+0000 7fa7b959a700 1 --2- 172.21.15.2:0/3317552636 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7a4003a00 0x7fa7a4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:42.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.006+0000 7fa7b959a700 1 --2- 172.21.15.2:0/3317552636 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa7a4004850 0x7fa7a4004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:42.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.006+0000 7fa7b959a700 1 --2- 172.21.15.2:0/3317552636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7a4005210 0x7fa7a400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:42.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.006+0000 7fa7b959a700 1 -- 172.21.15.2:0/3317552636 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc15b550 con 0x7fa7a4003a00 2023-08-25T03:34:42.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.006+0000 7fa7b959a700 1 -- 172.21.15.2:0/3317552636 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a400c0b0 con 0x7fa7a4005210 2023-08-25T03:34:42.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.006+0000 7fa7b959a700 1 -- 172.21.15.2:0/3317552636 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a400c270 con 0x7fa7a4004850 2023-08-25T03:34:42.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.006+0000 7fa7ba59c700 1 --2- 172.21.15.2:0/3317552636 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7a4003a00 0x7fa7a4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:42.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.006+0000 7fa7bad9d700 1 --2- 172.21.15.2:0/3317552636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7a4005210 0x7fa7a400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:42.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.006+0000 7fa7b9d9b700 1 --2- 172.21.15.2:0/3317552636 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa7a4004850 0x7fa7a4004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:42.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.006+0000 7fa7ba59c700 1 -- 172.21.15.2:0/3317552636 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa7a4004850 msgr2=0x7fa7a4004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:42.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.006+0000 7fa7ba59c700 1 --2- 172.21.15.2:0/3317552636 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa7a4004850 0x7fa7a4004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:42.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.006+0000 7fa7ba59c700 1 -- 172.21.15.2:0/3317552636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7a4005210 msgr2=0x7fa7a400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:42.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.006+0000 7fa7ba59c700 1 --2- 172.21.15.2:0/3317552636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7a4005210 0x7fa7a400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:42.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.006+0000 7fa7ba59c700 1 -- 172.21.15.2:0/3317552636 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a40035b0 con 0x7fa7a4003a00 2023-08-25T03:34:42.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.006+0000 7fa7bad9d700 1 -- 172.21.15.2:0/3317552636 reap_dead start 2023-08-25T03:34:42.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.006+0000 7fa7ba59c700 1 --2- 172.21.15.2:0/3317552636 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7a4003a00 0x7fa7a4003e20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009b40 tx=0x7fa7a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c4a7ae58a6cbe93 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:42.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.006+0000 7fa7b959a700 1 -- 172.21.15.2:0/3317552636 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac005a20 con 0x7fa7a4003a00 2023-08-25T03:34:42.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.006+0000 7fa7b959a700 1 -- 172.21.15.2:0/3317552636 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ac005b80 con 0x7fa7a4003a00 2023-08-25T03:34:42.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.006+0000 7fa7b959a700 1 -- 172.21.15.2:0/3317552636 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac00e640 con 0x7fa7a4003a00 2023-08-25T03:34:42.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.007+0000 7fa7c0ba0700 1 -- 172.21.15.2:0/3317552636 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7a4003a00 msgr2=0x7fa7a4003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:42.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.007+0000 7fa7c0ba0700 1 --2- 172.21.15.2:0/3317552636 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7a4003a00 0x7fa7a4003e20 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac009b40 tx=0x7fa7a4005880 comp rx=0 tx=0).stop 2023-08-25T03:34:42.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.007+0000 7fa7c0ba0700 1 -- 172.21.15.2:0/3317552636 shutdown_connections 2023-08-25T03:34:42.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.007+0000 7fa7c0ba0700 1 --2- 172.21.15.2:0/3317552636 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7a4003a00 0x7fa7a4003e20 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:42.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.007+0000 7fa7c0ba0700 1 -- 172.21.15.2:0/3317552636 >> 172.21.15.2:0/3317552636 conn(0x7fa7bc09a200 msgr2=0x7fa7bc0d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:42.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.007+0000 7fa7c0ba0700 1 -- 172.21.15.2:0/3317552636 shutdown_connections 2023-08-25T03:34:42.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.007+0000 7fa7c0ba0700 1 -- 172.21.15.2:0/3317552636 wait complete. 2023-08-25T03:34:42.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.008+0000 7fa7c0ba0700 1 Processor -- start 2023-08-25T03:34:42.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.008+0000 7fa7c0ba0700 1 -- start start 2023-08-25T03:34:42.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.009+0000 7fa7c0ba0700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa7a4003a00 0x7fa7bc15b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:42.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.009+0000 7fa7c0ba0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7bc0b1a50 0x7fa7bc0b1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:42.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.009+0000 7fa7c0ba0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7bc0ca2c0 0x7fa7bc0ca740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:42.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.009+0000 7fa7c0ba0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc0c44c0 con 0x7fa7bc0b1a50 2023-08-25T03:34:42.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.009+0000 7fa7c0ba0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc0b2410 con 0x7fa7bc0ca2c0 2023-08-25T03:34:42.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.009+0000 7fa7c0ba0700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7bc15c0d0 con 0x7fa7a4003a00 2023-08-25T03:34:42.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.009+0000 7fa7b9d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7bc0b1a50 0x7fa7bc0b1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:42.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.009+0000 7fa7b9d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7bc0b1a50 0x7fa7bc0b1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60464/0 (socket says 172.21.15.2:60464) 2023-08-25T03:34:42.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.009+0000 7fa7b9d9b700 1 -- 172.21.15.2:0/2484522245 learned_addr learned my addr 172.21.15.2:0/2484522245 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:42.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.010+0000 7fa7ba59c700 1 --2- 172.21.15.2:0/2484522245 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa7a4003a00 0x7fa7bc15b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:42.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.010+0000 7fa7bad9d700 1 --2- 172.21.15.2:0/2484522245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7bc0ca2c0 0x7fa7bc0ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:42.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.010+0000 7fa7b9d9b700 1 -- 172.21.15.2:0/2484522245 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa7a4003a00 msgr2=0x7fa7bc15b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:42.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.010+0000 7fa7b9d9b700 1 --2- 172.21.15.2:0/2484522245 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa7a4003a00 0x7fa7bc15b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:42.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.010+0000 7fa7b9d9b700 1 -- 172.21.15.2:0/2484522245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7bc0ca2c0 msgr2=0x7fa7bc0ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:42.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.010+0000 7fa7b9d9b700 1 --2- 172.21.15.2:0/2484522245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7bc0ca2c0 0x7fa7bc0ca740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:42.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.010+0000 7fa7b9d9b700 1 -- 172.21.15.2:0/2484522245 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7bc15c3b0 con 0x7fa7bc0b1a50 2023-08-25T03:34:42.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.010+0000 7fa7b9d9b700 1 --2- 172.21.15.2:0/2484522245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7bc0b1a50 0x7fa7bc0b1ed0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000f770 tx=0x7fa7b000fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:42.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.010+0000 7fa7aaffd700 1 -- 172.21.15.2:0/2484522245 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b00052e0 con 0x7fa7bc0b1a50 2023-08-25T03:34:42.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.010+0000 7fa7aaffd700 1 -- 172.21.15.2:0/2484522245 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b0005440 con 0x7fa7bc0b1a50 2023-08-25T03:34:42.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.010+0000 7fa7c0ba0700 1 -- 172.21.15.2:0/2484522245 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7bc15c690 con 0x7fa7bc0b1a50 2023-08-25T03:34:42.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.010+0000 7fa7aaffd700 1 -- 172.21.15.2:0/2484522245 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b00038a0 con 0x7fa7bc0b1a50 2023-08-25T03:34:42.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.010+0000 7fa7c0ba0700 1 -- 172.21.15.2:0/2484522245 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7bc15cc50 con 0x7fa7bc0b1a50 2023-08-25T03:34:42.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.011+0000 7fa7c0ba0700 1 -- 172.21.15.2:0/2484522245 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7bc0746d0 con 0x7fa7bc0b1a50 2023-08-25T03:34:42.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.011+0000 7fa7aaffd700 1 -- 172.21.15.2:0/2484522245 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa7b0003a00 con 0x7fa7bc0b1a50 2023-08-25T03:34:42.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.012+0000 7fa7aaffd700 1 --2- 172.21.15.2:0/2484522245 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa79c06a110 0x7fa79c06c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:42.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.012+0000 7fa7aaffd700 1 -- 172.21.15.2:0/2484522245 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa7b0088330 con 0x7fa7bc0b1a50 2023-08-25T03:34:42.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.012+0000 7fa7ba59c700 1 --2- 172.21.15.2:0/2484522245 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa79c06a110 0x7fa79c06c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:42.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.012+0000 7fa7ba59c700 1 --2- 172.21.15.2:0/2484522245 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa79c06a110 0x7fa79c06c5d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac000c00 tx=0x7fa7ac008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:42.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.016+0000 7fa7aaffd700 1 -- 172.21.15.2:0/2484522245 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa7b0053990 con 0x7fa7bc0b1a50 2023-08-25T03:34:42.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.218+0000 7fa7c0ba0700 1 -- 172.21.15.2:0/2484522245 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa7bc0ce620 con 0x7fa7bc0b1a50 2023-08-25T03:34:42.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.218+0000 7fa7aaffd700 1 -- 172.21.15.2:0/2484522245 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa7b0003cb0 con 0x7fa7bc0b1a50 2023-08-25T03:34:42.221 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-08-25T03:34:42.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.220+0000 7fa7c0ba0700 1 -- 172.21.15.2:0/2484522245 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa79c06a110 msgr2=0x7fa79c06c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:42.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.220+0000 7fa7c0ba0700 1 --2- 172.21.15.2:0/2484522245 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa79c06a110 0x7fa79c06c5d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac000c00 tx=0x7fa7ac008030 comp rx=0 tx=0).stop 2023-08-25T03:34:42.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.220+0000 7fa7c0ba0700 1 -- 172.21.15.2:0/2484522245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7bc0b1a50 msgr2=0x7fa7bc0b1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:42.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.220+0000 7fa7c0ba0700 1 --2- 172.21.15.2:0/2484522245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7bc0b1a50 0x7fa7bc0b1ed0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa7b000f770 tx=0x7fa7b000fb30 comp rx=0 tx=0).stop 2023-08-25T03:34:42.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.220+0000 7fa7c0ba0700 1 -- 172.21.15.2:0/2484522245 shutdown_connections 2023-08-25T03:34:42.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.220+0000 7fa7c0ba0700 1 --2- 172.21.15.2:0/2484522245 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7bc0ca2c0 0x7fa7bc0ca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:42.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.220+0000 7fa7c0ba0700 1 --2- 172.21.15.2:0/2484522245 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa7a4003a00 0x7fa7bc15b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:42.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.220+0000 7fa7c0ba0700 1 --2- 172.21.15.2:0/2484522245 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa79c06a110 0x7fa79c06c5d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:42.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.220+0000 7fa7c0ba0700 1 --2- 172.21.15.2:0/2484522245 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa7bc0b1a50 0x7fa7bc0b1ed0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:42.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.220+0000 7fa7c0ba0700 1 -- 172.21.15.2:0/2484522245 >> 172.21.15.2:0/2484522245 conn(0x7fa7bc09a200 msgr2=0x7fa7bc06e040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:42.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.220+0000 7fa7c0ba0700 1 -- 172.21.15.2:0/2484522245 shutdown_connections 2023-08-25T03:34:42.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.220+0000 7fa7c0ba0700 1 -- 172.21.15.2:0/2484522245 wait complete. 2023-08-25T03:34:42.235 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-08-25T03:34:42.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.361+0000 7f086d530700 1 Processor -- start 2023-08-25T03:34:42.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.362+0000 7f086d530700 1 -- start start 2023-08-25T03:34:42.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.362+0000 7f086d530700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f086806d380 0x7f08680cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:42.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.362+0000 7f086d530700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f08680ca3e0 0x7f08680cb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:42.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.362+0000 7f086d530700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08680cb550 con 0x7f086815d1a0 2023-08-25T03:34:42.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.362+0000 7f086d530700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0868154090 con 0x7f086806d380 2023-08-25T03:34:42.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.362+0000 7f086d530700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08680ca890 con 0x7f08680ca3e0 2023-08-25T03:34:42.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.362+0000 7f0866ffd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f086806d380 0x7f08680cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:42.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.363+0000 7f0866ffd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f086806d380 0x7f08680cd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60470/0 (socket says 172.21.15.2:60470) 2023-08-25T03:34:42.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.363+0000 7f0866ffd700 1 -- 172.21.15.2:0/250833883 learned_addr learned my addr 172.21.15.2:0/250833883 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:42.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.363+0000 7f08667fc700 1 --2- 172.21.15.2:0/250833883 >> v2:172.21.15.31:3300/0 conn(0x7f08680ca3e0 0x7f08680cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:42.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.363+0000 7f0866ffd700 1 -- 172.21.15.2:0/250833883 >> v2:172.21.15.31:3300/0 conn(0x7f08680ca3e0 msgr2=0x7f08680cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:42.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.363+0000 7f0866ffd700 1 --2- 172.21.15.2:0/250833883 >> v2:172.21.15.31:3300/0 conn(0x7f08680ca3e0 0x7f08680cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:42.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.363+0000 7f0866ffd700 1 -- 172.21.15.2:0/250833883 >> v1:172.21.15.31:6789/0 conn(0x7f086815d1a0 legacy=0x7f08680c3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:42.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.363+0000 7f0866ffd700 1 -- 172.21.15.2:0/250833883 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086806d760 con 0x7f086806d380 2023-08-25T03:34:42.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.363+0000 7f0866ffd700 1 --2- 172.21.15.2:0/250833883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f086806d380 0x7f08680cd250 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f086806e0a0 tx=0x7f085c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154324cb813d1cc9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:42.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.363+0000 7f0865ffb700 1 -- 172.21.15.2:0/250833883 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c00b870 con 0x7f086806d380 2023-08-25T03:34:42.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.363+0000 7f0865ffb700 1 -- 172.21.15.2:0/250833883 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085c00b9d0 con 0x7f086806d380 2023-08-25T03:34:42.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.363+0000 7f0865ffb700 1 -- 172.21.15.2:0/250833883 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c00bd60 con 0x7f086806d380 2023-08-25T03:34:42.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.364+0000 7f086d530700 1 -- 172.21.15.2:0/250833883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f086806d380 msgr2=0x7f08680cd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:42.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.364+0000 7f086d530700 1 --2- 172.21.15.2:0/250833883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f086806d380 0x7f08680cd250 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f086806e0a0 tx=0x7f085c009bd0 comp rx=0 tx=0).stop 2023-08-25T03:34:42.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.364+0000 7f086d530700 1 -- 172.21.15.2:0/250833883 shutdown_connections 2023-08-25T03:34:42.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.364+0000 7f086d530700 1 --2- 172.21.15.2:0/250833883 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f086806d380 0x7f08680cd250 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:42.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.364+0000 7f086d530700 1 --2- 172.21.15.2:0/250833883 >> v2:172.21.15.31:3300/0 conn(0x7f08680ca3e0 0x7f08680cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:42.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.364+0000 7f086d530700 1 -- 172.21.15.2:0/250833883 >> 172.21.15.2:0/250833883 conn(0x7f08680b1240 msgr2=0x7f086806a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:42.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.364+0000 7f086d530700 1 -- 172.21.15.2:0/250833883 shutdown_connections 2023-08-25T03:34:42.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.364+0000 7f086d530700 1 -- 172.21.15.2:0/250833883 wait complete. 2023-08-25T03:34:42.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.365+0000 7f086d530700 1 Processor -- start 2023-08-25T03:34:42.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.365+0000 7f086d530700 1 -- start start 2023-08-25T03:34:42.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.366+0000 7f086d530700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f086806d380 0x7f0868141df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:42.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.366+0000 7f086d530700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08680ca3e0 0x7f0868130380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:42.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.366+0000 7f086d530700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086815d1a0 0x7f08681308c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:42.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.366+0000 7f086d530700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08680c6d10 con 0x7f08680ca3e0 2023-08-25T03:34:42.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.366+0000 7f086d530700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08680c6ff0 con 0x7f086815d1a0 2023-08-25T03:34:42.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.366+0000 7f086d530700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08680c72d0 con 0x7f086806d380 2023-08-25T03:34:42.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.366+0000 7f08667fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08680ca3e0 0x7f0868130380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:42.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.366+0000 7f08667fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08680ca3e0 0x7f0868130380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60478/0 (socket says 172.21.15.2:60478) 2023-08-25T03:34:42.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.366+0000 7f08667fc700 1 -- 172.21.15.2:0/962103459 learned_addr learned my addr 172.21.15.2:0/962103459 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:42.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.366+0000 7f0866ffd700 1 --2- 172.21.15.2:0/962103459 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f086806d380 0x7f0868141df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:42.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.366+0000 7f08677fe700 1 --2- 172.21.15.2:0/962103459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086815d1a0 0x7f08681308c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:42.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.366+0000 7f08667fc700 1 -- 172.21.15.2:0/962103459 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f086806d380 msgr2=0x7f0868141df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:42.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.366+0000 7f08667fc700 1 --2- 172.21.15.2:0/962103459 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f086806d380 0x7f0868141df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:42.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.366+0000 7f08667fc700 1 -- 172.21.15.2:0/962103459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086815d1a0 msgr2=0x7f08681308c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:42.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.366+0000 7f08667fc700 1 --2- 172.21.15.2:0/962103459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086815d1a0 0x7f08681308c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:42.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.366+0000 7f08667fc700 1 -- 172.21.15.2:0/962103459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f085c009550 con 0x7f08680ca3e0 2023-08-25T03:34:42.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.366+0000 7f08667fc700 1 --2- 172.21.15.2:0/962103459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08680ca3e0 0x7f0868130380 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f085800e810 tx=0x7f085800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:42.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.367+0000 7f084f7fe700 1 -- 172.21.15.2:0/962103459 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085800c070 con 0x7f08680ca3e0 2023-08-25T03:34:42.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.367+0000 7f084f7fe700 1 -- 172.21.15.2:0/962103459 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0858013610 con 0x7f08680ca3e0 2023-08-25T03:34:42.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.367+0000 7f086d530700 1 -- 172.21.15.2:0/962103459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08680c75b0 con 0x7f08680ca3e0 2023-08-25T03:34:42.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.367+0000 7f084f7fe700 1 -- 172.21.15.2:0/962103459 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08580139c0 con 0x7f08680ca3e0 2023-08-25T03:34:42.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.367+0000 7f086d530700 1 -- 172.21.15.2:0/962103459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08680c5f80 con 0x7f08680ca3e0 2023-08-25T03:34:42.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.368+0000 7f086d530700 1 -- 172.21.15.2:0/962103459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08680c7ad0 con 0x7f08680ca3e0 2023-08-25T03:34:42.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.368+0000 7f084f7fe700 1 -- 172.21.15.2:0/962103459 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0858013b20 con 0x7f08680ca3e0 2023-08-25T03:34:42.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.369+0000 7f084f7fe700 1 --2- 172.21.15.2:0/962103459 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f085406a0e0 0x7f085406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:42.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.369+0000 7f0866ffd700 1 --2- 172.21.15.2:0/962103459 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f085406a0e0 0x7f085406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:42.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.369+0000 7f084f7fe700 1 -- 172.21.15.2:0/962103459 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f0858088180 con 0x7f08680ca3e0 2023-08-25T03:34:42.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.369+0000 7f0866ffd700 1 --2- 172.21.15.2:0/962103459 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f085406a0e0 0x7f085406c5a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f085c014040 tx=0x7f085c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:42.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.373+0000 7f084f7fe700 1 -- 172.21.15.2:0/962103459 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f08680c7ad0 con 0x7f08680ca3e0 2023-08-25T03:34:42.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.577+0000 7f086d530700 1 -- 172.21.15.2:0/962103459 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f08680c7ad0 con 0x7f08680ca3e0 2023-08-25T03:34:42.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.581+0000 7f084f7fe700 1 -- 172.21.15.2:0/962103459 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f08680c7ad0 con 0x7f08680ca3e0 2023-08-25T03:34:42.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.583+0000 7f086d530700 1 -- 172.21.15.2:0/962103459 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f085406a0e0 msgr2=0x7f085406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:42.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.583+0000 7f086d530700 1 --2- 172.21.15.2:0/962103459 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f085406a0e0 0x7f085406c5a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f085c014040 tx=0x7f085c008030 comp rx=0 tx=0).stop 2023-08-25T03:34:42.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.583+0000 7f086d530700 1 -- 172.21.15.2:0/962103459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08680ca3e0 msgr2=0x7f0868130380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:42.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.583+0000 7f086d530700 1 --2- 172.21.15.2:0/962103459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08680ca3e0 0x7f0868130380 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f085800e810 tx=0x7f085800eb20 comp rx=0 tx=0).stop 2023-08-25T03:34:42.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.583+0000 7f086d530700 1 -- 172.21.15.2:0/962103459 shutdown_connections 2023-08-25T03:34:42.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.583+0000 7f086d530700 1 --2- 172.21.15.2:0/962103459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f086815d1a0 0x7f08681308c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:42.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.583+0000 7f086d530700 1 --2- 172.21.15.2:0/962103459 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f086806d380 0x7f0868141df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:42.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.583+0000 7f086d530700 1 --2- 172.21.15.2:0/962103459 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f085406a0e0 0x7f085406c5a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:42.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.583+0000 7f086d530700 1 --2- 172.21.15.2:0/962103459 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f08680ca3e0 0x7f0868130380 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:42.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.583+0000 7f086d530700 1 -- 172.21.15.2:0/962103459 >> 172.21.15.2:0/962103459 conn(0x7f08680b1240 msgr2=0x7f08680cb6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:42.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.583+0000 7f086d530700 1 -- 172.21.15.2:0/962103459 shutdown_connections 2023-08-25T03:34:42.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.584+0000 7f086d530700 1 -- 172.21.15.2:0/962103459 wait complete. 2023-08-25T03:34:42.597 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-25T03:34:42.598 DEBUG:teuthology.orchestra.run.smithi002:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T03:34:42.636 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T03:34:42.688 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-25T03:34:42.689 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:34:42.689 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:34:42.707 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:34:42.708 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:34:42.765 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:34:42.766 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:34:42.823 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:34:42.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.985+0000 7f253518a700 1 Processor -- start 2023-08-25T03:34:42.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.985+0000 7f253518a700 1 -- start start 2023-08-25T03:34:42.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.986+0000 7f253518a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f25300cc2c0 0x7f25300716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:42.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.986+0000 7f253518a700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25300cc730 con 0x7f25300b20b0 2023-08-25T03:34:42.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.986+0000 7f253518a700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f253006de40 con 0x7f253015f770 2023-08-25T03:34:42.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.986+0000 7f253518a700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f253015edd0 con 0x7f25300cc2c0 2023-08-25T03:34:42.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.986+0000 7f252e59c700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f25300b20b0 0x7f253015b7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47150/0 (socket says 172.21.15.2:47150) 2023-08-25T03:34:42.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.986+0000 7f252e59c700 1 -- 172.21.15.2:0/1414312418 learned_addr learned my addr 172.21.15.2:0/1414312418 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:42.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.987+0000 7f252dd9b700 1 -- 172.21.15.2:0/1414312418 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2290957167 0 0) 0x7f25300cc730 con 0x7f25300b20b0 2023-08-25T03:34:42.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.987+0000 7f252f59e700 1 --2- 172.21.15.2:0/1414312418 >> v2:172.21.15.31:3300/0 conn(0x7f25300cc2c0 0x7f25300716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:42.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.987+0000 7f252dd9b700 1 -- 172.21.15.2:0/1414312418 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f251c0035b0 con 0x7f25300b20b0 2023-08-25T03:34:42.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.987+0000 7f252dd9b700 1 -- 172.21.15.2:0/1414312418 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2982207687 0 0) 0x7f251c0035b0 con 0x7f25300b20b0 2023-08-25T03:34:42.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.987+0000 7f252dd9b700 1 -- 172.21.15.2:0/1414312418 >> v2:172.21.15.31:3300/0 conn(0x7f25300cc2c0 msgr2=0x7f25300716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:42.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.987+0000 7f252dd9b700 1 --2- 172.21.15.2:0/1414312418 >> v2:172.21.15.31:3300/0 conn(0x7f25300cc2c0 0x7f25300716a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:42.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.987+0000 7f252dd9b700 1 -- 172.21.15.2:0/1414312418 >> v1:172.21.15.31:6789/0 conn(0x7f253015f770 legacy=0x7f25300b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:42.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.987+0000 7f252dd9b700 1 -- 172.21.15.2:0/1414312418 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253006d080 con 0x7f25300b20b0 2023-08-25T03:34:42.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.987+0000 7f252dd9b700 1 -- 172.21.15.2:0/1414312418 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f2518001670 con 0x7f25300b20b0 2023-08-25T03:34:42.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.987+0000 7f252dd9b700 1 -- 172.21.15.2:0/1414312418 >> v1:172.21.15.2:6789/0 conn(0x7f25300b20b0 legacy=0x7f253015b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:42.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.987+0000 7f252dd9b700 1 --2- 172.21.15.2:0/1414312418 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f251c003a00 0x7f251c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:42.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.987+0000 7f252dd9b700 1 --2- 172.21.15.2:0/1414312418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f251c004850 0x7f251c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:42.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.987+0000 7f252dd9b700 1 --2- 172.21.15.2:0/1414312418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f251c005210 0x7f251c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:42.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.987+0000 7f252dd9b700 1 -- 172.21.15.2:0/1414312418 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251c00c080 con 0x7f251c004850 2023-08-25T03:34:42.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.987+0000 7f252dd9b700 1 -- 172.21.15.2:0/1414312418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f251c00c1f0 con 0x7f251c005210 2023-08-25T03:34:42.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.987+0000 7f252dd9b700 1 -- 172.21.15.2:0/1414312418 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f251c00c3b0 con 0x7f251c003a00 2023-08-25T03:34:42.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.987+0000 7f252e59c700 1 --2- 172.21.15.2:0/1414312418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f251c004850 0x7f251c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:42.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.987+0000 7f252ed9d700 1 --2- 172.21.15.2:0/1414312418 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f251c003a00 0x7f251c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:42.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.987+0000 7f252f59e700 1 --2- 172.21.15.2:0/1414312418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f251c005210 0x7f251c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:42.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.987+0000 7f252e59c700 1 -- 172.21.15.2:0/1414312418 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f251c003a00 msgr2=0x7f251c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:42.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.987+0000 7f252e59c700 1 --2- 172.21.15.2:0/1414312418 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f251c003a00 0x7f251c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:42.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.987+0000 7f252e59c700 1 -- 172.21.15.2:0/1414312418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f251c005210 msgr2=0x7f251c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:42.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.987+0000 7f252e59c700 1 --2- 172.21.15.2:0/1414312418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f251c005210 0x7f251c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:42.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.987+0000 7f252e59c700 1 -- 172.21.15.2:0/1414312418 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f251c0035b0 con 0x7f251c004850 2023-08-25T03:34:42.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.987+0000 7f252f59e700 1 -- 172.21.15.2:0/1414312418 reap_dead start 2023-08-25T03:34:42.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.988+0000 7f252e59c700 1 --2- 172.21.15.2:0/1414312418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f251c004850 0x7f251c004cd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f251800c210 tx=0x7f251c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912efc4302ad94d5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:42.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.988+0000 7f252dd9b700 1 -- 172.21.15.2:0/1414312418 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2518007c10 con 0x7f251c004850 2023-08-25T03:34:42.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.988+0000 7f252dd9b700 1 -- 172.21.15.2:0/1414312418 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251800cdb0 con 0x7f251c004850 2023-08-25T03:34:42.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.988+0000 7f252dd9b700 1 -- 172.21.15.2:0/1414312418 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2518005ba0 con 0x7f251c004850 2023-08-25T03:34:42.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.988+0000 7f253518a700 1 -- 172.21.15.2:0/1414312418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f251c004850 msgr2=0x7f251c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:43.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.988+0000 7f253518a700 1 --2- 172.21.15.2:0/1414312418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f251c004850 0x7f251c004cd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f251800c210 tx=0x7f251c005880 comp rx=0 tx=0).stop 2023-08-25T03:34:43.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.988+0000 7f253518a700 1 -- 172.21.15.2:0/1414312418 shutdown_connections 2023-08-25T03:34:43.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.988+0000 7f253518a700 1 --2- 172.21.15.2:0/1414312418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f251c004850 0x7f251c004cd0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.988+0000 7f253518a700 1 -- 172.21.15.2:0/1414312418 >> 172.21.15.2:0/1414312418 conn(0x7f25300cfea0 msgr2=0x7f25300d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:43.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.988+0000 7f253518a700 1 -- 172.21.15.2:0/1414312418 shutdown_connections 2023-08-25T03:34:43.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.988+0000 7f253518a700 1 -- 172.21.15.2:0/1414312418 wait complete. 2023-08-25T03:34:43.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.990+0000 7f253518a700 1 Processor -- start 2023-08-25T03:34:43.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.990+0000 7f253518a700 1 -- start start 2023-08-25T03:34:43.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.991+0000 7f253518a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f251c004850 0x7f2530152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:43.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.991+0000 7f253518a700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2530157760 0x7f2530157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:43.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.991+0000 7f253518a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25300b1440 0x7f25300b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:43.005 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.991+0000 7f253518a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25301f05d0 con 0x7f251c004850 2023-08-25T03:34:43.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.991+0000 7f253518a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25301f4a70 con 0x7f25300b1440 2023-08-25T03:34:43.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.991+0000 7f253518a700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25300a8670 con 0x7f2530157760 2023-08-25T03:34:43.006 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.991+0000 7f252ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f251c004850 0x7f2530152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:43.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.991+0000 7f252ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f251c004850 0x7f2530152990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60496/0 (socket says 172.21.15.2:60496) 2023-08-25T03:34:43.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.991+0000 7f252ed9d700 1 -- 172.21.15.2:0/1618546783 learned_addr learned my addr 172.21.15.2:0/1618546783 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:43.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.991+0000 7f252e59c700 1 --2- 172.21.15.2:0/1618546783 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2530157760 0x7f2530157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:43.007 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.991+0000 7f252f59e700 1 --2- 172.21.15.2:0/1618546783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25300b1440 0x7f25300b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:43.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.991+0000 7f252ed9d700 1 -- 172.21.15.2:0/1618546783 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2530157760 msgr2=0x7f2530157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:43.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.991+0000 7f252ed9d700 1 --2- 172.21.15.2:0/1618546783 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2530157760 0x7f2530157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.008 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.991+0000 7f252ed9d700 1 -- 172.21.15.2:0/1618546783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25300b1440 msgr2=0x7f25300b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:43.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.991+0000 7f252ed9d700 1 --2- 172.21.15.2:0/1618546783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25300b1440 0x7f25300b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.991+0000 7f252ed9d700 1 -- 172.21.15.2:0/1618546783 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25301f3ba0 con 0x7f251c004850 2023-08-25T03:34:43.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.991+0000 7f252f59e700 1 --2- 172.21.15.2:0/1618546783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25300b1440 0x7f25300b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:43.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.992+0000 7f252ed9d700 1 --2- 172.21.15.2:0/1618546783 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f251c004850 0x7f2530152990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f252000f820 tx=0x7f252000fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:43.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.992+0000 7f25277fe700 1 -- 172.21.15.2:0/1618546783 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25200052e0 con 0x7f251c004850 2023-08-25T03:34:43.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.992+0000 7f253518a700 1 -- 172.21.15.2:0/1618546783 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25301f3df0 con 0x7f251c004850 2023-08-25T03:34:43.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.992+0000 7f25277fe700 1 -- 172.21.15.2:0/1618546783 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2520005440 con 0x7f251c004850 2023-08-25T03:34:43.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.992+0000 7f25277fe700 1 -- 172.21.15.2:0/1618546783 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25200038a0 con 0x7f251c004850 2023-08-25T03:34:43.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.992+0000 7f253518a700 1 -- 172.21.15.2:0/1618546783 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25301f43b0 con 0x7f251c004850 2023-08-25T03:34:43.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.993+0000 7f253518a700 1 -- 172.21.15.2:0/1618546783 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2530066f70 con 0x7f251c004850 2023-08-25T03:34:43.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.993+0000 7f25277fe700 1 -- 172.21.15.2:0/1618546783 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2520003a00 con 0x7f251c004850 2023-08-25T03:34:43.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.994+0000 7f25277fe700 1 --2- 172.21.15.2:0/1618546783 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f251006a110 0x7f251006c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:43.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.994+0000 7f25277fe700 1 -- 172.21.15.2:0/1618546783 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2520088180 con 0x7f251c004850 2023-08-25T03:34:43.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.994+0000 7f252e59c700 1 --2- 172.21.15.2:0/1618546783 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f251006a110 0x7f251006c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:43.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.994+0000 7f252e59c700 1 --2- 172.21.15.2:0/1618546783 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f251006a110 0x7f251006c5d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2518001930 tx=0x7f2518007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:43.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:42.998+0000 7f25277fe700 1 -- 172.21.15.2:0/1618546783 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2520053880 con 0x7f251c004850 2023-08-25T03:34:43.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.201+0000 7f253518a700 1 -- 172.21.15.2:0/1618546783 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25300af660 con 0x7f251c004850 2023-08-25T03:34:43.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.203+0000 7f25277fe700 1 -- 172.21.15.2:0/1618546783 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18135 (secure 0 0 0) 0x7f2520003cb0 con 0x7f251c004850 2023-08-25T03:34:43.205 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:43.205 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":24,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:34:04.604059+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:34:00.869118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T03:34:01.987016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:34:43.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.204+0000 7f253518a700 1 -- 172.21.15.2:0/1618546783 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f251006a110 msgr2=0x7f251006c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:43.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.204+0000 7f253518a700 1 --2- 172.21.15.2:0/1618546783 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f251006a110 0x7f251006c5d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2518001930 tx=0x7f2518007000 comp rx=0 tx=0).stop 2023-08-25T03:34:43.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.204+0000 7f253518a700 1 -- 172.21.15.2:0/1618546783 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f251c004850 msgr2=0x7f2530152990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:43.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.204+0000 7f253518a700 1 --2- 172.21.15.2:0/1618546783 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f251c004850 0x7f2530152990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f252000f820 tx=0x7f252000fb30 comp rx=0 tx=0).stop 2023-08-25T03:34:43.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.205+0000 7f253518a700 1 -- 172.21.15.2:0/1618546783 shutdown_connections 2023-08-25T03:34:43.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.205+0000 7f253518a700 1 --2- 172.21.15.2:0/1618546783 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f251006a110 0x7f251006c5d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.205+0000 7f253518a700 1 --2- 172.21.15.2:0/1618546783 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f251c004850 0x7f2530152990 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.205+0000 7f253518a700 1 --2- 172.21.15.2:0/1618546783 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25300b1440 0x7f25300b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.205+0000 7f253518a700 1 --2- 172.21.15.2:0/1618546783 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2530157760 0x7f2530157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.205+0000 7f253518a700 1 -- 172.21.15.2:0/1618546783 >> 172.21.15.2:0/1618546783 conn(0x7f25300cfea0 msgr2=0x7f253015a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:43.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.205+0000 7f253518a700 1 -- 172.21.15.2:0/1618546783 shutdown_connections 2023-08-25T03:34:43.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.205+0000 7f253518a700 1 -- 172.21.15.2:0/1618546783 wait complete. 2023-08-25T03:34:43.219 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T03:33:49.560643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T03:33:56.265184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T03:34:00.869118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T03:34:01.987016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T03:34:43.220 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T03:34:43.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.346+0000 7ffbc1448700 1 Processor -- start 2023-08-25T03:34:43.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.346+0000 7ffbc1448700 1 -- start start 2023-08-25T03:34:43.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.347+0000 7ffbc1448700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffbbc1607f0 0x7ffbbc070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:43.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.347+0000 7ffbc1448700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ffbbc0af180 0x7ffbbc0ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:43.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.347+0000 7ffbc1448700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbbc160bd0 con 0x7ffbbc06a1e0 2023-08-25T03:34:43.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.347+0000 7ffbc1448700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbbc0713a0 con 0x7ffbbc0af180 2023-08-25T03:34:43.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.347+0000 7ffbc1448700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc0718a0 con 0x7ffbbc1607f0 2023-08-25T03:34:43.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.347+0000 7ffbbb7fe700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7ffbbc06a1e0 0x7ffbbc06ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47174/0 (socket says 172.21.15.2:47174) 2023-08-25T03:34:43.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.347+0000 7ffbbb7fe700 1 -- 172.21.15.2:0/3438773893 learned_addr learned my addr 172.21.15.2:0/3438773893 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:43.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.347+0000 7ffbba7fc700 1 --2- 172.21.15.2:0/3438773893 >> v2:172.21.15.31:3300/0 conn(0x7ffbbc0af180 0x7ffbbc0ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:43.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.348+0000 7ffbbaffd700 1 --2- 172.21.15.2:0/3438773893 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffbbc1607f0 0x7ffbbc070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:43.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.348+0000 7ffbb9ffb700 1 -- 172.21.15.2:0/3438773893 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2058851304 0 0) 0x7ffbbc160bd0 con 0x7ffbbc06a1e0 2023-08-25T03:34:43.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.348+0000 7ffbb9ffb700 1 -- 172.21.15.2:0/3438773893 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffba40035b0 con 0x7ffbbc06a1e0 2023-08-25T03:34:43.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.348+0000 7ffbb9ffb700 1 -- 172.21.15.2:0/3438773893 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3487266738 0 0) 0x7ffba40035b0 con 0x7ffbbc06a1e0 2023-08-25T03:34:43.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.348+0000 7ffbb9ffb700 1 -- 172.21.15.2:0/3438773893 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffbbc1607f0 msgr2=0x7ffbbc070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:43.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.348+0000 7ffbb9ffb700 1 --2- 172.21.15.2:0/3438773893 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffbbc1607f0 0x7ffbbc070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.348+0000 7ffbb9ffb700 1 -- 172.21.15.2:0/3438773893 >> v2:172.21.15.31:3300/0 conn(0x7ffbbc0af180 msgr2=0x7ffbbc0ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:43.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.348+0000 7ffbb9ffb700 1 --2- 172.21.15.2:0/3438773893 >> v2:172.21.15.31:3300/0 conn(0x7ffbbc0af180 0x7ffbbc0ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.348+0000 7ffbb9ffb700 1 -- 172.21.15.2:0/3438773893 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbbc0cdd00 con 0x7ffbbc06a1e0 2023-08-25T03:34:43.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.348+0000 7ffbb9ffb700 1 -- 172.21.15.2:0/3438773893 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7ffbb4001a40 con 0x7ffbbc06a1e0 2023-08-25T03:34:43.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.348+0000 7ffbb9ffb700 1 -- 172.21.15.2:0/3438773893 >> v1:172.21.15.2:6789/0 conn(0x7ffbbc06a1e0 legacy=0x7ffbbc06ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:43.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.348+0000 7ffbb9ffb700 1 --2- 172.21.15.2:0/3438773893 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffbbc1607f0 0x7ffba4003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:43.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.348+0000 7ffbb9ffb700 1 --2- 172.21.15.2:0/3438773893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffba4003f80 0x7ffba4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:43.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.348+0000 7ffbb9ffb700 1 --2- 172.21.15.2:0/3438773893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffba4004d90 0x7ffba4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:43.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.348+0000 7ffbb9ffb700 1 -- 172.21.15.2:0/3438773893 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc0718a0 con 0x7ffba4004d90 2023-08-25T03:34:43.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.348+0000 7ffbb9ffb700 1 -- 172.21.15.2:0/3438773893 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffba400ba10 con 0x7ffba4003f80 2023-08-25T03:34:43.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.348+0000 7ffbb9ffb700 1 -- 172.21.15.2:0/3438773893 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffba400bb50 con 0x7ffbbc1607f0 2023-08-25T03:34:43.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.348+0000 7ffbbb7fe700 1 --2- 172.21.15.2:0/3438773893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffba4004d90 0x7ffba4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:43.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.348+0000 7ffbbaffd700 1 --2- 172.21.15.2:0/3438773893 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffbbc1607f0 0x7ffba4003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:43.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.348+0000 7ffbba7fc700 1 --2- 172.21.15.2:0/3438773893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffba4003f80 0x7ffba4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:43.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.348+0000 7ffbbb7fe700 1 -- 172.21.15.2:0/3438773893 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffbbc1607f0 msgr2=0x7ffba4003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:43.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.348+0000 7ffbbb7fe700 1 --2- 172.21.15.2:0/3438773893 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffbbc1607f0 0x7ffba4003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.348+0000 7ffbbb7fe700 1 -- 172.21.15.2:0/3438773893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffba4003f80 msgr2=0x7ffba4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:43.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.348+0000 7ffbbb7fe700 1 --2- 172.21.15.2:0/3438773893 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffba4003f80 0x7ffba4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.348+0000 7ffbbb7fe700 1 -- 172.21.15.2:0/3438773893 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba40035b0 con 0x7ffba4004d90 2023-08-25T03:34:43.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.349+0000 7ffbbb7fe700 1 --2- 172.21.15.2:0/3438773893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffba4004d90 0x7ffba4005210 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400a870 tx=0x7ffba4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd7389f5630150fc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:43.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.349+0000 7ffbb9ffb700 1 -- 172.21.15.2:0/3438773893 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4012600 con 0x7ffba4004d90 2023-08-25T03:34:43.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.349+0000 7ffbb9ffb700 1 -- 172.21.15.2:0/3438773893 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb4012760 con 0x7ffba4004d90 2023-08-25T03:34:43.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.349+0000 7ffbb9ffb700 1 -- 172.21.15.2:0/3438773893 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4012b10 con 0x7ffba4004d90 2023-08-25T03:34:43.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.349+0000 7ffbc1448700 1 -- 172.21.15.2:0/3438773893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffba4004d90 msgr2=0x7ffba4005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:43.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.349+0000 7ffbc1448700 1 --2- 172.21.15.2:0/3438773893 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffba4004d90 0x7ffba4005210 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400a870 tx=0x7ffba4005880 comp rx=0 tx=0).stop 2023-08-25T03:34:43.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.349+0000 7ffbbb7fe700 1 -- 172.21.15.2:0/3438773893 reap_dead start 2023-08-25T03:34:43.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.349+0000 7ffbc1448700 1 -- 172.21.15.2:0/3438773893 shutdown_connections 2023-08-25T03:34:43.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.349+0000 7ffbc1448700 1 -- 172.21.15.2:0/3438773893 >> 172.21.15.2:0/3438773893 conn(0x7ffbbc0cfd90 msgr2=0x7ffbbc0d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:43.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.349+0000 7ffbc1448700 1 -- 172.21.15.2:0/3438773893 shutdown_connections 2023-08-25T03:34:43.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.349+0000 7ffbc1448700 1 -- 172.21.15.2:0/3438773893 wait complete. 2023-08-25T03:34:43.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.351+0000 7ffbc1448700 1 Processor -- start 2023-08-25T03:34:43.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.351+0000 7ffbc1448700 1 -- start start 2023-08-25T03:34:43.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.351+0000 7ffbc1448700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffbbc14f5b0 0x7ffbbc1f5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:43.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.352+0000 7ffbc1448700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffbbc1f62a0 0x7ffbbc0c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:43.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.352+0000 7ffbc1448700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbbc0c3de0 0x7ffbbc1f2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:43.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.352+0000 7ffbc1448700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc1f6720 con 0x7ffbbc14f5b0 2023-08-25T03:34:43.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.352+0000 7ffbc1448700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc0c4260 con 0x7ffbbc0c3de0 2023-08-25T03:34:43.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.352+0000 7ffbc1448700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbbc0c5090 con 0x7ffbbc1f62a0 2023-08-25T03:34:43.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.352+0000 7ffbbaffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffbbc14f5b0 0x7ffbbc1f5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:43.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.352+0000 7ffbbaffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffbbc14f5b0 0x7ffbbc1f5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60516/0 (socket says 172.21.15.2:60516) 2023-08-25T03:34:43.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.352+0000 7ffbbaffd700 1 -- 172.21.15.2:0/1288402311 learned_addr learned my addr 172.21.15.2:0/1288402311 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:43.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.352+0000 7ffbbb7fe700 1 --2- 172.21.15.2:0/1288402311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbbc0c3de0 0x7ffbbc1f2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:43.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.352+0000 7ffbba7fc700 1 --2- 172.21.15.2:0/1288402311 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffbbc1f62a0 0x7ffbbc0c38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:43.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.352+0000 7ffbbaffd700 1 -- 172.21.15.2:0/1288402311 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffbbc1f62a0 msgr2=0x7ffbbc0c38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:43.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.352+0000 7ffbbaffd700 1 --2- 172.21.15.2:0/1288402311 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffbbc1f62a0 0x7ffbbc0c38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.352+0000 7ffbbaffd700 1 -- 172.21.15.2:0/1288402311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbbc0c3de0 msgr2=0x7ffbbc1f2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:43.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.352+0000 7ffbbaffd700 1 --2- 172.21.15.2:0/1288402311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbbc0c3de0 0x7ffbbc1f2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.352+0000 7ffbbaffd700 1 -- 172.21.15.2:0/1288402311 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbbc1f6bd0 con 0x7ffbbc14f5b0 2023-08-25T03:34:43.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.352+0000 7ffbbb7fe700 1 --2- 172.21.15.2:0/1288402311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbbc0c3de0 0x7ffbbc1f2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:43.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.352+0000 7ffbbaffd700 1 --2- 172.21.15.2:0/1288402311 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffbbc14f5b0 0x7ffbbc1f5d60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0011b00 tx=0x7ffbb0011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:43.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.352+0000 7ffba37fe700 1 -- 172.21.15.2:0/1288402311 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb000f9a0 con 0x7ffbbc14f5b0 2023-08-25T03:34:43.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.352+0000 7ffba37fe700 1 -- 172.21.15.2:0/1288402311 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb000fb00 con 0x7ffbbc14f5b0 2023-08-25T03:34:43.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.352+0000 7ffbc1448700 1 -- 172.21.15.2:0/1288402311 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbbc0c4480 con 0x7ffbbc14f5b0 2023-08-25T03:34:43.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.352+0000 7ffba37fe700 1 -- 172.21.15.2:0/1288402311 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb000feb0 con 0x7ffbbc14f5b0 2023-08-25T03:34:43.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.352+0000 7ffbc1448700 1 -- 172.21.15.2:0/1288402311 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbbc0c4a40 con 0x7ffbbc14f5b0 2023-08-25T03:34:43.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.353+0000 7ffbc1448700 1 -- 172.21.15.2:0/1288402311 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbbc0746d0 con 0x7ffbbc14f5b0 2023-08-25T03:34:43.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.354+0000 7ffba37fe700 1 -- 172.21.15.2:0/1288402311 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ffbb0014440 con 0x7ffbbc14f5b0 2023-08-25T03:34:43.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.354+0000 7ffba37fe700 1 --2- 172.21.15.2:0/1288402311 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ffba806a0e0 0x7ffba806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:43.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.354+0000 7ffba37fe700 1 -- 172.21.15.2:0/1288402311 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ffbb0088cf0 con 0x7ffbbc14f5b0 2023-08-25T03:34:43.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.354+0000 7ffbba7fc700 1 --2- 172.21.15.2:0/1288402311 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ffba806a0e0 0x7ffba806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:43.376 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.354+0000 7ffbba7fc700 1 --2- 172.21.15.2:0/1288402311 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ffba806a0e0 0x7ffba806c5a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc0cbd70 tx=0x7ffbac008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:43.376 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.358+0000 7ffba37fe700 1 -- 172.21.15.2:0/1288402311 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ffbb008d070 con 0x7ffbbc14f5b0 2023-08-25T03:34:43.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.563+0000 7ffbc1448700 1 -- 172.21.15.2:0/1288402311 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffbbc071500 con 0x7ffbbc14f5b0 2023-08-25T03:34:43.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.563+0000 7ffba37fe700 1 -- 172.21.15.2:0/1288402311 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffbb0057f30 con 0x7ffbbc14f5b0 2023-08-25T03:34:43.566 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-08-25T03:34:43.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.565+0000 7ffbc1448700 1 -- 172.21.15.2:0/1288402311 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ffba806a0e0 msgr2=0x7ffba806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:43.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.565+0000 7ffbc1448700 1 --2- 172.21.15.2:0/1288402311 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ffba806a0e0 0x7ffba806c5a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc0cbd70 tx=0x7ffbac008040 comp rx=0 tx=0).stop 2023-08-25T03:34:43.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.565+0000 7ffbc1448700 1 -- 172.21.15.2:0/1288402311 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffbbc14f5b0 msgr2=0x7ffbbc1f5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:43.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.565+0000 7ffbc1448700 1 --2- 172.21.15.2:0/1288402311 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffbbc14f5b0 0x7ffbbc1f5d60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0011b00 tx=0x7ffbb0011ec0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.565+0000 7ffbc1448700 1 -- 172.21.15.2:0/1288402311 shutdown_connections 2023-08-25T03:34:43.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.566+0000 7ffbc1448700 1 --2- 172.21.15.2:0/1288402311 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ffba806a0e0 0x7ffba806c5a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.566+0000 7ffbc1448700 1 --2- 172.21.15.2:0/1288402311 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffbbc14f5b0 0x7ffbbc1f5d60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.566+0000 7ffbc1448700 1 --2- 172.21.15.2:0/1288402311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbbc0c3de0 0x7ffbbc1f2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.566+0000 7ffbc1448700 1 --2- 172.21.15.2:0/1288402311 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffbbc1f62a0 0x7ffbbc0c38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.566+0000 7ffbc1448700 1 -- 172.21.15.2:0/1288402311 >> 172.21.15.2:0/1288402311 conn(0x7ffbbc0cfd90 msgr2=0x7ffbbc0ae020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:43.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.566+0000 7ffbc1448700 1 -- 172.21.15.2:0/1288402311 shutdown_connections 2023-08-25T03:34:43.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.566+0000 7ffbc1448700 1 -- 172.21.15.2:0/1288402311 wait complete. 2023-08-25T03:34:43.580 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T03:34:43.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.711+0000 7f6065c9f700 1 Processor -- start 2023-08-25T03:34:43.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.711+0000 7f6065c9f700 1 -- start start 2023-08-25T03:34:43.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.712+0000 7f6065c9f700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f60600af180 0x7f6060070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:43.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.712+0000 7f6065c9f700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f606006a1e0 0x7f606006a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:43.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.712+0000 7f6065c9f700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6060160bd0 con 0x7f60601607f0 2023-08-25T03:34:43.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.712+0000 7f6065c9f700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60600713a0 con 0x7f606006a1e0 2023-08-25T03:34:43.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.712+0000 7f6065c9f700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60600718a0 con 0x7f60600af180 2023-08-25T03:34:43.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.712+0000 7f605effd700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f60600af180 0x7f6060070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:43.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.712+0000 7f605effd700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f60600af180 0x7f6060070e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.2:60774/0 (socket says 172.21.15.2:60774) 2023-08-25T03:34:43.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.712+0000 7f605effd700 1 -- 172.21.15.2:0/1657094761 learned_addr learned my addr 172.21.15.2:0/1657094761 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:43.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.712+0000 7f605f7fe700 1 --1- 172.21.15.2:0/1657094761 >> v1:172.21.15.31:6789/0 conn(0x7f60601607f0 0x7f60600ac8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.2:55218/0 (socket says 172.21.15.2:55218) 2023-08-25T03:34:43.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.712+0000 7f605ffff700 1 --2- 172.21.15.2:0/1657094761 >> v2:172.21.15.31:3300/0 conn(0x7f606006a1e0 0x7f606006a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:43.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.713+0000 7f605effd700 1 -- 172.21.15.2:0/1657094761 >> v2:172.21.15.31:3300/0 conn(0x7f606006a1e0 msgr2=0x7f606006a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:43.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.713+0000 7f605effd700 1 --2- 172.21.15.2:0/1657094761 >> v2:172.21.15.31:3300/0 conn(0x7f606006a1e0 0x7f606006a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.713+0000 7f605effd700 1 -- 172.21.15.2:0/1657094761 >> v1:172.21.15.31:6789/0 conn(0x7f60601607f0 legacy=0x7f60600ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:43.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.713+0000 7f605effd700 1 -- 172.21.15.2:0/1657094761 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60600cdd00 con 0x7f60600af180 2023-08-25T03:34:43.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.713+0000 7f605effd700 1 --2- 172.21.15.2:0/1657094761 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f60600af180 0x7f6060070e60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f60500098f0 tx=0x7f6050009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e55e78fa253aa3ad server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:43.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.713+0000 7f605e7fc700 1 -- 172.21.15.2:0/1657094761 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6050011830 con 0x7f60600af180 2023-08-25T03:34:43.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.713+0000 7f605e7fc700 1 -- 172.21.15.2:0/1657094761 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6050011990 con 0x7f60600af180 2023-08-25T03:34:43.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.713+0000 7f605e7fc700 1 -- 172.21.15.2:0/1657094761 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6050011d20 con 0x7f60600af180 2023-08-25T03:34:43.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.713+0000 7f6065c9f700 1 -- 172.21.15.2:0/1657094761 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f60600af180 msgr2=0x7f6060070e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:43.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.713+0000 7f6065c9f700 1 --2- 172.21.15.2:0/1657094761 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f60600af180 0x7f6060070e60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f60500098f0 tx=0x7f6050009c00 comp rx=0 tx=0).stop 2023-08-25T03:34:43.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.713+0000 7f6065c9f700 1 -- 172.21.15.2:0/1657094761 shutdown_connections 2023-08-25T03:34:43.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.713+0000 7f6065c9f700 1 --2- 172.21.15.2:0/1657094761 >> v2:172.21.15.31:3300/0 conn(0x7f606006a1e0 0x7f606006a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.713+0000 7f6065c9f700 1 --2- 172.21.15.2:0/1657094761 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f60600af180 0x7f6060070e60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.713+0000 7f6065c9f700 1 -- 172.21.15.2:0/1657094761 >> 172.21.15.2:0/1657094761 conn(0x7f60600cfd90 msgr2=0x7f60600d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:43.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.714+0000 7f6065c9f700 1 -- 172.21.15.2:0/1657094761 shutdown_connections 2023-08-25T03:34:43.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.714+0000 7f6065c9f700 1 -- 172.21.15.2:0/1657094761 wait complete. 2023-08-25T03:34:43.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.715+0000 7f6065c9f700 1 Processor -- start 2023-08-25T03:34:43.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.715+0000 7f6065c9f700 1 -- start start 2023-08-25T03:34:43.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.716+0000 7f6065c9f700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f606006a1e0 0x7f606014d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:43.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.716+0000 7f6065c9f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60600af180 0x7f60600c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:43.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.716+0000 7f6065c9f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60601607f0 0x7f60600cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:43.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.716+0000 7f6065c9f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60600c9950 con 0x7f60601607f0 2023-08-25T03:34:43.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.716+0000 7f6065c9f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60600c8cd0 con 0x7f60600af180 2023-08-25T03:34:43.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.716+0000 7f6065c9f700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60600c8fb0 con 0x7f606006a1e0 2023-08-25T03:34:43.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.716+0000 7f605ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60601607f0 0x7f60600cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:43.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.716+0000 7f605ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60601607f0 0x7f60600cb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60538/0 (socket says 172.21.15.2:60538) 2023-08-25T03:34:43.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.716+0000 7f605ffff700 1 -- 172.21.15.2:0/2704701216 learned_addr learned my addr 172.21.15.2:0/2704701216 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:43.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.716+0000 7f605effd700 1 --2- 172.21.15.2:0/2704701216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60600af180 0x7f60600c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:43.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.716+0000 7f605f7fe700 1 --2- 172.21.15.2:0/2704701216 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f606006a1e0 0x7f606014d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:43.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.716+0000 7f605ffff700 1 -- 172.21.15.2:0/2704701216 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f606006a1e0 msgr2=0x7f606014d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:43.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.716+0000 7f605ffff700 1 --2- 172.21.15.2:0/2704701216 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f606006a1e0 0x7f606014d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.716+0000 7f605ffff700 1 -- 172.21.15.2:0/2704701216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60600af180 msgr2=0x7f60600c8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:43.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.716+0000 7f605ffff700 1 --2- 172.21.15.2:0/2704701216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60600af180 0x7f60600c8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.716+0000 7f605ffff700 1 -- 172.21.15.2:0/2704701216 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6050009550 con 0x7f60601607f0 2023-08-25T03:34:43.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.716+0000 7f605f7fe700 1 --2- 172.21.15.2:0/2704701216 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f606006a1e0 0x7f606014d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:43.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.716+0000 7f605effd700 1 --2- 172.21.15.2:0/2704701216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60600af180 0x7f60600c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:43.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.716+0000 7f605ffff700 1 --2- 172.21.15.2:0/2704701216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60601607f0 0x7f60600cb2c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f605800efc0 tx=0x7f605800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:43.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.716+0000 7f6057fff700 1 -- 172.21.15.2:0/2704701216 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6058009ba0 con 0x7f60601607f0 2023-08-25T03:34:43.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.716+0000 7f6057fff700 1 -- 172.21.15.2:0/2704701216 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f605800f040 con 0x7f60601607f0 2023-08-25T03:34:43.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.716+0000 7f6065c9f700 1 -- 172.21.15.2:0/2704701216 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60600c9290 con 0x7f60601607f0 2023-08-25T03:34:43.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.717+0000 7f6057fff700 1 -- 172.21.15.2:0/2704701216 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6058014480 con 0x7f60601607f0 2023-08-25T03:34:43.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.717+0000 7f6065c9f700 1 -- 172.21.15.2:0/2704701216 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f606014d6b0 con 0x7f60601607f0 2023-08-25T03:34:43.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.718+0000 7f6065c9f700 1 -- 172.21.15.2:0/2704701216 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60600c9a90 con 0x7f60601607f0 2023-08-25T03:34:43.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.718+0000 7f6057fff700 1 -- 172.21.15.2:0/2704701216 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f60580145e0 con 0x7f60601607f0 2023-08-25T03:34:43.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.718+0000 7f6057fff700 1 --2- 172.21.15.2:0/2704701216 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f604006a0e0 0x7f604006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:43.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.719+0000 7f6057fff700 1 -- 172.21.15.2:0/2704701216 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f6058088e60 con 0x7f60601607f0 2023-08-25T03:34:43.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.719+0000 7f605f7fe700 1 --2- 172.21.15.2:0/2704701216 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f604006a0e0 0x7f604006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:43.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.719+0000 7f605f7fe700 1 --2- 172.21.15.2:0/2704701216 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f604006a0e0 0x7f604006c5a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6048000f60 tx=0x7f6048009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:43.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.723+0000 7f6057fff700 1 -- 172.21.15.2:0/2704701216 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f60600c9a90 con 0x7f60601607f0 2023-08-25T03:34:43.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.924+0000 7f6065c9f700 1 -- 172.21.15.2:0/2704701216 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f60600c9a90 con 0x7f60601607f0 2023-08-25T03:34:43.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.925+0000 7f6057fff700 1 -- 172.21.15.2:0/2704701216 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f60600c9a90 con 0x7f60601607f0 2023-08-25T03:34:43.927 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 8 2023-08-25T03:34:43.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.926+0000 7f6065c9f700 1 -- 172.21.15.2:0/2704701216 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f604006a0e0 msgr2=0x7f604006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:43.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.926+0000 7f6065c9f700 1 --2- 172.21.15.2:0/2704701216 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f604006a0e0 0x7f604006c5a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6048000f60 tx=0x7f6048009040 comp rx=0 tx=0).stop 2023-08-25T03:34:43.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.926+0000 7f6065c9f700 1 -- 172.21.15.2:0/2704701216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60601607f0 msgr2=0x7f60600cb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:43.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.926+0000 7f6065c9f700 1 --2- 172.21.15.2:0/2704701216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60601607f0 0x7f60600cb2c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f605800efc0 tx=0x7f605800c3a0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.927+0000 7f6065c9f700 1 -- 172.21.15.2:0/2704701216 shutdown_connections 2023-08-25T03:34:43.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.927+0000 7f6065c9f700 1 --2- 172.21.15.2:0/2704701216 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f604006a0e0 0x7f604006c5a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.927+0000 7f6065c9f700 1 --2- 172.21.15.2:0/2704701216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f60601607f0 0x7f60600cb2c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.927+0000 7f6065c9f700 1 --2- 172.21.15.2:0/2704701216 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f606006a1e0 0x7f606014d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.927+0000 7f6065c9f700 1 --2- 172.21.15.2:0/2704701216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f60600af180 0x7f60600c8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:43.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.927+0000 7f6065c9f700 1 -- 172.21.15.2:0/2704701216 >> 172.21.15.2:0/2704701216 conn(0x7f60600cfd90 msgr2=0x7f60600cce60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:43.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.927+0000 7f6065c9f700 1 -- 172.21.15.2:0/2704701216 shutdown_connections 2023-08-25T03:34:43.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:43.927+0000 7f6065c9f700 1 -- 172.21.15.2:0/2704701216 wait complete. 2023-08-25T03:34:43.940 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T03:34:44.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.067+0000 7f52a8d62700 1 Processor -- start 2023-08-25T03:34:44.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.067+0000 7f52a8d62700 1 -- start start 2023-08-25T03:34:44.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.068+0000 7f52a8d62700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f52a4159f10 0x7f52a415dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:44.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.068+0000 7f52a8d62700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52a406eaf0 con 0x7f52a40cedf0 2023-08-25T03:34:44.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.068+0000 7f52a8d62700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52a415e020 con 0x7f52a40cacf0 2023-08-25T03:34:44.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.068+0000 7f52a8d62700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52a406e850 con 0x7f52a4159f10 2023-08-25T03:34:44.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.069+0000 7f52a259c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f52a4159f10 0x7f52a415dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:44.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.069+0000 7f52a1d9b700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f52a40cedf0 0x7f52a406ef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47210/0 (socket says 172.21.15.2:47210) 2023-08-25T03:34:44.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.069+0000 7f52a1d9b700 1 -- 172.21.15.2:0/161823740 learned_addr learned my addr 172.21.15.2:0/161823740 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:44.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.069+0000 7f52a259c700 1 -- 172.21.15.2:0/161823740 >> v1:172.21.15.31:6789/0 conn(0x7f52a40cacf0 legacy=0x7f52a406cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:44.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.069+0000 7f52a259c700 1 -- 172.21.15.2:0/161823740 >> v1:172.21.15.2:6789/0 conn(0x7f52a40cedf0 legacy=0x7f52a406ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:44.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.069+0000 7f52a259c700 1 -- 172.21.15.2:0/161823740 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a415e2c0 con 0x7f52a4159f10 2023-08-25T03:34:44.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.069+0000 7f52a259c700 1 --2- 172.21.15.2:0/161823740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52a4159f10 0x7f52a415dae0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f529800bae0 tx=0x7f529800bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19edbb4a258f9f49 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:44.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.069+0000 7f52a159a700 1 -- 172.21.15.2:0/161823740 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529800dd30 con 0x7f52a4159f10 2023-08-25T03:34:44.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.069+0000 7f52a159a700 1 -- 172.21.15.2:0/161823740 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529800de90 con 0x7f52a4159f10 2023-08-25T03:34:44.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.069+0000 7f52a159a700 1 -- 172.21.15.2:0/161823740 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5298010690 con 0x7f52a4159f10 2023-08-25T03:34:44.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.070+0000 7f52a8d62700 1 -- 172.21.15.2:0/161823740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52a4159f10 msgr2=0x7f52a415dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:44.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.070+0000 7f52a8d62700 1 --2- 172.21.15.2:0/161823740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52a4159f10 0x7f52a415dae0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f529800bae0 tx=0x7f529800bdf0 comp rx=0 tx=0).stop 2023-08-25T03:34:44.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.070+0000 7f52a8d62700 1 -- 172.21.15.2:0/161823740 shutdown_connections 2023-08-25T03:34:44.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.070+0000 7f52a8d62700 1 --2- 172.21.15.2:0/161823740 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52a4159f10 0x7f52a415dae0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:44.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.070+0000 7f52a8d62700 1 -- 172.21.15.2:0/161823740 >> 172.21.15.2:0/161823740 conn(0x7f52a40cfd90 msgr2=0x7f52a40d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:44.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.070+0000 7f52a8d62700 1 -- 172.21.15.2:0/161823740 shutdown_connections 2023-08-25T03:34:44.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.070+0000 7f52a8d62700 1 -- 172.21.15.2:0/161823740 wait complete. 2023-08-25T03:34:44.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.071+0000 7f52a8d62700 1 Processor -- start 2023-08-25T03:34:44.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.071+0000 7f52a8d62700 1 -- start start 2023-08-25T03:34:44.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.072+0000 7f52a8d62700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a40cacf0 0x7f52a40c3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:44.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.072+0000 7f52a8d62700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52a40cedf0 0x7f52a41f54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:44.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.072+0000 7f52a8d62700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f52a4159f10 0x7f52a41f5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:44.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.072+0000 7f52a8d62700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a41573c0 con 0x7f52a40cedf0 2023-08-25T03:34:44.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.072+0000 7f52a8d62700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52a41f5f70 con 0x7f52a40cacf0 2023-08-25T03:34:44.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.072+0000 7f52a8d62700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52a4156a60 con 0x7f52a4159f10 2023-08-25T03:34:44.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.072+0000 7f52a1d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52a40cedf0 0x7f52a41f54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:44.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.072+0000 7f52a1d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52a40cedf0 0x7f52a41f54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60550/0 (socket says 172.21.15.2:60550) 2023-08-25T03:34:44.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.072+0000 7f52a1d9b700 1 -- 172.21.15.2:0/861264194 learned_addr learned my addr 172.21.15.2:0/861264194 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:44.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.072+0000 7f52a259c700 1 --2- 172.21.15.2:0/861264194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a40cacf0 0x7f52a40c3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:44.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.072+0000 7f52a2d9d700 1 --2- 172.21.15.2:0/861264194 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f52a4159f10 0x7f52a41f5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:44.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.072+0000 7f52a1d9b700 1 -- 172.21.15.2:0/861264194 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f52a4159f10 msgr2=0x7f52a41f5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:44.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.072+0000 7f52a1d9b700 1 --2- 172.21.15.2:0/861264194 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f52a4159f10 0x7f52a41f5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:44.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.072+0000 7f52a1d9b700 1 -- 172.21.15.2:0/861264194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a40cacf0 msgr2=0x7f52a40c3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:44.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.072+0000 7f52a1d9b700 1 --2- 172.21.15.2:0/861264194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a40cacf0 0x7f52a40c3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:44.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.072+0000 7f52a1d9b700 1 -- 172.21.15.2:0/861264194 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529800b740 con 0x7f52a40cedf0 2023-08-25T03:34:44.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.073+0000 7f52a1d9b700 1 --2- 172.21.15.2:0/861264194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52a40cedf0 0x7f52a41f54f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f529400c520 tx=0x7f529400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:44.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.073+0000 7f5292ffd700 1 -- 172.21.15.2:0/861264194 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5294005d70 con 0x7f52a40cedf0 2023-08-25T03:34:44.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.073+0000 7f5292ffd700 1 -- 172.21.15.2:0/861264194 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52940070d0 con 0x7f52a40cedf0 2023-08-25T03:34:44.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.073+0000 7f52a8d62700 1 -- 172.21.15.2:0/861264194 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a40c66f0 con 0x7f52a40cedf0 2023-08-25T03:34:44.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.073+0000 7f5292ffd700 1 -- 172.21.15.2:0/861264194 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5294007440 con 0x7f52a40cedf0 2023-08-25T03:34:44.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.073+0000 7f52a8d62700 1 -- 172.21.15.2:0/861264194 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52a4156fe0 con 0x7f52a40cedf0 2023-08-25T03:34:44.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.074+0000 7f52a8d62700 1 -- 172.21.15.2:0/861264194 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52a4157500 con 0x7f52a40cedf0 2023-08-25T03:34:44.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.074+0000 7f5292ffd700 1 -- 172.21.15.2:0/861264194 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f52940075a0 con 0x7f52a40cedf0 2023-08-25T03:34:44.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.075+0000 7f5292ffd700 1 --2- 172.21.15.2:0/861264194 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f528406a0e0 0x7f528406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:44.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.075+0000 7f5292ffd700 1 -- 172.21.15.2:0/861264194 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f5294087890 con 0x7f52a40cedf0 2023-08-25T03:34:44.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.075+0000 7f52a259c700 1 --2- 172.21.15.2:0/861264194 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f528406a0e0 0x7f528406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:44.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.075+0000 7f52a259c700 1 --2- 172.21.15.2:0/861264194 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f528406a0e0 0x7f528406c5a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f529800b2c0 tx=0x7f529801c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:44.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.079+0000 7f5292ffd700 1 -- 172.21.15.2:0/861264194 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f52a4157500 con 0x7f52a40cedf0 2023-08-25T03:34:44.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.280+0000 7f52a8d62700 1 -- 172.21.15.2:0/861264194 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f52a4157500 con 0x7f52a40cedf0 2023-08-25T03:34:44.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.280+0000 7f5292ffd700 1 -- 172.21.15.2:0/861264194 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f52a4157500 con 0x7f52a40cedf0 2023-08-25T03:34:44.283 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-08-25T03:34:44.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.282+0000 7f52a8d62700 1 -- 172.21.15.2:0/861264194 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f528406a0e0 msgr2=0x7f528406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:44.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.282+0000 7f52a8d62700 1 --2- 172.21.15.2:0/861264194 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f528406a0e0 0x7f528406c5a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f529800b2c0 tx=0x7f529801c040 comp rx=0 tx=0).stop 2023-08-25T03:34:44.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.282+0000 7f52a8d62700 1 -- 172.21.15.2:0/861264194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52a40cedf0 msgr2=0x7f52a41f54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:44.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.282+0000 7f52a8d62700 1 --2- 172.21.15.2:0/861264194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52a40cedf0 0x7f52a41f54f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f529400c520 tx=0x7f529400c830 comp rx=0 tx=0).stop 2023-08-25T03:34:44.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.283+0000 7f52a8d62700 1 -- 172.21.15.2:0/861264194 shutdown_connections 2023-08-25T03:34:44.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.283+0000 7f52a8d62700 1 --2- 172.21.15.2:0/861264194 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f52a4159f10 0x7f52a41f5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:44.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.283+0000 7f52a8d62700 1 --2- 172.21.15.2:0/861264194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52a40cacf0 0x7f52a40c3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:44.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.283+0000 7f52a8d62700 1 --2- 172.21.15.2:0/861264194 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f528406a0e0 0x7f528406c5a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:44.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.283+0000 7f52a8d62700 1 --2- 172.21.15.2:0/861264194 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f52a40cedf0 0x7f52a41f54f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:44.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.283+0000 7f52a8d62700 1 -- 172.21.15.2:0/861264194 >> 172.21.15.2:0/861264194 conn(0x7f52a40cfd90 msgr2=0x7f52a40cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:44.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.283+0000 7f52a8d62700 1 -- 172.21.15.2:0/861264194 shutdown_connections 2023-08-25T03:34:44.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.283+0000 7f52a8d62700 1 -- 172.21.15.2:0/861264194 wait complete. 2023-08-25T03:34:44.297 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T03:34:44.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.423+0000 7f1f51e64700 1 Processor -- start 2023-08-25T03:34:44.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.423+0000 7f1f51e64700 1 -- start start 2023-08-25T03:34:44.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.424+0000 7f1f51e64700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1f4c159f10 0x7f1f4c15dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:44.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.424+0000 7f1f51e64700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f1f4c0cedf0 0x7f1f4c06ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:44.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.424+0000 7f1f51e64700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1f4c06f4c0 0x7f1f4c0cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:44.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.424+0000 7f1f51e64700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f4c06e850 con 0x7f1f4c0cedf0 2023-08-25T03:34:44.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.424+0000 7f1f51e64700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f4c06eaf0 con 0x7f1f4c06f4c0 2023-08-25T03:34:44.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.424+0000 7f1f51e64700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c15e020 con 0x7f1f4c159f10 2023-08-25T03:34:44.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.424+0000 7f1f4affd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f1f4c0cedf0 0x7f1f4c06ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:44.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.424+0000 7f1f4affd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f1f4c0cedf0 0x7f1f4c06ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60558/0 (socket says 172.21.15.2:60558) 2023-08-25T03:34:44.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.424+0000 7f1f4affd700 1 -- 172.21.15.2:0/2798879745 learned_addr learned my addr 172.21.15.2:0/2798879745 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:44.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.425+0000 7f1f4bfff700 1 --2- 172.21.15.2:0/2798879745 >> v2:172.21.15.31:3300/0 conn(0x7f1f4c06f4c0 0x7f1f4c0cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:44.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.425+0000 7f1f4b7fe700 1 --2- 172.21.15.2:0/2798879745 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1f4c159f10 0x7f1f4c15dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:44.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.425+0000 7f1f4affd700 1 -- 172.21.15.2:0/2798879745 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1f4c159f10 msgr2=0x7f1f4c15dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:44.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.425+0000 7f1f4affd700 1 --2- 172.21.15.2:0/2798879745 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1f4c159f10 0x7f1f4c15dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:44.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.425+0000 7f1f4affd700 1 -- 172.21.15.2:0/2798879745 >> v2:172.21.15.31:3300/0 conn(0x7f1f4c06f4c0 msgr2=0x7f1f4c0cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:44.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.425+0000 7f1f4affd700 1 --2- 172.21.15.2:0/2798879745 >> v2:172.21.15.31:3300/0 conn(0x7f1f4c06f4c0 0x7f1f4c0cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:44.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.425+0000 7f1f4affd700 1 -- 172.21.15.2:0/2798879745 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c15e2c0 con 0x7f1f4c0cedf0 2023-08-25T03:34:44.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.425+0000 7f1f4affd700 1 --2- 172.21.15.2:0/2798879745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f4c0cedf0 0x7f1f4c06ef80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0cfb00 tx=0x7f1f34009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7203a4dad2a31b4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:44.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.425+0000 7f1f4a7fc700 1 -- 172.21.15.2:0/2798879745 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3400b870 con 0x7f1f4c0cedf0 2023-08-25T03:34:44.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.425+0000 7f1f4a7fc700 1 -- 172.21.15.2:0/2798879745 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3400b9d0 con 0x7f1f4c0cedf0 2023-08-25T03:34:44.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.425+0000 7f1f4a7fc700 1 -- 172.21.15.2:0/2798879745 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3400bd60 con 0x7f1f4c0cedf0 2023-08-25T03:34:44.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.426+0000 7f1f51e64700 1 -- 172.21.15.2:0/2798879745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f4c0cedf0 msgr2=0x7f1f4c06ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:44.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.426+0000 7f1f51e64700 1 --2- 172.21.15.2:0/2798879745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f4c0cedf0 0x7f1f4c06ef80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c0cfb00 tx=0x7f1f34009bd0 comp rx=0 tx=0).stop 2023-08-25T03:34:44.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.426+0000 7f1f51e64700 1 -- 172.21.15.2:0/2798879745 shutdown_connections 2023-08-25T03:34:44.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.426+0000 7f1f51e64700 1 --2- 172.21.15.2:0/2798879745 >> v2:172.21.15.31:3300/0 conn(0x7f1f4c06f4c0 0x7f1f4c0cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:44.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.426+0000 7f1f51e64700 1 --2- 172.21.15.2:0/2798879745 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1f4c159f10 0x7f1f4c15dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:44.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.426+0000 7f1f51e64700 1 --2- 172.21.15.2:0/2798879745 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f4c0cedf0 0x7f1f4c06ef80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:44.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.426+0000 7f1f51e64700 1 -- 172.21.15.2:0/2798879745 >> 172.21.15.2:0/2798879745 conn(0x7f1f4c0cfd90 msgr2=0x7f1f4c0d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:44.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.426+0000 7f1f51e64700 1 -- 172.21.15.2:0/2798879745 shutdown_connections 2023-08-25T03:34:44.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.426+0000 7f1f51e64700 1 -- 172.21.15.2:0/2798879745 wait complete. 2023-08-25T03:34:44.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.427+0000 7f1f51e64700 1 Processor -- start 2023-08-25T03:34:44.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.427+0000 7f1f51e64700 1 -- start start 2023-08-25T03:34:44.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.428+0000 7f1f51e64700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1f4c06f4c0 0x7f1f4c0b4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:44.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.428+0000 7f1f51e64700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f4c0cedf0 0x7f1f4c0ad350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:44.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.428+0000 7f1f51e64700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f4c159f10 0x7f1f4c14f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:44.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.428+0000 7f1f51e64700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c0ac8e0 con 0x7f1f4c0cedf0 2023-08-25T03:34:44.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.428+0000 7f1f51e64700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c0ad0f0 con 0x7f1f4c159f10 2023-08-25T03:34:44.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.428+0000 7f1f51e64700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c0acb30 con 0x7f1f4c06f4c0 2023-08-25T03:34:44.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.428+0000 7f1f4affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f4c0cedf0 0x7f1f4c0ad350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:44.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.428+0000 7f1f4affd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f4c0cedf0 0x7f1f4c0ad350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60562/0 (socket says 172.21.15.2:60562) 2023-08-25T03:34:44.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.428+0000 7f1f4affd700 1 -- 172.21.15.2:0/1991582625 learned_addr learned my addr 172.21.15.2:0/1991582625 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:44.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.428+0000 7f1f4bfff700 1 --2- 172.21.15.2:0/1991582625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f4c159f10 0x7f1f4c14f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:44.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.428+0000 7f1f4b7fe700 1 --2- 172.21.15.2:0/1991582625 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1f4c06f4c0 0x7f1f4c0b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:44.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.428+0000 7f1f4affd700 1 -- 172.21.15.2:0/1991582625 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1f4c06f4c0 msgr2=0x7f1f4c0b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:44.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.428+0000 7f1f4affd700 1 --2- 172.21.15.2:0/1991582625 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1f4c06f4c0 0x7f1f4c0b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:44.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.428+0000 7f1f4affd700 1 -- 172.21.15.2:0/1991582625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f4c159f10 msgr2=0x7f1f4c14f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:44.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.428+0000 7f1f4affd700 1 --2- 172.21.15.2:0/1991582625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f4c159f10 0x7f1f4c14f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:44.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.428+0000 7f1f4affd700 1 -- 172.21.15.2:0/1991582625 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f34009550 con 0x7f1f4c0cedf0 2023-08-25T03:34:44.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.428+0000 7f1f4bfff700 1 --2- 172.21.15.2:0/1991582625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f4c159f10 0x7f1f4c14f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:44.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.428+0000 7f1f4b7fe700 1 --2- 172.21.15.2:0/1991582625 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1f4c06f4c0 0x7f1f4c0b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:44.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.428+0000 7f1f4affd700 1 --2- 172.21.15.2:0/1991582625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f4c0cedf0 0x7f1f4c0ad350 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1f34000c00 tx=0x7f1f34004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:44.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.429+0000 7f1f43fff700 1 -- 172.21.15.2:0/1991582625 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f34004540 con 0x7f1f4c0cedf0 2023-08-25T03:34:44.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.429+0000 7f1f43fff700 1 -- 172.21.15.2:0/1991582625 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f340036a0 con 0x7f1f4c0cedf0 2023-08-25T03:34:44.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.429+0000 7f1f43fff700 1 -- 172.21.15.2:0/1991582625 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f34003a50 con 0x7f1f4c0cedf0 2023-08-25T03:34:44.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.429+0000 7f1f51e64700 1 -- 172.21.15.2:0/1991582625 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f4c0ace10 con 0x7f1f4c0cedf0 2023-08-25T03:34:44.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.429+0000 7f1f51e64700 1 -- 172.21.15.2:0/1991582625 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f4c0ad890 con 0x7f1f4c0cedf0 2023-08-25T03:34:44.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.430+0000 7f1f51e64700 1 -- 172.21.15.2:0/1991582625 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f4c0b3770 con 0x7f1f4c0cedf0 2023-08-25T03:34:44.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.430+0000 7f1f43fff700 1 -- 172.21.15.2:0/1991582625 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1f34021070 con 0x7f1f4c0cedf0 2023-08-25T03:34:44.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.430+0000 7f1f43fff700 1 --2- 172.21.15.2:0/1991582625 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1f2c06a0e0 0x7f1f2c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:44.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.430+0000 7f1f43fff700 1 -- 172.21.15.2:0/1991582625 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f1f34089c80 con 0x7f1f4c0cedf0 2023-08-25T03:34:44.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.431+0000 7f1f4b7fe700 1 --2- 172.21.15.2:0/1991582625 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1f2c06a0e0 0x7f1f2c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:44.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.431+0000 7f1f4b7fe700 1 --2- 172.21.15.2:0/1991582625 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1f2c06a0e0 0x7f1f2c06c5a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009a00 tx=0x7f1f3c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:44.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.436+0000 7f1f43fff700 1 -- 172.21.15.2:0/1991582625 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f1f4c0b3770 con 0x7f1f4c0cedf0 2023-08-25T03:34:44.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.637+0000 7f1f51e64700 1 -- 172.21.15.2:0/1991582625 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1f4c0b3770 con 0x7f1f4c0cedf0 2023-08-25T03:34:44.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.637+0000 7f1f43fff700 1 -- 172.21.15.2:0/1991582625 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1f4c0b3770 con 0x7f1f4c0cedf0 2023-08-25T03:34:44.643 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-08-25T03:34:44.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.639+0000 7f1f51e64700 1 -- 172.21.15.2:0/1991582625 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1f2c06a0e0 msgr2=0x7f1f2c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:44.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.639+0000 7f1f51e64700 1 --2- 172.21.15.2:0/1991582625 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1f2c06a0e0 0x7f1f2c06c5a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c009a00 tx=0x7f1f3c008040 comp rx=0 tx=0).stop 2023-08-25T03:34:44.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.639+0000 7f1f51e64700 1 -- 172.21.15.2:0/1991582625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f4c0cedf0 msgr2=0x7f1f4c0ad350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:44.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.639+0000 7f1f51e64700 1 --2- 172.21.15.2:0/1991582625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f4c0cedf0 0x7f1f4c0ad350 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1f34000c00 tx=0x7f1f34004750 comp rx=0 tx=0).stop 2023-08-25T03:34:44.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.639+0000 7f1f51e64700 1 -- 172.21.15.2:0/1991582625 shutdown_connections 2023-08-25T03:34:44.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.639+0000 7f1f51e64700 1 --2- 172.21.15.2:0/1991582625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1f4c159f10 0x7f1f4c14f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:44.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.639+0000 7f1f51e64700 1 --2- 172.21.15.2:0/1991582625 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1f4c06f4c0 0x7f1f4c0b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:44.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.639+0000 7f1f51e64700 1 --2- 172.21.15.2:0/1991582625 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1f2c06a0e0 0x7f1f2c06c5a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:44.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.639+0000 7f1f51e64700 1 --2- 172.21.15.2:0/1991582625 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1f4c0cedf0 0x7f1f4c0ad350 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:44.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.640+0000 7f1f51e64700 1 -- 172.21.15.2:0/1991582625 >> 172.21.15.2:0/1991582625 conn(0x7f1f4c0cfd90 msgr2=0x7f1f4c0b55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:44.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.640+0000 7f1f51e64700 1 -- 172.21.15.2:0/1991582625 shutdown_connections 2023-08-25T03:34:44.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.640+0000 7f1f51e64700 1 -- 172.21.15.2:0/1991582625 wait complete. 2023-08-25T03:34:44.654 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T03:34:44.780 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.779+0000 7ffa4f24d700 1 Processor -- start 2023-08-25T03:34:44.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.779+0000 7ffa4f24d700 1 -- start start 2023-08-25T03:34:44.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.779+0000 7ffa4f24d700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffa4815d370 0x7ffa480d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:44.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.779+0000 7ffa4f24d700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ffa480b8aa0 0x7ffa480cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:44.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.779+0000 7ffa4f24d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa4815d750 con 0x7ffa480d2b40 2023-08-25T03:34:44.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.779+0000 7ffa4f24d700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa480d1aa0 con 0x7ffa480b8aa0 2023-08-25T03:34:44.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.779+0000 7ffa4f24d700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa480d1d80 con 0x7ffa4815d370 2023-08-25T03:34:44.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.780+0000 7ffa47fff700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ffa480b8aa0 0x7ffa480cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:44.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.780+0000 7ffa47fff700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ffa480b8aa0 0x7ffa480cdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60570/0 (socket says 172.21.15.2:60570) 2023-08-25T03:34:44.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.780+0000 7ffa47fff700 1 -- 172.21.15.2:0/2928452769 learned_addr learned my addr 172.21.15.2:0/2928452769 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:44.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.780+0000 7ffa47fff700 1 -- 172.21.15.2:0/2928452769 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffa4815d370 msgr2=0x7ffa480d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:44.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.780+0000 7ffa4cfe9700 1 --2- 172.21.15.2:0/2928452769 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffa4815d370 0x7ffa480d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:44.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.780+0000 7ffa47fff700 1 --2- 172.21.15.2:0/2928452769 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffa4815d370 0x7ffa480d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:44.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.780+0000 7ffa47fff700 1 -- 172.21.15.2:0/2928452769 >> v1:172.21.15.31:6789/0 conn(0x7ffa480d2b40 legacy=0x7ffa480b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:44.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.780+0000 7ffa47fff700 1 -- 172.21.15.2:0/2928452769 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa480d2060 con 0x7ffa480b8aa0 2023-08-25T03:34:44.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.780+0000 7ffa4cfe9700 1 --2- 172.21.15.2:0/2928452769 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffa4815d370 0x7ffa480d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:44.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.780+0000 7ffa47fff700 1 --2- 172.21.15.2:0/2928452769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa480b8aa0 0x7ffa480cdde0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ffa340098f0 tx=0x7ffa34009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ce10c20e27e738 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:44.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.780+0000 7ffa477fe700 1 -- 172.21.15.2:0/2928452769 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3400b8a0 con 0x7ffa480b8aa0 2023-08-25T03:34:44.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.780+0000 7ffa477fe700 1 -- 172.21.15.2:0/2928452769 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3400ba00 con 0x7ffa480b8aa0 2023-08-25T03:34:44.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.780+0000 7ffa477fe700 1 -- 172.21.15.2:0/2928452769 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3400bd90 con 0x7ffa480b8aa0 2023-08-25T03:34:44.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.781+0000 7ffa4f24d700 1 -- 172.21.15.2:0/2928452769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa480b8aa0 msgr2=0x7ffa480cdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:44.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.781+0000 7ffa4f24d700 1 --2- 172.21.15.2:0/2928452769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa480b8aa0 0x7ffa480cdde0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ffa340098f0 tx=0x7ffa34009c00 comp rx=0 tx=0).stop 2023-08-25T03:34:44.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.781+0000 7ffa4f24d700 1 -- 172.21.15.2:0/2928452769 shutdown_connections 2023-08-25T03:34:44.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.781+0000 7ffa4f24d700 1 --2- 172.21.15.2:0/2928452769 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffa4815d370 0x7ffa480d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:44.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.781+0000 7ffa4f24d700 1 --2- 172.21.15.2:0/2928452769 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa480b8aa0 0x7ffa480cdde0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:44.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.781+0000 7ffa4f24d700 1 -- 172.21.15.2:0/2928452769 >> 172.21.15.2:0/2928452769 conn(0x7ffa4806ab50 msgr2=0x7ffa4806af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:44.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.781+0000 7ffa4f24d700 1 -- 172.21.15.2:0/2928452769 shutdown_connections 2023-08-25T03:34:44.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.781+0000 7ffa4f24d700 1 -- 172.21.15.2:0/2928452769 wait complete. 2023-08-25T03:34:44.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.782+0000 7ffa4f24d700 1 Processor -- start 2023-08-25T03:34:44.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.782+0000 7ffa4f24d700 1 -- start start 2023-08-25T03:34:44.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.783+0000 7ffa4f24d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa480b8aa0 0x7ffa481f9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:44.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.783+0000 7ffa4f24d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa480d2b40 0x7ffa480b6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:44.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.783+0000 7ffa4f24d700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffa4815d370 0x7ffa480b67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:44.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.783+0000 7ffa4f24d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa48156970 con 0x7ffa480d2b40 2023-08-25T03:34:44.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.783+0000 7ffa4f24d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa48157cd0 con 0x7ffa480b8aa0 2023-08-25T03:34:44.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.783+0000 7ffa4f24d700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4815fb10 con 0x7ffa4815d370 2023-08-25T03:34:44.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.783+0000 7ffa47fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa480d2b40 0x7ffa480b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:44.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.783+0000 7ffa47fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa480d2b40 0x7ffa480b6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60578/0 (socket says 172.21.15.2:60578) 2023-08-25T03:34:44.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.783+0000 7ffa47fff700 1 -- 172.21.15.2:0/1640559316 learned_addr learned my addr 172.21.15.2:0/1640559316 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:44.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.783+0000 7ffa4cfe9700 1 --2- 172.21.15.2:0/1640559316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa480b8aa0 0x7ffa481f9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:44.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.783+0000 7ffa4d7ea700 1 --2- 172.21.15.2:0/1640559316 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffa4815d370 0x7ffa480b67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:44.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.783+0000 7ffa47fff700 1 -- 172.21.15.2:0/1640559316 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffa4815d370 msgr2=0x7ffa480b67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:44.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.783+0000 7ffa47fff700 1 --2- 172.21.15.2:0/1640559316 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffa4815d370 0x7ffa480b67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:44.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.783+0000 7ffa47fff700 1 -- 172.21.15.2:0/1640559316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa480b8aa0 msgr2=0x7ffa481f9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:44.794 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.783+0000 7ffa47fff700 1 --2- 172.21.15.2:0/1640559316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa480b8aa0 0x7ffa481f9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:44.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.783+0000 7ffa47fff700 1 -- 172.21.15.2:0/1640559316 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa34009550 con 0x7ffa480d2b40 2023-08-25T03:34:44.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.784+0000 7ffa47fff700 1 --2- 172.21.15.2:0/1640559316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa480d2b40 0x7ffa480b6270 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ffa34014040 tx=0x7ffa34004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:44.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.784+0000 7ffa457fa700 1 -- 172.21.15.2:0/1640559316 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa340036a0 con 0x7ffa480d2b40 2023-08-25T03:34:44.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.784+0000 7ffa457fa700 1 -- 172.21.15.2:0/1640559316 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3401e070 con 0x7ffa480d2b40 2023-08-25T03:34:44.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.784+0000 7ffa4f24d700 1 -- 172.21.15.2:0/1640559316 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa48155fd0 con 0x7ffa480d2b40 2023-08-25T03:34:44.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.784+0000 7ffa457fa700 1 -- 172.21.15.2:0/1640559316 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa34003a40 con 0x7ffa480d2b40 2023-08-25T03:34:44.795 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.784+0000 7ffa4f24d700 1 -- 172.21.15.2:0/1640559316 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa48156590 con 0x7ffa480d2b40 2023-08-25T03:34:44.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.785+0000 7ffa4f24d700 1 -- 172.21.15.2:0/1640559316 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa48156ab0 con 0x7ffa480d2b40 2023-08-25T03:34:44.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.785+0000 7ffa457fa700 1 -- 172.21.15.2:0/1640559316 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ffa34003ba0 con 0x7ffa480d2b40 2023-08-25T03:34:44.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.786+0000 7ffa457fa700 1 --2- 172.21.15.2:0/1640559316 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ffa2c06a0e0 0x7ffa2c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:44.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.786+0000 7ffa457fa700 1 -- 172.21.15.2:0/1640559316 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ffa34020070 con 0x7ffa480d2b40 2023-08-25T03:34:44.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.786+0000 7ffa4cfe9700 1 --2- 172.21.15.2:0/1640559316 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ffa2c06a0e0 0x7ffa2c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:44.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.786+0000 7ffa4cfe9700 1 --2- 172.21.15.2:0/1640559316 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ffa2c06a0e0 0x7ffa2c06c5a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c006fd0 tx=0x7ffa3c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:44.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:44.791+0000 7ffa457fa700 1 -- 172.21.15.2:0/1640559316 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ffa48156ab0 con 0x7ffa480d2b40 2023-08-25T03:34:45.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.008+0000 7ffa4f24d700 1 -- 172.21.15.2:0/1640559316 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ffa48156ab0 con 0x7ffa480d2b40 2023-08-25T03:34:45.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.009+0000 7ffa457fa700 1 -- 172.21.15.2:0/1640559316 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7ffa48156ab0 con 0x7ffa480d2b40 2023-08-25T03:34:45.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.011+0000 7ffa4f24d700 1 -- 172.21.15.2:0/1640559316 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ffa2c06a0e0 msgr2=0x7ffa2c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:45.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.011+0000 7ffa4f24d700 1 --2- 172.21.15.2:0/1640559316 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ffa2c06a0e0 0x7ffa2c06c5a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c006fd0 tx=0x7ffa3c008040 comp rx=0 tx=0).stop 2023-08-25T03:34:45.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.011+0000 7ffa4f24d700 1 -- 172.21.15.2:0/1640559316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa480d2b40 msgr2=0x7ffa480b6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:45.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.011+0000 7ffa4f24d700 1 --2- 172.21.15.2:0/1640559316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa480d2b40 0x7ffa480b6270 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ffa34014040 tx=0x7ffa34004750 comp rx=0 tx=0).stop 2023-08-25T03:34:45.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.011+0000 7ffa4f24d700 1 -- 172.21.15.2:0/1640559316 shutdown_connections 2023-08-25T03:34:45.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.011+0000 7ffa4f24d700 1 --2- 172.21.15.2:0/1640559316 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffa4815d370 0x7ffa480b67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:45.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.011+0000 7ffa4f24d700 1 --2- 172.21.15.2:0/1640559316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa480b8aa0 0x7ffa481f9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:45.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.011+0000 7ffa4f24d700 1 --2- 172.21.15.2:0/1640559316 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ffa2c06a0e0 0x7ffa2c06c5a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:45.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.011+0000 7ffa4f24d700 1 --2- 172.21.15.2:0/1640559316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ffa480d2b40 0x7ffa480b6270 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:45.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.011+0000 7ffa4f24d700 1 -- 172.21.15.2:0/1640559316 >> 172.21.15.2:0/1640559316 conn(0x7ffa4806ab50 msgr2=0x7ffa4815c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:45.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.011+0000 7ffa4f24d700 1 -- 172.21.15.2:0/1640559316 shutdown_connections 2023-08-25T03:34:45.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.011+0000 7ffa4f24d700 1 -- 172.21.15.2:0/1640559316 wait complete. 2023-08-25T03:34:45.020 INFO:teuthology.orchestra.run.smithi002.stderr:listed 0 entries 2023-08-25T03:34:45.025 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-08-25T03:34:45.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:45.025 DEBUG:teuthology.orchestra.run.smithi002:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-08-25T03:34:45.064 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi002.front.sepia.ceph.com 2023-08-25T03:34:45.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:45.065 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-25T03:34:45.128 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:34:45.128 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:34:45.150 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:34:45.150 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:34:45.210 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:34:45.211 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:34:45.272 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:34:45.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.398+0000 7fb7eb70f700 1 Processor -- start 2023-08-25T03:34:45.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.398+0000 7fb7eb70f700 1 -- start start 2023-08-25T03:34:45.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.399+0000 7fb7eb70f700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb7e415d1f0 0x7fb7e40ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:45.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.399+0000 7fb7eb70f700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fb7e40d0720 0x7fb7e406c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:45.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.399+0000 7fb7eb70f700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7e406d1f0 con 0x7fb7e406ce10 2023-08-25T03:34:45.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.399+0000 7fb7eb70f700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7e40d0bb0 con 0x7fb7e40d0720 2023-08-25T03:34:45.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.399+0000 7fb7eb70f700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7e40cf120 con 0x7fb7e415d1f0 2023-08-25T03:34:45.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.399+0000 7fb7e8caa700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fb7e40d0720 0x7fb7e406c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:45.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.399+0000 7fb7e8caa700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fb7e40d0720 0x7fb7e406c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60586/0 (socket says 172.21.15.2:60586) 2023-08-25T03:34:45.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.399+0000 7fb7e8caa700 1 -- 172.21.15.2:0/435466248 learned_addr learned my addr 172.21.15.2:0/435466248 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:45.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.400+0000 7fb7e94ab700 1 --2- 172.21.15.2:0/435466248 >> v2:172.21.15.31:3300/0 conn(0x7fb7e415d1f0 0x7fb7e40ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:45.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.400+0000 7fb7e8caa700 1 -- 172.21.15.2:0/435466248 >> v2:172.21.15.31:3300/0 conn(0x7fb7e415d1f0 msgr2=0x7fb7e40ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:45.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.400+0000 7fb7e8caa700 1 --2- 172.21.15.2:0/435466248 >> v2:172.21.15.31:3300/0 conn(0x7fb7e415d1f0 0x7fb7e40ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:45.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.400+0000 7fb7e8caa700 1 -- 172.21.15.2:0/435466248 >> v1:172.21.15.31:6789/0 conn(0x7fb7e406ce10 legacy=0x7fb7e406b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.400+0000 7fb7e8caa700 1 -- 172.21.15.2:0/435466248 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e40b29f0 con 0x7fb7e40d0720 2023-08-25T03:34:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.400+0000 7fb7e8caa700 1 --2- 172.21.15.2:0/435466248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7e40d0720 0x7fb7e406c8d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb7e406fe80 tx=0x7fb7e0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d988e609f316e7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.400+0000 7fb7dbfff700 1 -- 172.21.15.2:0/435466248 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e000b870 con 0x7fb7e40d0720 2023-08-25T03:34:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.400+0000 7fb7dbfff700 1 -- 172.21.15.2:0/435466248 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e000b9d0 con 0x7fb7e40d0720 2023-08-25T03:34:45.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.400+0000 7fb7dbfff700 1 -- 172.21.15.2:0/435466248 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e000bd60 con 0x7fb7e40d0720 2023-08-25T03:34:45.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.401+0000 7fb7eb70f700 1 -- 172.21.15.2:0/435466248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7e40d0720 msgr2=0x7fb7e406c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:45.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.401+0000 7fb7eb70f700 1 --2- 172.21.15.2:0/435466248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7e40d0720 0x7fb7e406c8d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb7e406fe80 tx=0x7fb7e0009bd0 comp rx=0 tx=0).stop 2023-08-25T03:34:45.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.401+0000 7fb7eb70f700 1 -- 172.21.15.2:0/435466248 shutdown_connections 2023-08-25T03:34:45.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.401+0000 7fb7eb70f700 1 --2- 172.21.15.2:0/435466248 >> v2:172.21.15.31:3300/0 conn(0x7fb7e415d1f0 0x7fb7e40ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:45.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.401+0000 7fb7eb70f700 1 --2- 172.21.15.2:0/435466248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7e40d0720 0x7fb7e406c8d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:45.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.401+0000 7fb7eb70f700 1 -- 172.21.15.2:0/435466248 >> 172.21.15.2:0/435466248 conn(0x7fb7e40d0f70 msgr2=0x7fb7e40d1350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:45.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.401+0000 7fb7eb70f700 1 -- 172.21.15.2:0/435466248 shutdown_connections 2023-08-25T03:34:45.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.401+0000 7fb7eb70f700 1 -- 172.21.15.2:0/435466248 wait complete. 2023-08-25T03:34:45.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.402+0000 7fb7eb70f700 1 Processor -- start 2023-08-25T03:34:45.411 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.402+0000 7fb7eb70f700 1 -- start start 2023-08-25T03:34:45.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.403+0000 7fb7eb70f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7e406ce10 0x7fb7e40c5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:45.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.403+0000 7fb7eb70f700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb7e40d0720 0x7fb7e4160540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:45.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.403+0000 7fb7eb70f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7e415d1f0 0x7fb7e415e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:45.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.403+0000 7fb7eb70f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e4160a80 con 0x7fb7e406ce10 2023-08-25T03:34:45.412 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.403+0000 7fb7eb70f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e40c5d90 con 0x7fb7e415d1f0 2023-08-25T03:34:45.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.403+0000 7fb7eb70f700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7e40c7350 con 0x7fb7e40d0720 2023-08-25T03:34:45.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.403+0000 7fb7e94ab700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7e406ce10 0x7fb7e40c5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:45.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.403+0000 7fb7e94ab700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7e406ce10 0x7fb7e40c5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60590/0 (socket says 172.21.15.2:60590) 2023-08-25T03:34:45.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.403+0000 7fb7e94ab700 1 -- 172.21.15.2:0/3966564732 learned_addr learned my addr 172.21.15.2:0/3966564732 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:45.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.403+0000 7fb7e9cac700 1 --2- 172.21.15.2:0/3966564732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7e415d1f0 0x7fb7e415e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:45.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.403+0000 7fb7e8caa700 1 --2- 172.21.15.2:0/3966564732 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb7e40d0720 0x7fb7e4160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:45.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.403+0000 7fb7e94ab700 1 -- 172.21.15.2:0/3966564732 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb7e40d0720 msgr2=0x7fb7e4160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:45.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.403+0000 7fb7e94ab700 1 --2- 172.21.15.2:0/3966564732 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb7e40d0720 0x7fb7e4160540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:45.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.403+0000 7fb7e94ab700 1 -- 172.21.15.2:0/3966564732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7e415d1f0 msgr2=0x7fb7e415e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:45.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.403+0000 7fb7e94ab700 1 --2- 172.21.15.2:0/3966564732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7e415d1f0 0x7fb7e415e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:45.414 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.403+0000 7fb7e94ab700 1 -- 172.21.15.2:0/3966564732 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e0009550 con 0x7fb7e406ce10 2023-08-25T03:34:45.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.403+0000 7fb7e94ab700 1 --2- 172.21.15.2:0/3966564732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7e406ce10 0x7fb7e40c5420 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb7d400e810 tx=0x7fb7d400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:45.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.403+0000 7fb7d9ffb700 1 -- 172.21.15.2:0/3966564732 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d4013610 con 0x7fb7e406ce10 2023-08-25T03:34:45.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.403+0000 7fb7d9ffb700 1 -- 172.21.15.2:0/3966564732 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d4013770 con 0x7fb7e406ce10 2023-08-25T03:34:45.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.403+0000 7fb7eb70f700 1 -- 172.21.15.2:0/3966564732 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e415eab0 con 0x7fb7e406ce10 2023-08-25T03:34:45.415 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.403+0000 7fb7d9ffb700 1 -- 172.21.15.2:0/3966564732 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d4013b20 con 0x7fb7e406ce10 2023-08-25T03:34:45.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.404+0000 7fb7eb70f700 1 -- 172.21.15.2:0/3966564732 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e4153e80 con 0x7fb7e406ce10 2023-08-25T03:34:45.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.404+0000 7fb7eb70f700 1 -- 172.21.15.2:0/3966564732 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7e4153ce0 con 0x7fb7e406ce10 2023-08-25T03:34:45.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.405+0000 7fb7d9ffb700 1 -- 172.21.15.2:0/3966564732 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb7d4013c80 con 0x7fb7e406ce10 2023-08-25T03:34:45.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.405+0000 7fb7d9ffb700 1 --2- 172.21.15.2:0/3966564732 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb7c806a0e0 0x7fb7c806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:45.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.405+0000 7fb7d9ffb700 1 -- 172.21.15.2:0/3966564732 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fb7d4088310 con 0x7fb7e406ce10 2023-08-25T03:34:45.416 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.405+0000 7fb7e8caa700 1 --2- 172.21.15.2:0/3966564732 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb7c806a0e0 0x7fb7c806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:45.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.406+0000 7fb7e8caa700 1 --2- 172.21.15.2:0/3966564732 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb7c806a0e0 0x7fb7c806c5a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0000c00 tx=0x7fb7e0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:45.417 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.410+0000 7fb7d9ffb700 1 -- 172.21.15.2:0/3966564732 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb7e4153ce0 con 0x7fb7e406ce10 2023-08-25T03:34:45.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.610+0000 7fb7eb70f700 1 -- 172.21.15.2:0/3966564732 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb7e4153ce0 con 0x7fb7e406ce10 2023-08-25T03:34:45.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.612+0000 7fb7d9ffb700 1 -- 172.21.15.2:0/3966564732 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18230 (secure 0 0 0) 0x7fb7e4153ce0 con 0x7fb7e406ce10 2023-08-25T03:34:45.614 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:45.614 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:34:44.588865+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:34:00.869118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T03:34:01.987016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.f","mappings":[{"from":1,"to":5}]},{"pgid":"4.2b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:34:45.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.614+0000 7fb7eb70f700 1 -- 172.21.15.2:0/3966564732 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb7c806a0e0 msgr2=0x7fb7c806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:45.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.614+0000 7fb7eb70f700 1 --2- 172.21.15.2:0/3966564732 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb7c806a0e0 0x7fb7c806c5a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0000c00 tx=0x7fb7e0008030 comp rx=0 tx=0).stop 2023-08-25T03:34:45.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.614+0000 7fb7eb70f700 1 -- 172.21.15.2:0/3966564732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7e406ce10 msgr2=0x7fb7e40c5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:45.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.614+0000 7fb7eb70f700 1 --2- 172.21.15.2:0/3966564732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7e406ce10 0x7fb7e40c5420 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb7d400e810 tx=0x7fb7d400ebd0 comp rx=0 tx=0).stop 2023-08-25T03:34:45.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.614+0000 7fb7eb70f700 1 -- 172.21.15.2:0/3966564732 shutdown_connections 2023-08-25T03:34:45.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.614+0000 7fb7eb70f700 1 --2- 172.21.15.2:0/3966564732 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb7c806a0e0 0x7fb7c806c5a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:45.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.614+0000 7fb7eb70f700 1 --2- 172.21.15.2:0/3966564732 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7e406ce10 0x7fb7e40c5420 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:45.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.614+0000 7fb7eb70f700 1 --2- 172.21.15.2:0/3966564732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7e415d1f0 0x7fb7e415e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:45.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.614+0000 7fb7eb70f700 1 --2- 172.21.15.2:0/3966564732 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb7e40d0720 0x7fb7e4160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:45.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.614+0000 7fb7eb70f700 1 -- 172.21.15.2:0/3966564732 >> 172.21.15.2:0/3966564732 conn(0x7fb7e40d0f70 msgr2=0x7fb7e415c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:45.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.614+0000 7fb7eb70f700 1 -- 172.21.15.2:0/3966564732 shutdown_connections 2023-08-25T03:34:45.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.614+0000 7fb7eb70f700 1 -- 172.21.15.2:0/3966564732 wait complete. 2023-08-25T03:34:45.627 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T03:33:49.560643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T03:33:56.265184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T03:34:00.869118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T03:34:01.987016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T03:34:45.628 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T03:34:45.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.755+0000 7f49044db700 1 Processor -- start 2023-08-25T03:34:45.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.755+0000 7f49044db700 1 -- start start 2023-08-25T03:34:45.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.756+0000 7f49044db700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f48fc15b170 0x7f48fc0cfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:45.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.756+0000 7f49044db700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48fc0cc640 0x7f48fc0ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:45.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.756+0000 7f49044db700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48fc0cf6f0 con 0x7f48fc0cd7f0 2023-08-25T03:34:45.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.756+0000 7f49044db700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48fc0d1d50 con 0x7f48fc15b170 2023-08-25T03:34:45.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.756+0000 7f49044db700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc15b550 con 0x7f48fc0cc640 2023-08-25T03:34:45.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.756+0000 7f4902277700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f48fc15b170 0x7f48fc0cfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:45.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.756+0000 7f4902277700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f48fc15b170 0x7f48fc0cfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60598/0 (socket says 172.21.15.2:60598) 2023-08-25T03:34:45.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.756+0000 7f4902277700 1 -- 172.21.15.2:0/1101216418 learned_addr learned my addr 172.21.15.2:0/1101216418 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:45.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.757+0000 7f4901a76700 1 --2- 172.21.15.2:0/1101216418 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48fc0cc640 0x7f48fc0ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:45.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.757+0000 7f4902277700 1 -- 172.21.15.2:0/1101216418 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48fc0cc640 msgr2=0x7f48fc0ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:45.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.757+0000 7f4902277700 1 --2- 172.21.15.2:0/1101216418 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48fc0cc640 0x7f48fc0ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:45.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.757+0000 7f4902277700 1 -- 172.21.15.2:0/1101216418 >> v1:172.21.15.31:6789/0 conn(0x7f48fc0cd7f0 legacy=0x7f48fc06d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:45.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.757+0000 7f4902277700 1 -- 172.21.15.2:0/1101216418 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48fc0ceae0 con 0x7f48fc15b170 2023-08-25T03:34:45.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.757+0000 7f4902277700 1 --2- 172.21.15.2:0/1101216418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48fc15b170 0x7f48fc0cfb80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f48ec006fd0 tx=0x7f48ec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ab1ea842281d7e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:45.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.757+0000 7f4901275700 1 -- 172.21.15.2:0/1101216418 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec005e30 con 0x7f48fc15b170 2023-08-25T03:34:45.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.757+0000 7f4901275700 1 -- 172.21.15.2:0/1101216418 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ec006030 con 0x7f48fc15b170 2023-08-25T03:34:45.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.757+0000 7f4901275700 1 -- 172.21.15.2:0/1101216418 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec0063c0 con 0x7f48fc15b170 2023-08-25T03:34:45.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.758+0000 7f49044db700 1 -- 172.21.15.2:0/1101216418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48fc15b170 msgr2=0x7f48fc0cfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:45.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.758+0000 7f49044db700 1 --2- 172.21.15.2:0/1101216418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48fc15b170 0x7f48fc0cfb80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f48ec006fd0 tx=0x7f48ec009bd0 comp rx=0 tx=0).stop 2023-08-25T03:34:45.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.758+0000 7f49044db700 1 -- 172.21.15.2:0/1101216418 shutdown_connections 2023-08-25T03:34:45.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.758+0000 7f49044db700 1 --2- 172.21.15.2:0/1101216418 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48fc15b170 0x7f48fc0cfb80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:45.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.758+0000 7f49044db700 1 --2- 172.21.15.2:0/1101216418 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48fc0cc640 0x7f48fc0ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:45.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.758+0000 7f49044db700 1 -- 172.21.15.2:0/1101216418 >> 172.21.15.2:0/1101216418 conn(0x7f48fc09a200 msgr2=0x7f48fc0d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:45.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.758+0000 7f49044db700 1 -- 172.21.15.2:0/1101216418 shutdown_connections 2023-08-25T03:34:45.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.758+0000 7f49044db700 1 -- 172.21.15.2:0/1101216418 wait complete. 2023-08-25T03:34:45.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.759+0000 7f49044db700 1 Processor -- start 2023-08-25T03:34:45.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.759+0000 7f49044db700 1 -- start start 2023-08-25T03:34:45.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.760+0000 7f49044db700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48fc0cc640 0x7f48fc0aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:45.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.760+0000 7f49044db700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48fc0cd7f0 0x7f48fc152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:45.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.760+0000 7f49044db700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48fc15b170 0x7f48fc153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:45.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.760+0000 7f49044db700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc15b760 con 0x7f48fc0cc640 2023-08-25T03:34:45.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.760+0000 7f49044db700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc15ba40 con 0x7f48fc15b170 2023-08-25T03:34:45.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.760+0000 7f49044db700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48fc15bd20 con 0x7f48fc0cd7f0 2023-08-25T03:34:45.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.760+0000 7f4902277700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48fc0cc640 0x7f48fc0aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:45.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.760+0000 7f4902277700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48fc0cc640 0x7f48fc0aed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60604/0 (socket says 172.21.15.2:60604) 2023-08-25T03:34:45.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.760+0000 7f4902277700 1 -- 172.21.15.2:0/3904425383 learned_addr learned my addr 172.21.15.2:0/3904425383 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:45.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.760+0000 7f4902a78700 1 --2- 172.21.15.2:0/3904425383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48fc15b170 0x7f48fc153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:45.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.760+0000 7f4901a76700 1 --2- 172.21.15.2:0/3904425383 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48fc0cd7f0 0x7f48fc152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:45.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.760+0000 7f4902277700 1 -- 172.21.15.2:0/3904425383 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48fc0cd7f0 msgr2=0x7f48fc152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:45.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.760+0000 7f4902277700 1 --2- 172.21.15.2:0/3904425383 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48fc0cd7f0 0x7f48fc152c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:45.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.760+0000 7f4902277700 1 -- 172.21.15.2:0/3904425383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48fc15b170 msgr2=0x7f48fc153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:45.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.760+0000 7f4902277700 1 --2- 172.21.15.2:0/3904425383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48fc15b170 0x7f48fc153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:45.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.760+0000 7f4902277700 1 -- 172.21.15.2:0/3904425383 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ec009500 con 0x7f48fc0cc640 2023-08-25T03:34:45.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.760+0000 7f4902a78700 1 --2- 172.21.15.2:0/3904425383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48fc15b170 0x7f48fc153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:45.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.760+0000 7f4902277700 1 --2- 172.21.15.2:0/3904425383 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48fc0cc640 0x7f48fc0aed20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f48ec009c10 tx=0x7f48ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:45.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.760+0000 7f48f2ffd700 1 -- 172.21.15.2:0/3904425383 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec025430 con 0x7f48fc0cc640 2023-08-25T03:34:45.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.760+0000 7f48f2ffd700 1 -- 172.21.15.2:0/3904425383 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48ec020070 con 0x7f48fc0cc640 2023-08-25T03:34:45.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.760+0000 7f49044db700 1 -- 172.21.15.2:0/3904425383 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48fc15c000 con 0x7f48fc0cc640 2023-08-25T03:34:45.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.760+0000 7f48f2ffd700 1 -- 172.21.15.2:0/3904425383 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48ec0257f0 con 0x7f48fc0cc640 2023-08-25T03:34:45.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.760+0000 7f49044db700 1 -- 172.21.15.2:0/3904425383 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48fc0c4130 con 0x7f48fc0cc640 2023-08-25T03:34:45.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.761+0000 7f49044db700 1 -- 172.21.15.2:0/3904425383 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48fc0cf080 con 0x7f48fc0cc640 2023-08-25T03:34:45.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.762+0000 7f48f2ffd700 1 -- 172.21.15.2:0/3904425383 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f48ec022030 con 0x7f48fc0cc640 2023-08-25T03:34:45.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.762+0000 7f48f2ffd700 1 --2- 172.21.15.2:0/3904425383 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f48e006a0e0 0x7f48e006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:45.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.762+0000 7f48f2ffd700 1 -- 172.21.15.2:0/3904425383 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f48ec089cb0 con 0x7f48fc0cc640 2023-08-25T03:34:45.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.762+0000 7f4901a76700 1 --2- 172.21.15.2:0/3904425383 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f48e006a0e0 0x7f48e006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:45.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.763+0000 7f4901a76700 1 --2- 172.21.15.2:0/3904425383 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f48e006a0e0 0x7f48e006c5a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0cc610 tx=0x7f48f8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:45.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.767+0000 7f48f2ffd700 1 -- 172.21.15.2:0/3904425383 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f48fc0cf080 con 0x7f48fc0cc640 2023-08-25T03:34:45.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.967+0000 7f49044db700 1 -- 172.21.15.2:0/3904425383 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f48fc0cf080 con 0x7f48fc0cc640 2023-08-25T03:34:46.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.968+0000 7f48f2ffd700 1 -- 172.21.15.2:0/3904425383 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f48fc0cf080 con 0x7f48fc0cc640 2023-08-25T03:34:46.058 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-08-25T03:34:46.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.970+0000 7f49044db700 1 -- 172.21.15.2:0/3904425383 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f48e006a0e0 msgr2=0x7f48e006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:46.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.970+0000 7f49044db700 1 --2- 172.21.15.2:0/3904425383 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f48e006a0e0 0x7f48e006c5a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f48fc0cc610 tx=0x7f48f8008040 comp rx=0 tx=0).stop 2023-08-25T03:34:46.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.970+0000 7f49044db700 1 -- 172.21.15.2:0/3904425383 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48fc0cc640 msgr2=0x7f48fc0aed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:46.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.970+0000 7f49044db700 1 --2- 172.21.15.2:0/3904425383 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48fc0cc640 0x7f48fc0aed20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f48ec009c10 tx=0x7f48ec004750 comp rx=0 tx=0).stop 2023-08-25T03:34:46.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.970+0000 7f49044db700 1 -- 172.21.15.2:0/3904425383 shutdown_connections 2023-08-25T03:34:46.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.970+0000 7f49044db700 1 --2- 172.21.15.2:0/3904425383 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f48e006a0e0 0x7f48e006c5a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.970+0000 7f49044db700 1 --2- 172.21.15.2:0/3904425383 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f48fc0cc640 0x7f48fc0aed20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.970+0000 7f49044db700 1 --2- 172.21.15.2:0/3904425383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48fc15b170 0x7f48fc153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.970+0000 7f49044db700 1 --2- 172.21.15.2:0/3904425383 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48fc0cd7f0 0x7f48fc152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.970+0000 7f49044db700 1 -- 172.21.15.2:0/3904425383 >> 172.21.15.2:0/3904425383 conn(0x7f48fc09a200 msgr2=0x7f48fc0d0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:46.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.970+0000 7f49044db700 1 -- 172.21.15.2:0/3904425383 shutdown_connections 2023-08-25T03:34:46.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:45.970+0000 7f49044db700 1 -- 172.21.15.2:0/3904425383 wait complete. 2023-08-25T03:34:46.062 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T03:34:46.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.190+0000 7fc91cbba700 1 Processor -- start 2023-08-25T03:34:46.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.190+0000 7fc91cbba700 1 -- start start 2023-08-25T03:34:46.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.191+0000 7fc91cbba700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fc918159f10 0x7fc91815dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:46.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.191+0000 7fc91cbba700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9180cedf0 0x7fc91806ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:46.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.191+0000 7fc91cbba700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc91806f4c0 0x7fc9180cacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:46.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.191+0000 7fc91cbba700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc91806e850 con 0x7fc918159f10 2023-08-25T03:34:46.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.191+0000 7fc91cbba700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc91806eaf0 con 0x7fc91806f4c0 2023-08-25T03:34:46.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.191+0000 7fc91cbba700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc91815e020 con 0x7fc9180cedf0 2023-08-25T03:34:46.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.191+0000 7fc91659c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fc918159f10 0x7fc91815dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:46.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.191+0000 7fc91659c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fc918159f10 0x7fc91815dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60612/0 (socket says 172.21.15.2:60612) 2023-08-25T03:34:46.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.191+0000 7fc91659c700 1 -- 172.21.15.2:0/3792849099 learned_addr learned my addr 172.21.15.2:0/3792849099 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:46.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.191+0000 7fc916d9d700 1 --2- 172.21.15.2:0/3792849099 >> v2:172.21.15.31:3300/0 conn(0x7fc91806f4c0 0x7fc9180cacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:46.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.191+0000 7fc915d9b700 1 --2- 172.21.15.2:0/3792849099 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9180cedf0 0x7fc91806ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:46.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.192+0000 7fc91659c700 1 -- 172.21.15.2:0/3792849099 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9180cedf0 msgr2=0x7fc91806ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:46.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.192+0000 7fc91659c700 1 --2- 172.21.15.2:0/3792849099 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9180cedf0 0x7fc91806ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.192+0000 7fc91659c700 1 -- 172.21.15.2:0/3792849099 >> v2:172.21.15.31:3300/0 conn(0x7fc91806f4c0 msgr2=0x7fc9180cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:46.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.192+0000 7fc91659c700 1 --2- 172.21.15.2:0/3792849099 >> v2:172.21.15.31:3300/0 conn(0x7fc91806f4c0 0x7fc9180cacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.192+0000 7fc91659c700 1 -- 172.21.15.2:0/3792849099 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc91815e2c0 con 0x7fc918159f10 2023-08-25T03:34:46.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.192+0000 7fc91659c700 1 --2- 172.21.15.2:0/3792849099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc918159f10 0x7fc91815dae0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc9180cfb00 tx=0x7fc90c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6831d6c7a23b99 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:46.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.192+0000 7fc91559a700 1 -- 172.21.15.2:0/3792849099 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c011830 con 0x7fc918159f10 2023-08-25T03:34:46.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.192+0000 7fc91559a700 1 -- 172.21.15.2:0/3792849099 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc90c011990 con 0x7fc918159f10 2023-08-25T03:34:46.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.192+0000 7fc91559a700 1 -- 172.21.15.2:0/3792849099 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c011d20 con 0x7fc918159f10 2023-08-25T03:34:46.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.192+0000 7fc91cbba700 1 -- 172.21.15.2:0/3792849099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc918159f10 msgr2=0x7fc91815dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:46.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.192+0000 7fc91cbba700 1 --2- 172.21.15.2:0/3792849099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc918159f10 0x7fc91815dae0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc9180cfb00 tx=0x7fc90c009bd0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.193+0000 7fc91cbba700 1 -- 172.21.15.2:0/3792849099 shutdown_connections 2023-08-25T03:34:46.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.193+0000 7fc91cbba700 1 --2- 172.21.15.2:0/3792849099 >> v2:172.21.15.31:3300/0 conn(0x7fc91806f4c0 0x7fc9180cacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.193+0000 7fc91cbba700 1 --2- 172.21.15.2:0/3792849099 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc918159f10 0x7fc91815dae0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.193+0000 7fc91cbba700 1 --2- 172.21.15.2:0/3792849099 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9180cedf0 0x7fc91806ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.193+0000 7fc91cbba700 1 -- 172.21.15.2:0/3792849099 >> 172.21.15.2:0/3792849099 conn(0x7fc9180cfd90 msgr2=0x7fc9180d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:46.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.193+0000 7fc91cbba700 1 -- 172.21.15.2:0/3792849099 shutdown_connections 2023-08-25T03:34:46.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.193+0000 7fc91cbba700 1 -- 172.21.15.2:0/3792849099 wait complete. 2023-08-25T03:34:46.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.194+0000 7fc91cbba700 1 Processor -- start 2023-08-25T03:34:46.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.194+0000 7fc91cbba700 1 -- start start 2023-08-25T03:34:46.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.195+0000 7fc91cbba700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc91806f4c0 0x7fc9180c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:46.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.195+0000 7fc91cbba700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9180cedf0 0x7fc9180b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:46.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.195+0000 7fc91cbba700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc918159f10 0x7fc918146e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:46.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.195+0000 7fc91cbba700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9180c5930 con 0x7fc91806f4c0 2023-08-25T03:34:46.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.195+0000 7fc91cbba700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9180b3fd0 con 0x7fc918159f10 2023-08-25T03:34:46.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.195+0000 7fc91cbba700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9180b4200 con 0x7fc9180cedf0 2023-08-25T03:34:46.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.195+0000 7fc91659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc91806f4c0 0x7fc9180c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:46.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.195+0000 7fc91659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc91806f4c0 0x7fc9180c6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60618/0 (socket says 172.21.15.2:60618) 2023-08-25T03:34:46.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.195+0000 7fc91659c700 1 -- 172.21.15.2:0/3577910614 learned_addr learned my addr 172.21.15.2:0/3577910614 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:46.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.195+0000 7fc91659c700 1 -- 172.21.15.2:0/3577910614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9180cedf0 msgr2=0x7fc9180b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:46.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.195+0000 7fc91659c700 1 --2- 172.21.15.2:0/3577910614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9180cedf0 0x7fc9180b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.195+0000 7fc916d9d700 1 --2- 172.21.15.2:0/3577910614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc918159f10 0x7fc918146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:46.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.195+0000 7fc91659c700 1 -- 172.21.15.2:0/3577910614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc918159f10 msgr2=0x7fc918146e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:46.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.195+0000 7fc91659c700 1 --2- 172.21.15.2:0/3577910614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc918159f10 0x7fc918146e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.195+0000 7fc91659c700 1 -- 172.21.15.2:0/3577910614 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90c009550 con 0x7fc91806f4c0 2023-08-25T03:34:46.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.195+0000 7fc91659c700 1 --2- 172.21.15.2:0/3577910614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc91806f4c0 0x7fc9180c6550 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc90c000c00 tx=0x7fc90c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:46.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.195+0000 7fc906ffd700 1 -- 172.21.15.2:0/3577910614 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c004300 con 0x7fc91806f4c0 2023-08-25T03:34:46.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.195+0000 7fc906ffd700 1 -- 172.21.15.2:0/3577910614 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc90c01f070 con 0x7fc91806f4c0 2023-08-25T03:34:46.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.195+0000 7fc91cbba700 1 -- 172.21.15.2:0/3577910614 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9180b44e0 con 0x7fc91806f4c0 2023-08-25T03:34:46.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.195+0000 7fc906ffd700 1 -- 172.21.15.2:0/3577910614 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c0036a0 con 0x7fc91806f4c0 2023-08-25T03:34:46.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.196+0000 7fc91cbba700 1 -- 172.21.15.2:0/3577910614 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9180b4aa0 con 0x7fc91806f4c0 2023-08-25T03:34:46.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.196+0000 7fc904ff9700 1 -- 172.21.15.2:0/3577910614 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f00052f0 con 0x7fc91806f4c0 2023-08-25T03:34:46.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.197+0000 7fc906ffd700 1 -- 172.21.15.2:0/3577910614 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc90c021030 con 0x7fc91806f4c0 2023-08-25T03:34:46.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.197+0000 7fc906ffd700 1 --2- 172.21.15.2:0/3577910614 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc8f806a0e0 0x7fc8f806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:46.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.197+0000 7fc906ffd700 1 -- 172.21.15.2:0/3577910614 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc90c089830 con 0x7fc91806f4c0 2023-08-25T03:34:46.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.197+0000 7fc915d9b700 1 --2- 172.21.15.2:0/3577910614 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc8f806a0e0 0x7fc8f806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:46.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.198+0000 7fc915d9b700 1 --2- 172.21.15.2:0/3577910614 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc8f806a0e0 0x7fc8f806c5a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc908006b20 tx=0x7fc908006ab0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:46.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.202+0000 7fc906ffd700 1 -- 172.21.15.2:0/3577910614 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc90c054ef0 con 0x7fc91806f4c0 2023-08-25T03:34:46.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.402+0000 7fc904ff9700 1 -- 172.21.15.2:0/3577910614 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc8f0005b20 con 0x7fc91806f4c0 2023-08-25T03:34:46.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.402+0000 7fc906ffd700 1 -- 172.21.15.2:0/3577910614 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc90c058510 con 0x7fc91806f4c0 2023-08-25T03:34:46.403 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 8 2023-08-25T03:34:46.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.405+0000 7fc904ff9700 1 -- 172.21.15.2:0/3577910614 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc8f806a0e0 msgr2=0x7fc8f806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:46.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.405+0000 7fc904ff9700 1 --2- 172.21.15.2:0/3577910614 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc8f806a0e0 0x7fc8f806c5a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc908006b20 tx=0x7fc908006ab0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.405+0000 7fc904ff9700 1 -- 172.21.15.2:0/3577910614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc91806f4c0 msgr2=0x7fc9180c6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:46.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.405+0000 7fc904ff9700 1 --2- 172.21.15.2:0/3577910614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc91806f4c0 0x7fc9180c6550 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc90c000c00 tx=0x7fc90c004750 comp rx=0 tx=0).stop 2023-08-25T03:34:46.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.405+0000 7fc904ff9700 1 -- 172.21.15.2:0/3577910614 shutdown_connections 2023-08-25T03:34:46.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.405+0000 7fc904ff9700 1 --2- 172.21.15.2:0/3577910614 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc8f806a0e0 0x7fc8f806c5a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.405+0000 7fc904ff9700 1 --2- 172.21.15.2:0/3577910614 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc91806f4c0 0x7fc9180c6550 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.405+0000 7fc904ff9700 1 --2- 172.21.15.2:0/3577910614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc918159f10 0x7fc918146e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.405+0000 7fc904ff9700 1 --2- 172.21.15.2:0/3577910614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9180cedf0 0x7fc9180b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.405+0000 7fc904ff9700 1 -- 172.21.15.2:0/3577910614 >> 172.21.15.2:0/3577910614 conn(0x7fc9180cfd90 msgr2=0x7fc9180cb740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:46.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.405+0000 7fc904ff9700 1 -- 172.21.15.2:0/3577910614 shutdown_connections 2023-08-25T03:34:46.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.405+0000 7fc904ff9700 1 -- 172.21.15.2:0/3577910614 wait complete. 2023-08-25T03:34:46.418 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T03:34:46.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.545+0000 7f7807cec700 1 Processor -- start 2023-08-25T03:34:46.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.545+0000 7f7807cec700 1 -- start start 2023-08-25T03:34:46.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.546+0000 7f7807cec700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f780006a660 0x7f7800072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:46.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.546+0000 7f7807cec700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7800066050 0x7f780015f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:46.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.546+0000 7f7807cec700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78000b1e30 con 0x7f780006aa40 2023-08-25T03:34:46.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.546+0000 7f7807cec700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78000b12b0 con 0x7f780006a660 2023-08-25T03:34:46.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.546+0000 7f7807cec700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78000b1590 con 0x7f7800066050 2023-08-25T03:34:46.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.546+0000 7f7805a88700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f780006a660 0x7f7800072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:46.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.546+0000 7f7805a88700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f780006a660 0x7f7800072330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60626/0 (socket says 172.21.15.2:60626) 2023-08-25T03:34:46.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.546+0000 7f7805a88700 1 -- 172.21.15.2:0/4180690598 learned_addr learned my addr 172.21.15.2:0/4180690598 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:46.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.546+0000 7f7805287700 1 --2- 172.21.15.2:0/4180690598 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7800066050 0x7f780015f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:46.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.546+0000 7f7805a88700 1 -- 172.21.15.2:0/4180690598 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7800066050 msgr2=0x7f780015f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:46.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.546+0000 7f7805a88700 1 --2- 172.21.15.2:0/4180690598 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7800066050 0x7f780015f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.546+0000 7f7805a88700 1 -- 172.21.15.2:0/4180690598 >> v1:172.21.15.31:6789/0 conn(0x7f780006aa40 legacy=0x7f780015f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:46.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.546+0000 7f7805a88700 1 -- 172.21.15.2:0/4180690598 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78000b1870 con 0x7f780006a660 2023-08-25T03:34:46.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.546+0000 7f7805a88700 1 --2- 172.21.15.2:0/4180690598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f780006a660 0x7f7800072330 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f77fc006fd0 tx=0x7f77fc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f95c91ac768e04 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:46.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.546+0000 7f7804a86700 1 -- 172.21.15.2:0/4180690598 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc005e30 con 0x7f780006a660 2023-08-25T03:34:46.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.546+0000 7f7804a86700 1 -- 172.21.15.2:0/4180690598 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77fc006030 con 0x7f780006a660 2023-08-25T03:34:46.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.547+0000 7f7804a86700 1 -- 172.21.15.2:0/4180690598 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0063c0 con 0x7f780006a660 2023-08-25T03:34:46.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.547+0000 7f7807cec700 1 -- 172.21.15.2:0/4180690598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f780006a660 msgr2=0x7f7800072330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:46.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.547+0000 7f7807cec700 1 --2- 172.21.15.2:0/4180690598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f780006a660 0x7f7800072330 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f77fc006fd0 tx=0x7f77fc009bd0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.547+0000 7f7807cec700 1 -- 172.21.15.2:0/4180690598 shutdown_connections 2023-08-25T03:34:46.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.547+0000 7f7807cec700 1 --2- 172.21.15.2:0/4180690598 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f780006a660 0x7f7800072330 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.547+0000 7f7807cec700 1 --2- 172.21.15.2:0/4180690598 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7800066050 0x7f780015f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.547+0000 7f7807cec700 1 -- 172.21.15.2:0/4180690598 >> 172.21.15.2:0/4180690598 conn(0x7f78000cf500 msgr2=0x7f78000cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:46.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.547+0000 7f7807cec700 1 -- 172.21.15.2:0/4180690598 shutdown_connections 2023-08-25T03:34:46.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.547+0000 7f7807cec700 1 -- 172.21.15.2:0/4180690598 wait complete. 2023-08-25T03:34:46.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.549+0000 7f7807cec700 1 Processor -- start 2023-08-25T03:34:46.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.549+0000 7f7807cec700 1 -- start start 2023-08-25T03:34:46.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.550+0000 7f7807cec700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7800066050 0x7f780014f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:46.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.550+0000 7f7807cec700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f780006a660 0x7f780015d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:46.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.550+0000 7f7807cec700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f780006aa40 0x7f780015af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:46.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.550+0000 7f7807cec700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78000c5d70 con 0x7f780006a660 2023-08-25T03:34:46.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.550+0000 7f7807cec700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780014f9e0 con 0x7f7800066050 2023-08-25T03:34:46.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.550+0000 7f7807cec700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780014ff00 con 0x7f780006aa40 2023-08-25T03:34:46.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.550+0000 7f7805287700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f780006a660 0x7f780015d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:46.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.550+0000 7f7805287700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f780006a660 0x7f780015d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60632/0 (socket says 172.21.15.2:60632) 2023-08-25T03:34:46.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.550+0000 7f7805287700 1 -- 172.21.15.2:0/4268934631 learned_addr learned my addr 172.21.15.2:0/4268934631 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:46.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.550+0000 7f7805a88700 1 --2- 172.21.15.2:0/4268934631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7800066050 0x7f780014f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:46.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.550+0000 7f7806289700 1 --2- 172.21.15.2:0/4268934631 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f780006aa40 0x7f780015af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:46.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.550+0000 7f7805287700 1 -- 172.21.15.2:0/4268934631 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f780006aa40 msgr2=0x7f780015af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:46.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.550+0000 7f7805287700 1 --2- 172.21.15.2:0/4268934631 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f780006aa40 0x7f780015af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.550+0000 7f7805287700 1 -- 172.21.15.2:0/4268934631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7800066050 msgr2=0x7f780014f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:46.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.550+0000 7f7805287700 1 --2- 172.21.15.2:0/4268934631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7800066050 0x7f780014f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.550+0000 7f7805287700 1 -- 172.21.15.2:0/4268934631 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77fc009500 con 0x7f780006a660 2023-08-25T03:34:46.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.550+0000 7f7806289700 1 --2- 172.21.15.2:0/4268934631 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f780006aa40 0x7f780015af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:46.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.550+0000 7f7805287700 1 --2- 172.21.15.2:0/4268934631 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f780006a660 0x7f780015d7a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f77f000e840 tx=0x7f77f000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:46.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.551+0000 7f77f67fc700 1 -- 172.21.15.2:0/4268934631 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f0013610 con 0x7f780006a660 2023-08-25T03:34:46.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.551+0000 7f77f67fc700 1 -- 172.21.15.2:0/4268934631 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f0013770 con 0x7f780006a660 2023-08-25T03:34:46.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.551+0000 7f77f67fc700 1 -- 172.21.15.2:0/4268934631 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f0013b20 con 0x7f780006a660 2023-08-25T03:34:46.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.551+0000 7f7807cec700 1 -- 172.21.15.2:0/4268934631 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78000c4e80 con 0x7f780006a660 2023-08-25T03:34:46.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.551+0000 7f7807cec700 1 -- 172.21.15.2:0/4268934631 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78000c5440 con 0x7f780006a660 2023-08-25T03:34:46.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.551+0000 7f7807cec700 1 -- 172.21.15.2:0/4268934631 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78000c5f20 con 0x7f780006a660 2023-08-25T03:34:46.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.552+0000 7f77f67fc700 1 -- 172.21.15.2:0/4268934631 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f77f0013c80 con 0x7f780006a660 2023-08-25T03:34:46.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.552+0000 7f77f67fc700 1 --2- 172.21.15.2:0/4268934631 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f77e406a0e0 0x7f77e406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:46.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.552+0000 7f77f67fc700 1 -- 172.21.15.2:0/4268934631 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f77f0088270 con 0x7f780006a660 2023-08-25T03:34:46.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.552+0000 7f7805a88700 1 --2- 172.21.15.2:0/4268934631 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f77e406a0e0 0x7f77e406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:46.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.553+0000 7f7805a88700 1 --2- 172.21.15.2:0/4268934631 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f77e406a0e0 0x7f77e406c5a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f77fc013040 tx=0x7f77fc008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:46.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.557+0000 7f77f67fc700 1 -- 172.21.15.2:0/4268934631 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f78000c5f20 con 0x7f780006a660 2023-08-25T03:34:46.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.757+0000 7f7807cec700 1 -- 172.21.15.2:0/4268934631 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f78000c5f20 con 0x7f780006a660 2023-08-25T03:34:46.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.757+0000 7f77f67fc700 1 -- 172.21.15.2:0/4268934631 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f78000c5f20 con 0x7f780006a660 2023-08-25T03:34:46.759 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-08-25T03:34:46.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.760+0000 7f7807cec700 1 -- 172.21.15.2:0/4268934631 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f77e406a0e0 msgr2=0x7f77e406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:46.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.760+0000 7f7807cec700 1 --2- 172.21.15.2:0/4268934631 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f77e406a0e0 0x7f77e406c5a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f77fc013040 tx=0x7f77fc008070 comp rx=0 tx=0).stop 2023-08-25T03:34:46.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.760+0000 7f7807cec700 1 -- 172.21.15.2:0/4268934631 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f780006a660 msgr2=0x7f780015d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:46.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.760+0000 7f7807cec700 1 --2- 172.21.15.2:0/4268934631 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f780006a660 0x7f780015d7a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f77f000e840 tx=0x7f77f000ec00 comp rx=0 tx=0).stop 2023-08-25T03:34:46.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.760+0000 7f7807cec700 1 -- 172.21.15.2:0/4268934631 shutdown_connections 2023-08-25T03:34:46.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.760+0000 7f7807cec700 1 --2- 172.21.15.2:0/4268934631 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f780006aa40 0x7f780015af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.760+0000 7f7807cec700 1 --2- 172.21.15.2:0/4268934631 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7800066050 0x7f780014f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.760+0000 7f7807cec700 1 --2- 172.21.15.2:0/4268934631 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f77e406a0e0 0x7f77e406c5a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.760+0000 7f7807cec700 1 --2- 172.21.15.2:0/4268934631 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f780006a660 0x7f780015d7a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.760+0000 7f7807cec700 1 -- 172.21.15.2:0/4268934631 >> 172.21.15.2:0/4268934631 conn(0x7f78000cf500 msgr2=0x7f780015e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:46.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.760+0000 7f7807cec700 1 -- 172.21.15.2:0/4268934631 shutdown_connections 2023-08-25T03:34:46.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.760+0000 7f7807cec700 1 -- 172.21.15.2:0/4268934631 wait complete. 2023-08-25T03:34:46.772 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T03:34:46.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.900+0000 7fdca7b16700 1 Processor -- start 2023-08-25T03:34:46.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.901+0000 7fdca7b16700 1 -- start start 2023-08-25T03:34:46.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.901+0000 7fdca7b16700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdca006a8f0 0x7fdca0070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:46.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.901+0000 7fdca7b16700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca0160bd0 con 0x7fdca00af180 2023-08-25T03:34:46.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.901+0000 7fdca7b16700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdca00cdd00 con 0x7fdca01607f0 2023-08-25T03:34:46.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.901+0000 7fdca7b16700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdca00713a0 con 0x7fdca006a8f0 2023-08-25T03:34:46.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.901+0000 7fdca50b1700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fdca00af180 0x7fdca006a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47304/0 (socket says 172.21.15.2:47304) 2023-08-25T03:34:46.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.901+0000 7fdca50b1700 1 -- 172.21.15.2:0/2255141069 learned_addr learned my addr 172.21.15.2:0/2255141069 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:46.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.902+0000 7fdca60b3700 1 --2- 172.21.15.2:0/2255141069 >> v2:172.21.15.31:3300/0 conn(0x7fdca006a8f0 0x7fdca0070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:46.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.902+0000 7fdca48b0700 1 -- 172.21.15.2:0/2255141069 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2998836212 0 0) 0x7fdca0160bd0 con 0x7fdca00af180 2023-08-25T03:34:46.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.902+0000 7fdca48b0700 1 -- 172.21.15.2:0/2255141069 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc8c0035b0 con 0x7fdca00af180 2023-08-25T03:34:46.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.902+0000 7fdca48b0700 1 -- 172.21.15.2:0/2255141069 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3884644863 0 0) 0x7fdc8c0035b0 con 0x7fdca00af180 2023-08-25T03:34:46.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.902+0000 7fdca60b3700 1 -- 172.21.15.2:0/2255141069 >> v1:172.21.15.31:6789/0 conn(0x7fdca01607f0 legacy=0x7fdca00ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:46.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.902+0000 7fdca60b3700 1 -- 172.21.15.2:0/2255141069 >> v1:172.21.15.2:6789/0 conn(0x7fdca00af180 legacy=0x7fdca006a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:46.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.902+0000 7fdca60b3700 1 -- 172.21.15.2:0/2255141069 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdca00cda00 con 0x7fdca006a8f0 2023-08-25T03:34:46.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.902+0000 7fdca60b3700 1 --2- 172.21.15.2:0/2255141069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca006a8f0 0x7fdca0070e60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdc98009d90 tx=0x7fdc9800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48d79f6b64e968e4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:46.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.902+0000 7fdca48b0700 1 -- 172.21.15.2:0/2255141069 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9800e070 con 0x7fdca006a8f0 2023-08-25T03:34:46.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.902+0000 7fdca48b0700 1 -- 172.21.15.2:0/2255141069 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc98003a70 con 0x7fdca006a8f0 2023-08-25T03:34:46.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.902+0000 7fdca48b0700 1 -- 172.21.15.2:0/2255141069 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c001670 con 0x7fdca006a8f0 2023-08-25T03:34:46.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.903+0000 7fdca7b16700 1 -- 172.21.15.2:0/2255141069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca006a8f0 msgr2=0x7fdca0070e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:46.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.903+0000 7fdca7b16700 1 --2- 172.21.15.2:0/2255141069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca006a8f0 0x7fdca0070e60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdc98009d90 tx=0x7fdc9800b330 comp rx=0 tx=0).stop 2023-08-25T03:34:46.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.903+0000 7fdca7b16700 1 -- 172.21.15.2:0/2255141069 shutdown_connections 2023-08-25T03:34:46.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.903+0000 7fdca7b16700 1 --2- 172.21.15.2:0/2255141069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca006a8f0 0x7fdca0070e60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.903+0000 7fdca7b16700 1 -- 172.21.15.2:0/2255141069 >> 172.21.15.2:0/2255141069 conn(0x7fdca00cfd90 msgr2=0x7fdca00d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:46.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.903+0000 7fdca7b16700 1 -- 172.21.15.2:0/2255141069 shutdown_connections 2023-08-25T03:34:46.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.903+0000 7fdca7b16700 1 -- 172.21.15.2:0/2255141069 wait complete. 2023-08-25T03:34:46.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.904+0000 7fdca7b16700 1 Processor -- start 2023-08-25T03:34:46.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.904+0000 7fdca7b16700 1 -- start start 2023-08-25T03:34:46.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.905+0000 7fdca7b16700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdca006a8f0 0x7fdca01f0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:46.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.905+0000 7fdca7b16700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdca00af180 0x7fdca00cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:46.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.905+0000 7fdca7b16700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca01607f0 0x7fdca00cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:46.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.905+0000 7fdca7b16700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca01f07d0 con 0x7fdca00af180 2023-08-25T03:34:46.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.905+0000 7fdca7b16700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca00b04a0 con 0x7fdca01607f0 2023-08-25T03:34:46.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.905+0000 7fdca7b16700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdca00b0780 con 0x7fdca006a8f0 2023-08-25T03:34:46.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.905+0000 7fdca50b1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdca00af180 0x7fdca00cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:46.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.905+0000 7fdca50b1700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdca00af180 0x7fdca00cabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60644/0 (socket says 172.21.15.2:60644) 2023-08-25T03:34:46.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.905+0000 7fdca60b3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca01607f0 0x7fdca00cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:46.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.905+0000 7fdca50b1700 1 -- 172.21.15.2:0/1509502403 learned_addr learned my addr 172.21.15.2:0/1509502403 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:46.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.905+0000 7fdca58b2700 1 --2- 172.21.15.2:0/1509502403 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdca006a8f0 0x7fdca01f0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:46.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.906+0000 7fdca50b1700 1 -- 172.21.15.2:0/1509502403 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdca006a8f0 msgr2=0x7fdca01f0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:46.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.906+0000 7fdca50b1700 1 --2- 172.21.15.2:0/1509502403 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdca006a8f0 0x7fdca01f0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.906+0000 7fdca50b1700 1 -- 172.21.15.2:0/1509502403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca01607f0 msgr2=0x7fdca00cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:46.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.906+0000 7fdca50b1700 1 --2- 172.21.15.2:0/1509502403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca01607f0 0x7fdca00cb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:46.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.906+0000 7fdca50b1700 1 -- 172.21.15.2:0/1509502403 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc98009a70 con 0x7fdca00af180 2023-08-25T03:34:46.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.906+0000 7fdca60b3700 1 --2- 172.21.15.2:0/1509502403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca01607f0 0x7fdca00cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:34:46.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.906+0000 7fdca50b1700 1 --2- 172.21.15.2:0/1509502403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdca00af180 0x7fdca00cabd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00c770 tx=0x7fdc9c00ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:46.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.906+0000 7fdc967fc700 1 -- 172.21.15.2:0/1509502403 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c004c80 con 0x7fdca00af180 2023-08-25T03:34:46.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.906+0000 7fdc967fc700 1 -- 172.21.15.2:0/1509502403 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9c004de0 con 0x7fdca00af180 2023-08-25T03:34:46.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.906+0000 7fdca7b16700 1 -- 172.21.15.2:0/1509502403 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca00b0a60 con 0x7fdca00af180 2023-08-25T03:34:46.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.906+0000 7fdc967fc700 1 -- 172.21.15.2:0/1509502403 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9c0072d0 con 0x7fdca00af180 2023-08-25T03:34:46.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.906+0000 7fdca7b16700 1 -- 172.21.15.2:0/1509502403 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca00b1020 con 0x7fdca00af180 2023-08-25T03:34:46.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.907+0000 7fdca7b16700 1 -- 172.21.15.2:0/1509502403 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca00746d0 con 0x7fdca00af180 2023-08-25T03:34:46.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.908+0000 7fdc967fc700 1 -- 172.21.15.2:0/1509502403 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdc9c007430 con 0x7fdca00af180 2023-08-25T03:34:46.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.908+0000 7fdc967fc700 1 --2- 172.21.15.2:0/1509502403 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fdc8406a0e0 0x7fdc8406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:46.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.908+0000 7fdc967fc700 1 -- 172.21.15.2:0/1509502403 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fdc9c0883d0 con 0x7fdca00af180 2023-08-25T03:34:46.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.908+0000 7fdca58b2700 1 --2- 172.21.15.2:0/1509502403 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fdc8406a0e0 0x7fdc8406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:46.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.908+0000 7fdca58b2700 1 --2- 172.21.15.2:0/1509502403 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fdc8406a0e0 0x7fdc8406c5a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdc90000f60 tx=0x7fdc90009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:46.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:46.913+0000 7fdc967fc700 1 -- 172.21.15.2:0/1509502403 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdc9c053a80 con 0x7fdca00af180 2023-08-25T03:34:47.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.112+0000 7fdca7b16700 1 -- 172.21.15.2:0/1509502403 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdca00ae800 con 0x7fdca00af180 2023-08-25T03:34:47.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.112+0000 7fdc967fc700 1 -- 172.21.15.2:0/1509502403 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdc9c0570a0 con 0x7fdca00af180 2023-08-25T03:34:47.114 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-08-25T03:34:47.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.114+0000 7fdca7b16700 1 -- 172.21.15.2:0/1509502403 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fdc8406a0e0 msgr2=0x7fdc8406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:47.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.114+0000 7fdca7b16700 1 --2- 172.21.15.2:0/1509502403 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fdc8406a0e0 0x7fdc8406c5a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdc90000f60 tx=0x7fdc90009040 comp rx=0 tx=0).stop 2023-08-25T03:34:47.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.114+0000 7fdca7b16700 1 -- 172.21.15.2:0/1509502403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdca00af180 msgr2=0x7fdca00cabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:47.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.114+0000 7fdca7b16700 1 --2- 172.21.15.2:0/1509502403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdca00af180 0x7fdca00cabd0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c00c770 tx=0x7fdc9c00ca80 comp rx=0 tx=0).stop 2023-08-25T03:34:47.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.114+0000 7fdca7b16700 1 -- 172.21.15.2:0/1509502403 shutdown_connections 2023-08-25T03:34:47.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.114+0000 7fdca7b16700 1 --2- 172.21.15.2:0/1509502403 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdca01607f0 0x7fdca00cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:47.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.114+0000 7fdca7b16700 1 --2- 172.21.15.2:0/1509502403 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdca006a8f0 0x7fdca01f0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:47.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.114+0000 7fdca7b16700 1 --2- 172.21.15.2:0/1509502403 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fdc8406a0e0 0x7fdc8406c5a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:47.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.114+0000 7fdca7b16700 1 --2- 172.21.15.2:0/1509502403 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdca00af180 0x7fdca00cabd0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:47.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.114+0000 7fdca7b16700 1 -- 172.21.15.2:0/1509502403 >> 172.21.15.2:0/1509502403 conn(0x7fdca00cfd90 msgr2=0x7fdca00ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:47.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.114+0000 7fdca7b16700 1 -- 172.21.15.2:0/1509502403 shutdown_connections 2023-08-25T03:34:47.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.114+0000 7fdca7b16700 1 -- 172.21.15.2:0/1509502403 wait complete. 2023-08-25T03:34:47.128 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T03:34:47.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.254+0000 7fa84e2b9700 1 Processor -- start 2023-08-25T03:34:47.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.254+0000 7fa84e2b9700 1 -- start start 2023-08-25T03:34:47.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.255+0000 7fa84e2b9700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa8480b1fb0 0x7fa848160f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:47.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.255+0000 7fa84e2b9700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa848161450 0x7fa8480cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:47.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.255+0000 7fa84e2b9700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa84806d450 con 0x7fa8480ce0b0 2023-08-25T03:34:47.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.255+0000 7fa84e2b9700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa84806d730 con 0x7fa848161450 2023-08-25T03:34:47.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.255+0000 7fa84e2b9700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84806da10 con 0x7fa8480b1fb0 2023-08-25T03:34:47.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.255+0000 7fa84c856700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fa8480ce0b0 0x7fa84806b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47320/0 (socket says 172.21.15.2:47320) 2023-08-25T03:34:47.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.255+0000 7fa84c856700 1 -- 172.21.15.2:0/3357031595 learned_addr learned my addr 172.21.15.2:0/3357031595 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:47.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.255+0000 7fa8477fe700 1 --2- 172.21.15.2:0/3357031595 >> v2:172.21.15.31:3300/0 conn(0x7fa848161450 0x7fa8480cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:47.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.255+0000 7fa847fff700 1 --2- 172.21.15.2:0/3357031595 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa8480b1fb0 0x7fa848160f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:47.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.256+0000 7fa846ffd700 1 -- 172.21.15.2:0/3357031595 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2720024593 0 0) 0x7fa84806d450 con 0x7fa8480ce0b0 2023-08-25T03:34:47.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.256+0000 7fa846ffd700 1 -- 172.21.15.2:0/3357031595 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8300035b0 con 0x7fa8480ce0b0 2023-08-25T03:34:47.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.256+0000 7fa8477fe700 1 -- 172.21.15.2:0/3357031595 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa8480b1fb0 msgr2=0x7fa848160f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:47.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.256+0000 7fa8477fe700 1 --2- 172.21.15.2:0/3357031595 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa8480b1fb0 0x7fa848160f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:47.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.256+0000 7fa8477fe700 1 -- 172.21.15.2:0/3357031595 >> v1:172.21.15.2:6789/0 conn(0x7fa8480ce0b0 legacy=0x7fa84806b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:47.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.256+0000 7fa8477fe700 1 -- 172.21.15.2:0/3357031595 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa84806dcf0 con 0x7fa848161450 2023-08-25T03:34:47.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.256+0000 7fa8477fe700 1 --2- 172.21.15.2:0/3357031595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa848161450 0x7fa8480cdb70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009840 tx=0x7fa83c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c74544b98c342703 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:47.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.256+0000 7fa846ffd700 1 -- 172.21.15.2:0/3357031595 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c011830 con 0x7fa848161450 2023-08-25T03:34:47.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.256+0000 7fa846ffd700 1 -- 172.21.15.2:0/3357031595 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83c011990 con 0x7fa848161450 2023-08-25T03:34:47.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.256+0000 7fa846ffd700 1 -- 172.21.15.2:0/3357031595 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c011d20 con 0x7fa848161450 2023-08-25T03:34:47.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.257+0000 7fa84e2b9700 1 -- 172.21.15.2:0/3357031595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa848161450 msgr2=0x7fa8480cdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:47.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.257+0000 7fa84e2b9700 1 --2- 172.21.15.2:0/3357031595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa848161450 0x7fa8480cdb70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fa83c009840 tx=0x7fa83c009b50 comp rx=0 tx=0).stop 2023-08-25T03:34:47.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.257+0000 7fa84e2b9700 1 -- 172.21.15.2:0/3357031595 shutdown_connections 2023-08-25T03:34:47.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.257+0000 7fa84e2b9700 1 --2- 172.21.15.2:0/3357031595 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa8480b1fb0 0x7fa848160f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:47.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.257+0000 7fa84e2b9700 1 --2- 172.21.15.2:0/3357031595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa848161450 0x7fa8480cdb70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:47.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.257+0000 7fa84e2b9700 1 -- 172.21.15.2:0/3357031595 >> 172.21.15.2:0/3357031595 conn(0x7fa8480cfea0 msgr2=0x7fa8480d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:47.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.257+0000 7fa84e2b9700 1 -- 172.21.15.2:0/3357031595 shutdown_connections 2023-08-25T03:34:47.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.257+0000 7fa84e2b9700 1 -- 172.21.15.2:0/3357031595 wait complete. 2023-08-25T03:34:47.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.258+0000 7fa84e2b9700 1 Processor -- start 2023-08-25T03:34:47.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.258+0000 7fa84e2b9700 1 -- start start 2023-08-25T03:34:47.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.259+0000 7fa84e2b9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8480b1fb0 0x7fa84815bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:47.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.259+0000 7fa84e2b9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8480ce0b0 0x7fa84815ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:47.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.259+0000 7fa84e2b9700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa84815f0f0 0x7fa8480c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:47.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.259+0000 7fa84e2b9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84815f570 con 0x7fa8480ce0b0 2023-08-25T03:34:47.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.259+0000 7fa84e2b9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8480c8ce0 con 0x7fa8480b1fb0 2023-08-25T03:34:47.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.259+0000 7fa84e2b9700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8480c80d0 con 0x7fa84815f0f0 2023-08-25T03:34:47.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.259+0000 7fa8477fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8480ce0b0 0x7fa84815ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:47.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.259+0000 7fa8477fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8480ce0b0 0x7fa84815ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60658/0 (socket says 172.21.15.2:60658) 2023-08-25T03:34:47.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.259+0000 7fa8477fe700 1 -- 172.21.15.2:0/1341750935 learned_addr learned my addr 172.21.15.2:0/1341750935 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:47.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.259+0000 7fa847fff700 1 --2- 172.21.15.2:0/1341750935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8480b1fb0 0x7fa84815bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:47.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.259+0000 7fa84c856700 1 --2- 172.21.15.2:0/1341750935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa84815f0f0 0x7fa8480c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:47.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.259+0000 7fa8477fe700 1 -- 172.21.15.2:0/1341750935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa84815f0f0 msgr2=0x7fa8480c5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:47.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.259+0000 7fa8477fe700 1 --2- 172.21.15.2:0/1341750935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa84815f0f0 0x7fa8480c5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:47.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.259+0000 7fa8477fe700 1 -- 172.21.15.2:0/1341750935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8480b1fb0 msgr2=0x7fa84815bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:47.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.259+0000 7fa8477fe700 1 --2- 172.21.15.2:0/1341750935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8480b1fb0 0x7fa84815bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:47.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.259+0000 7fa8477fe700 1 -- 172.21.15.2:0/1341750935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83c0094a0 con 0x7fa8480ce0b0 2023-08-25T03:34:47.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.259+0000 7fa847fff700 1 --2- 172.21.15.2:0/1341750935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8480b1fb0 0x7fa84815bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:47.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.259+0000 7fa8477fe700 1 --2- 172.21.15.2:0/1341750935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8480ce0b0 0x7fa84815ebb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0090b0 tx=0x7fa83c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:47.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.260+0000 7fa844ff9700 1 -- 172.21.15.2:0/1341750935 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c0036a0 con 0x7fa8480ce0b0 2023-08-25T03:34:47.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.260+0000 7fa844ff9700 1 -- 172.21.15.2:0/1341750935 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83c01f070 con 0x7fa8480ce0b0 2023-08-25T03:34:47.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.260+0000 7fa844ff9700 1 -- 172.21.15.2:0/1341750935 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83c003a60 con 0x7fa8480ce0b0 2023-08-25T03:34:47.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.260+0000 7fa84e2b9700 1 -- 172.21.15.2:0/1341750935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8480c83b0 con 0x7fa8480ce0b0 2023-08-25T03:34:47.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.260+0000 7fa84e2b9700 1 -- 172.21.15.2:0/1341750935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8480c8970 con 0x7fa8480ce0b0 2023-08-25T03:34:47.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.261+0000 7fa84e2b9700 1 -- 172.21.15.2:0/1341750935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa848066f70 con 0x7fa8480ce0b0 2023-08-25T03:34:47.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.261+0000 7fa844ff9700 1 -- 172.21.15.2:0/1341750935 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa83c003bc0 con 0x7fa8480ce0b0 2023-08-25T03:34:47.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.261+0000 7fa844ff9700 1 --2- 172.21.15.2:0/1341750935 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa83406a0e0 0x7fa83406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:47.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.262+0000 7fa844ff9700 1 -- 172.21.15.2:0/1341750935 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa83c021070 con 0x7fa8480ce0b0 2023-08-25T03:34:47.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.262+0000 7fa847fff700 1 --2- 172.21.15.2:0/1341750935 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa83406a0e0 0x7fa83406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:47.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.262+0000 7fa847fff700 1 --2- 172.21.15.2:0/1341750935 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa83406a0e0 0x7fa83406c5a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa838009b90 tx=0x7fa838008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:47.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.266+0000 7fa844ff9700 1 -- 172.21.15.2:0/1341750935 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa83c055490 con 0x7fa8480ce0b0 2023-08-25T03:34:47.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.485+0000 7fa84e2b9700 1 -- 172.21.15.2:0/1341750935 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa84815d8f0 con 0x7fa8480ce0b0 2023-08-25T03:34:47.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.485+0000 7fa844ff9700 1 -- 172.21.15.2:0/1341750935 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa83c02b080 con 0x7fa8480ce0b0 2023-08-25T03:34:47.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.488+0000 7fa84e2b9700 1 -- 172.21.15.2:0/1341750935 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa83406a0e0 msgr2=0x7fa83406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:47.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.488+0000 7fa84e2b9700 1 --2- 172.21.15.2:0/1341750935 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa83406a0e0 0x7fa83406c5a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa838009b90 tx=0x7fa838008040 comp rx=0 tx=0).stop 2023-08-25T03:34:47.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.488+0000 7fa84e2b9700 1 -- 172.21.15.2:0/1341750935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8480ce0b0 msgr2=0x7fa84815ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:47.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.488+0000 7fa84e2b9700 1 --2- 172.21.15.2:0/1341750935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8480ce0b0 0x7fa84815ebb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0090b0 tx=0x7fa83c004750 comp rx=0 tx=0).stop 2023-08-25T03:34:47.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.488+0000 7fa84e2b9700 1 -- 172.21.15.2:0/1341750935 shutdown_connections 2023-08-25T03:34:47.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.488+0000 7fa84e2b9700 1 --2- 172.21.15.2:0/1341750935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa84815f0f0 0x7fa8480c5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:47.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.488+0000 7fa84e2b9700 1 --2- 172.21.15.2:0/1341750935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa8480b1fb0 0x7fa84815bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:47.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.488+0000 7fa84e2b9700 1 --2- 172.21.15.2:0/1341750935 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa83406a0e0 0x7fa83406c5a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:47.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.488+0000 7fa84e2b9700 1 --2- 172.21.15.2:0/1341750935 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa8480ce0b0 0x7fa84815ebb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:47.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.488+0000 7fa84e2b9700 1 -- 172.21.15.2:0/1341750935 >> 172.21.15.2:0/1341750935 conn(0x7fa8480cfea0 msgr2=0x7fa848160310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:47.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.488+0000 7fa84e2b9700 1 -- 172.21.15.2:0/1341750935 shutdown_connections 2023-08-25T03:34:47.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.488+0000 7fa84e2b9700 1 -- 172.21.15.2:0/1341750935 wait complete. 2023-08-25T03:34:47.490 INFO:teuthology.orchestra.run.smithi002.stderr:listed 0 entries 2023-08-25T03:34:47.501 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-08-25T03:34:47.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:47.503 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-08-25T03:34:47.579 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-08-25T03:34:47.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:47.580 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-25T03:34:47.608 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:34:47.608 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:34:47.630 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:34:47.631 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:34:47.686 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:34:47.687 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:34:47.744 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:34:47.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.908+0000 7f22f5626700 1 Processor -- start 2023-08-25T03:34:47.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.908+0000 7f22f5626700 1 -- start start 2023-08-25T03:34:47.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.909+0000 7f22f5626700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f22f00cd0a0 0x7f22f0161020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:47.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.909+0000 7f22f5626700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f22f0071d90 0x7f22f00cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:47.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.909+0000 7f22f5626700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22f0072240 con 0x7f22f00ce050 2023-08-25T03:34:47.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.909+0000 7f22f5626700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22f0161fa0 con 0x7f22f0071d90 2023-08-25T03:34:47.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.909+0000 7f22f5626700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22f00ce460 con 0x7f22f00cd0a0 2023-08-25T03:34:47.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.909+0000 7f22ee7fc700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f22f0071d90 0x7f22f00cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:47.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.909+0000 7f22ee7fc700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f22f0071d90 0x7f22f00cdb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60664/0 (socket says 172.21.15.2:60664) 2023-08-25T03:34:47.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.909+0000 7f22ef7fe700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f22f00ce050 0x7f22f006d010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47334/0 (socket says 172.21.15.2:47334) 2023-08-25T03:34:47.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.909+0000 7f22ee7fc700 1 -- 172.21.15.2:0/1025110445 learned_addr learned my addr 172.21.15.2:0/1025110445 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:47.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.910+0000 7f22eeffd700 1 --2- 172.21.15.2:0/1025110445 >> v2:172.21.15.31:3300/0 conn(0x7f22f00cd0a0 0x7f22f0161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:47.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.910+0000 7f22ee7fc700 1 -- 172.21.15.2:0/1025110445 >> v2:172.21.15.31:3300/0 conn(0x7f22f00cd0a0 msgr2=0x7f22f0161020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:47.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.910+0000 7f22ee7fc700 1 --2- 172.21.15.2:0/1025110445 >> v2:172.21.15.31:3300/0 conn(0x7f22f00cd0a0 0x7f22f0161020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:47.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.910+0000 7f22ee7fc700 1 -- 172.21.15.2:0/1025110445 >> v1:172.21.15.2:6789/0 conn(0x7f22f00ce050 legacy=0x7f22f006d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:47.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.910+0000 7f22edffb700 1 -- 172.21.15.2:0/1025110445 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1239536434 0 0) 0x7f22e8001490 con 0x7f22f00ce050 2023-08-25T03:34:47.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.910+0000 7f22ee7fc700 1 -- 172.21.15.2:0/1025110445 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f0161700 con 0x7f22f0071d90 2023-08-25T03:34:47.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.910+0000 7f22eeffd700 1 --2- 172.21.15.2:0/1025110445 >> v2:172.21.15.31:3300/0 conn(0x7f22f00cd0a0 0x7f22f0161020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:47.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.910+0000 7f22ee7fc700 1 --2- 172.21.15.2:0/1025110445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22f0071d90 0x7f22f00cdb10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f22f00723e0 tx=0x7f22e4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8cf1ce3e7af335 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:47.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.910+0000 7f22edffb700 1 -- 172.21.15.2:0/1025110445 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e400b870 con 0x7f22f0071d90 2023-08-25T03:34:47.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.910+0000 7f22edffb700 1 -- 172.21.15.2:0/1025110445 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e400b9d0 con 0x7f22f0071d90 2023-08-25T03:34:47.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.910+0000 7f22edffb700 1 -- 172.21.15.2:0/1025110445 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e400bd60 con 0x7f22f0071d90 2023-08-25T03:34:47.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.911+0000 7f22f5626700 1 -- 172.21.15.2:0/1025110445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22f0071d90 msgr2=0x7f22f00cdb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:47.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.911+0000 7f22f5626700 1 --2- 172.21.15.2:0/1025110445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22f0071d90 0x7f22f00cdb10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f22f00723e0 tx=0x7f22e4009bd0 comp rx=0 tx=0).stop 2023-08-25T03:34:47.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.911+0000 7f22f5626700 1 -- 172.21.15.2:0/1025110445 shutdown_connections 2023-08-25T03:34:47.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.911+0000 7f22f5626700 1 --2- 172.21.15.2:0/1025110445 >> v2:172.21.15.31:3300/0 conn(0x7f22f00cd0a0 0x7f22f0161020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:47.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.911+0000 7f22f5626700 1 --2- 172.21.15.2:0/1025110445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22f0071d90 0x7f22f00cdb10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:47.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.911+0000 7f22f5626700 1 -- 172.21.15.2:0/1025110445 >> 172.21.15.2:0/1025110445 conn(0x7f22f00b1cf0 msgr2=0x7f22f00b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:47.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.911+0000 7f22f5626700 1 -- 172.21.15.2:0/1025110445 shutdown_connections 2023-08-25T03:34:47.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.911+0000 7f22f5626700 1 -- 172.21.15.2:0/1025110445 wait complete. 2023-08-25T03:34:47.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.912+0000 7f22f5626700 1 Processor -- start 2023-08-25T03:34:47.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.912+0000 7f22f5626700 1 -- start start 2023-08-25T03:34:47.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.913+0000 7f22f5626700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22f0071d90 0x7f22f00b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:47.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.913+0000 7f22f5626700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f00cd0a0 0x7f22f01598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:47.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.913+0000 7f22f5626700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f22f00ce050 0x7f22f00c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:47.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.913+0000 7f22f5626700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f0159de0 con 0x7f22f0071d90 2023-08-25T03:34:47.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.913+0000 7f22f5626700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f00c6db0 con 0x7f22f00cd0a0 2023-08-25T03:34:47.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.913+0000 7f22f5626700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22f00b3660 con 0x7f22f00ce050 2023-08-25T03:34:47.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.913+0000 7f22eeffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22f0071d90 0x7f22f00b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:47.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.913+0000 7f22eeffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22f0071d90 0x7f22f00b1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60670/0 (socket says 172.21.15.2:60670) 2023-08-25T03:34:47.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.913+0000 7f22eeffd700 1 -- 172.21.15.2:0/2699064188 learned_addr learned my addr 172.21.15.2:0/2699064188 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:47.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.913+0000 7f22eeffd700 1 -- 172.21.15.2:0/2699064188 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f22f00ce050 msgr2=0x7f22f00c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:47.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.913+0000 7f22ef7fe700 1 --2- 172.21.15.2:0/2699064188 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f22f00ce050 0x7f22f00c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:47.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.913+0000 7f22ee7fc700 1 --2- 172.21.15.2:0/2699064188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f00cd0a0 0x7f22f01598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:47.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.913+0000 7f22eeffd700 1 --2- 172.21.15.2:0/2699064188 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f22f00ce050 0x7f22f00c6560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:47.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.913+0000 7f22eeffd700 1 -- 172.21.15.2:0/2699064188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f00cd0a0 msgr2=0x7f22f01598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:47.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.913+0000 7f22eeffd700 1 --2- 172.21.15.2:0/2699064188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f00cd0a0 0x7f22f01598a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:47.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.913+0000 7f22eeffd700 1 -- 172.21.15.2:0/2699064188 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e4009550 con 0x7f22f0071d90 2023-08-25T03:34:47.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.913+0000 7f22eeffd700 1 --2- 172.21.15.2:0/2699064188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22f0071d90 0x7f22f00b1530 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f22e000c820 tx=0x7f22e000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:47.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.913+0000 7f22d77fe700 1 -- 172.21.15.2:0/2699064188 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e0011800 con 0x7f22f0071d90 2023-08-25T03:34:47.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.914+0000 7f22d77fe700 1 -- 172.21.15.2:0/2699064188 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e0011960 con 0x7f22f0071d90 2023-08-25T03:34:47.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.914+0000 7f22f5626700 1 -- 172.21.15.2:0/2699064188 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f00b2d30 con 0x7f22f0071d90 2023-08-25T03:34:47.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.914+0000 7f22d77fe700 1 -- 172.21.15.2:0/2699064188 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e0011d10 con 0x7f22f0071d90 2023-08-25T03:34:47.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.914+0000 7f22f5626700 1 -- 172.21.15.2:0/2699064188 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f00b32f0 con 0x7f22f0071d90 2023-08-25T03:34:47.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.914+0000 7f22f5626700 1 -- 172.21.15.2:0/2699064188 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22f0066f70 con 0x7f22f0071d90 2023-08-25T03:34:47.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.915+0000 7f22d77fe700 1 -- 172.21.15.2:0/2699064188 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f22e0018360 con 0x7f22f0071d90 2023-08-25T03:34:47.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.915+0000 7f22d77fe700 1 --2- 172.21.15.2:0/2699064188 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f22dc06a0e0 0x7f22dc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:47.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.916+0000 7f22d77fe700 1 -- 172.21.15.2:0/2699064188 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f22e0088560 con 0x7f22f0071d90 2023-08-25T03:34:47.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.916+0000 7f22ee7fc700 1 --2- 172.21.15.2:0/2699064188 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f22dc06a0e0 0x7f22dc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:47.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.916+0000 7f22ee7fc700 1 --2- 172.21.15.2:0/2699064188 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f22dc06a0e0 0x7f22dc06c5a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f22f01f5c30 tx=0x7f22e4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:47.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:47.920+0000 7f22d77fe700 1 -- 172.21.15.2:0/2699064188 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f22e00576b0 con 0x7f22f0071d90 2023-08-25T03:34:48.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.125+0000 7f22f5626700 1 -- 172.21.15.2:0/2699064188 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22f006a2f0 con 0x7f22f0071d90 2023-08-25T03:34:48.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.126+0000 7f22d77fe700 1 -- 172.21.15.2:0/2699064188 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18230 (secure 0 0 0) 0x7f22f006a2f0 con 0x7f22f0071d90 2023-08-25T03:34:48.127 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:48.127 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:34:44.588865+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:34:00.869118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T03:34:01.987016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.f","mappings":[{"from":1,"to":5}]},{"pgid":"4.2b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:34:48.136 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.128+0000 7f22f5626700 1 -- 172.21.15.2:0/2699064188 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f22dc06a0e0 msgr2=0x7f22dc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:48.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.128+0000 7f22f5626700 1 --2- 172.21.15.2:0/2699064188 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f22dc06a0e0 0x7f22dc06c5a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f22f01f5c30 tx=0x7f22e4008030 comp rx=0 tx=0).stop 2023-08-25T03:34:48.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.128+0000 7f22f5626700 1 -- 172.21.15.2:0/2699064188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22f0071d90 msgr2=0x7f22f00b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:48.137 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.128+0000 7f22f5626700 1 --2- 172.21.15.2:0/2699064188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22f0071d90 0x7f22f00b1530 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f22e000c820 tx=0x7f22e000cbe0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.128+0000 7f22f5626700 1 -- 172.21.15.2:0/2699064188 shutdown_connections 2023-08-25T03:34:48.138 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.128+0000 7f22f5626700 1 --2- 172.21.15.2:0/2699064188 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f22dc06a0e0 0x7f22dc06c5a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.139 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.128+0000 7f22f5626700 1 --2- 172.21.15.2:0/2699064188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f22f0071d90 0x7f22f00b1530 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.128+0000 7f22f5626700 1 --2- 172.21.15.2:0/2699064188 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f22f00ce050 0x7f22f00c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.128+0000 7f22f5626700 1 --2- 172.21.15.2:0/2699064188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22f00cd0a0 0x7f22f01598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.140 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.128+0000 7f22f5626700 1 -- 172.21.15.2:0/2699064188 >> 172.21.15.2:0/2699064188 conn(0x7f22f00b1cf0 msgr2=0x7f22f00d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:48.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.128+0000 7f22f5626700 1 -- 172.21.15.2:0/2699064188 shutdown_connections 2023-08-25T03:34:48.141 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.128+0000 7f22f5626700 1 -- 172.21.15.2:0/2699064188 wait complete. 2023-08-25T03:34:48.143 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T03:33:49.560643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T03:33:56.265184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T03:34:00.869118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T03:34:01.987016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T03:34:48.144 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T03:34:48.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.274+0000 7ff73c237700 1 Processor -- start 2023-08-25T03:34:48.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.274+0000 7ff73c237700 1 -- start start 2023-08-25T03:34:48.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.275+0000 7ff73c237700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff7340cedf0 0x7ff73406ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:48.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.275+0000 7ff73c237700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff73406eaf0 con 0x7ff734159f10 2023-08-25T03:34:48.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.275+0000 7ff73c237700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff73415e2c0 con 0x7ff73406f4c0 2023-08-25T03:34:48.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.275+0000 7ff73c237700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff73406e850 con 0x7ff7340cedf0 2023-08-25T03:34:48.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.275+0000 7ff7397d2700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff7340cedf0 0x7ff73406ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:48.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.275+0000 7ff739fd3700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7ff734159f10 0x7ff73415dae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47344/0 (socket says 172.21.15.2:47344) 2023-08-25T03:34:48.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.275+0000 7ff739fd3700 1 -- 172.21.15.2:0/3889615216 learned_addr learned my addr 172.21.15.2:0/3889615216 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:48.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.275+0000 7ff7397d2700 1 -- 172.21.15.2:0/3889615216 >> v1:172.21.15.31:6789/0 conn(0x7ff73406f4c0 legacy=0x7ff7340cacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:48.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.275+0000 7ff7397d2700 1 -- 172.21.15.2:0/3889615216 >> v1:172.21.15.2:6789/0 conn(0x7ff734159f10 legacy=0x7ff73415dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:48.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.275+0000 7ff7397d2700 1 -- 172.21.15.2:0/3889615216 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73415e5a0 con 0x7ff7340cedf0 2023-08-25T03:34:48.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.275+0000 7ff738fd1700 1 -- 172.21.15.2:0/3889615216 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3209900138 0 0) 0x7ff73406eaf0 con 0x7ff734159f10 2023-08-25T03:34:48.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.275+0000 7ff7397d2700 1 --2- 172.21.15.2:0/3889615216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7340cedf0 0x7ff73406ef80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff724009840 tx=0x7ff724009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3eb53eb4dfb3eda server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:48.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.275+0000 7ff738fd1700 1 -- 172.21.15.2:0/3889615216 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff724011830 con 0x7ff7340cedf0 2023-08-25T03:34:48.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.276+0000 7ff738fd1700 1 -- 172.21.15.2:0/3889615216 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff724011990 con 0x7ff7340cedf0 2023-08-25T03:34:48.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.276+0000 7ff738fd1700 1 -- 172.21.15.2:0/3889615216 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff724011d20 con 0x7ff7340cedf0 2023-08-25T03:34:48.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.276+0000 7ff73c237700 1 -- 172.21.15.2:0/3889615216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7340cedf0 msgr2=0x7ff73406ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:48.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.276+0000 7ff73c237700 1 --2- 172.21.15.2:0/3889615216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7340cedf0 0x7ff73406ef80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ff724009840 tx=0x7ff724009b50 comp rx=0 tx=0).stop 2023-08-25T03:34:48.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.276+0000 7ff73c237700 1 -- 172.21.15.2:0/3889615216 shutdown_connections 2023-08-25T03:34:48.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.276+0000 7ff73c237700 1 --2- 172.21.15.2:0/3889615216 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7340cedf0 0x7ff73406ef80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.276+0000 7ff73c237700 1 -- 172.21.15.2:0/3889615216 >> 172.21.15.2:0/3889615216 conn(0x7ff7340cfd90 msgr2=0x7ff7340d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:48.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.276+0000 7ff73c237700 1 -- 172.21.15.2:0/3889615216 shutdown_connections 2023-08-25T03:34:48.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.276+0000 7ff73c237700 1 -- 172.21.15.2:0/3889615216 wait complete. 2023-08-25T03:34:48.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.277+0000 7ff73c237700 1 Processor -- start 2023-08-25T03:34:48.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.277+0000 7ff73c237700 1 -- start start 2023-08-25T03:34:48.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.278+0000 7ff73c237700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff73406f4c0 0x7ff73414e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:48.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.278+0000 7ff73c237700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7340cedf0 0x7ff7341f03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:48.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.278+0000 7ff73c237700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff734159f10 0x7ff73415c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:48.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.278+0000 7ff73c237700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7341f1430 con 0x7ff7340cedf0 2023-08-25T03:34:48.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.278+0000 7ff73c237700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7341f0e70 con 0x7ff734159f10 2023-08-25T03:34:48.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.278+0000 7ff73c237700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7341f1150 con 0x7ff73406f4c0 2023-08-25T03:34:48.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.278+0000 7ff7397d2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7340cedf0 0x7ff7341f03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:48.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.278+0000 7ff7397d2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7340cedf0 0x7ff7341f03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60686/0 (socket says 172.21.15.2:60686) 2023-08-25T03:34:48.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.278+0000 7ff7397d2700 1 -- 172.21.15.2:0/3933011109 learned_addr learned my addr 172.21.15.2:0/3933011109 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:48.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.278+0000 7ff73a7d4700 1 --2- 172.21.15.2:0/3933011109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff734159f10 0x7ff73415c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:48.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.278+0000 7ff739fd3700 1 --2- 172.21.15.2:0/3933011109 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff73406f4c0 0x7ff73414e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:48.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.279+0000 7ff7397d2700 1 -- 172.21.15.2:0/3933011109 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff73406f4c0 msgr2=0x7ff73414e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:48.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.279+0000 7ff7397d2700 1 --2- 172.21.15.2:0/3933011109 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff73406f4c0 0x7ff73414e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.279+0000 7ff7397d2700 1 -- 172.21.15.2:0/3933011109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff734159f10 msgr2=0x7ff73415c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:48.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.279+0000 7ff7397d2700 1 --2- 172.21.15.2:0/3933011109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff734159f10 0x7ff73415c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.279+0000 7ff7397d2700 1 -- 172.21.15.2:0/3933011109 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7240094a0 con 0x7ff7340cedf0 2023-08-25T03:34:48.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.279+0000 7ff7397d2700 1 --2- 172.21.15.2:0/3933011109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7340cedf0 0x7ff7341f03f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff7240090b0 tx=0x7ff724004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:48.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.279+0000 7ff72a7fc700 1 -- 172.21.15.2:0/3933011109 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72401a980 con 0x7ff7340cedf0 2023-08-25T03:34:48.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.279+0000 7ff72a7fc700 1 -- 172.21.15.2:0/3933011109 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72401ab00 con 0x7ff7340cedf0 2023-08-25T03:34:48.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.279+0000 7ff72a7fc700 1 -- 172.21.15.2:0/3933011109 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff724024420 con 0x7ff7340cedf0 2023-08-25T03:34:48.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.279+0000 7ff73c237700 1 -- 172.21.15.2:0/3933011109 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7340b2b90 con 0x7ff7340cedf0 2023-08-25T03:34:48.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.279+0000 7ff73c237700 1 -- 172.21.15.2:0/3933011109 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7340c5760 con 0x7ff7340cedf0 2023-08-25T03:34:48.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.280+0000 7ff73c237700 1 -- 172.21.15.2:0/3933011109 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7340b29f0 con 0x7ff7340cedf0 2023-08-25T03:34:48.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.280+0000 7ff72a7fc700 1 -- 172.21.15.2:0/3933011109 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff724022070 con 0x7ff7340cedf0 2023-08-25T03:34:48.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.281+0000 7ff72a7fc700 1 --2- 172.21.15.2:0/3933011109 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff71806a0e0 0x7ff71806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:48.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.281+0000 7ff72a7fc700 1 -- 172.21.15.2:0/3933011109 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7ff724089890 con 0x7ff7340cedf0 2023-08-25T03:34:48.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.281+0000 7ff739fd3700 1 --2- 172.21.15.2:0/3933011109 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff71806a0e0 0x7ff71806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:48.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.281+0000 7ff739fd3700 1 --2- 172.21.15.2:0/3933011109 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff71806a0e0 0x7ff71806c5a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff730000f60 tx=0x7ff730007db0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:48.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.285+0000 7ff72a7fc700 1 -- 172.21.15.2:0/3933011109 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff7340b29f0 con 0x7ff7340cedf0 2023-08-25T03:34:48.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.485+0000 7ff73c237700 1 -- 172.21.15.2:0/3933011109 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff7340b30b0 con 0x7ff7340cedf0 2023-08-25T03:34:48.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.485+0000 7ff72a7fc700 1 -- 172.21.15.2:0/3933011109 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff7340b30b0 con 0x7ff7340cedf0 2023-08-25T03:34:48.486 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-08-25T03:34:48.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.487+0000 7ff73c237700 1 -- 172.21.15.2:0/3933011109 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff71806a0e0 msgr2=0x7ff71806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:48.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.487+0000 7ff73c237700 1 --2- 172.21.15.2:0/3933011109 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff71806a0e0 0x7ff71806c5a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff730000f60 tx=0x7ff730007db0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.487+0000 7ff73c237700 1 -- 172.21.15.2:0/3933011109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7340cedf0 msgr2=0x7ff7341f03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:48.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.487+0000 7ff73c237700 1 --2- 172.21.15.2:0/3933011109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7340cedf0 0x7ff7341f03f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ff7240090b0 tx=0x7ff724004750 comp rx=0 tx=0).stop 2023-08-25T03:34:48.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.487+0000 7ff73c237700 1 -- 172.21.15.2:0/3933011109 shutdown_connections 2023-08-25T03:34:48.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.487+0000 7ff73c237700 1 --2- 172.21.15.2:0/3933011109 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff734159f10 0x7ff73415c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.487+0000 7ff73c237700 1 --2- 172.21.15.2:0/3933011109 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff73406f4c0 0x7ff73414e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.487+0000 7ff73c237700 1 --2- 172.21.15.2:0/3933011109 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff71806a0e0 0x7ff71806c5a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.487+0000 7ff73c237700 1 --2- 172.21.15.2:0/3933011109 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff7340cedf0 0x7ff7341f03f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.487+0000 7ff73c237700 1 -- 172.21.15.2:0/3933011109 >> 172.21.15.2:0/3933011109 conn(0x7ff7340cfd90 msgr2=0x7ff734158080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:48.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.487+0000 7ff73c237700 1 -- 172.21.15.2:0/3933011109 shutdown_connections 2023-08-25T03:34:48.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.487+0000 7ff73c237700 1 -- 172.21.15.2:0/3933011109 wait complete. 2023-08-25T03:34:48.500 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T03:34:48.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.629+0000 7f783ceb5700 1 Processor -- start 2023-08-25T03:34:48.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.629+0000 7f783ceb5700 1 -- start start 2023-08-25T03:34:48.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.630+0000 7f783ceb5700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78380cfb80 0x7f78380cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:48.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.630+0000 7f783ceb5700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78380d0000 con 0x7f783815b170 2023-08-25T03:34:48.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.630+0000 7f783ceb5700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78380cf6f0 con 0x7f78380cdd30 2023-08-25T03:34:48.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.630+0000 7f783ceb5700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78380ce1c0 con 0x7f78380cfb80 2023-08-25T03:34:48.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.631+0000 7f783659c700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f783815b170 0x7f78380cc640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47358/0 (socket says 172.21.15.2:47358) 2023-08-25T03:34:48.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.631+0000 7f783659c700 1 -- 172.21.15.2:0/1408010460 learned_addr learned my addr 172.21.15.2:0/1408010460 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:48.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.631+0000 7f7835d9b700 1 --2- 172.21.15.2:0/1408010460 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78380cfb80 0x7f78380cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:48.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.631+0000 7f783559a700 1 -- 172.21.15.2:0/1408010460 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3268550472 0 0) 0x7f7828001290 con 0x7f783815b170 2023-08-25T03:34:48.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.631+0000 7f783559a700 1 -- 172.21.15.2:0/1408010460 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78200035b0 con 0x7f783815b170 2023-08-25T03:34:48.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.631+0000 7f783559a700 1 -- 172.21.15.2:0/1408010460 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1137407815 0 0) 0x7f78200035b0 con 0x7f783815b170 2023-08-25T03:34:48.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.631+0000 7f783559a700 1 -- 172.21.15.2:0/1408010460 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78380cfb80 msgr2=0x7f78380cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:48.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.631+0000 7f783559a700 1 --2- 172.21.15.2:0/1408010460 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78380cfb80 0x7f78380cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.631+0000 7f783559a700 1 -- 172.21.15.2:0/1408010460 >> v1:172.21.15.31:6789/0 conn(0x7f78380cdd30 legacy=0x7f783806d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:48.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.631+0000 7f783559a700 1 -- 172.21.15.2:0/1408010460 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78380d1d50 con 0x7f783815b170 2023-08-25T03:34:48.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.631+0000 7f783559a700 1 -- 172.21.15.2:0/1408010460 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f7828001980 con 0x7f783815b170 2023-08-25T03:34:48.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.631+0000 7f783559a700 1 -- 172.21.15.2:0/1408010460 >> v1:172.21.15.2:6789/0 conn(0x7f783815b170 legacy=0x7f78380cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:48.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.631+0000 7f783559a700 1 --2- 172.21.15.2:0/1408010460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7820003a30 0x7f7820003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:48.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.631+0000 7f783559a700 1 --2- 172.21.15.2:0/1408010460 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78380cfb80 0x7f7820004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:48.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.631+0000 7f783559a700 1 --2- 172.21.15.2:0/1408010460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7820004d90 0x7f7820005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:48.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.631+0000 7f783559a700 1 -- 172.21.15.2:0/1408010460 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78380ce1c0 con 0x7f7820003a30 2023-08-25T03:34:48.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.631+0000 7f783559a700 1 -- 172.21.15.2:0/1408010460 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782000bac0 con 0x7f7820004d90 2023-08-25T03:34:48.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.631+0000 7f783559a700 1 -- 172.21.15.2:0/1408010460 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f782000bc80 con 0x7f78380cfb80 2023-08-25T03:34:48.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.631+0000 7f783659c700 1 --2- 172.21.15.2:0/1408010460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7820003a30 0x7f7820003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:48.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.631+0000 7f7835d9b700 1 --2- 172.21.15.2:0/1408010460 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78380cfb80 0x7f7820004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:48.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.631+0000 7f7836d9d700 1 --2- 172.21.15.2:0/1408010460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7820004d90 0x7f7820005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:48.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.631+0000 7f783659c700 1 -- 172.21.15.2:0/1408010460 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78380cfb80 msgr2=0x7f7820004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:48.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.631+0000 7f783659c700 1 --2- 172.21.15.2:0/1408010460 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78380cfb80 0x7f7820004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.632+0000 7f783659c700 1 -- 172.21.15.2:0/1408010460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7820004d90 msgr2=0x7f7820005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:48.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.632+0000 7f783659c700 1 --2- 172.21.15.2:0/1408010460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7820004d90 0x7f7820005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.632+0000 7f783659c700 1 -- 172.21.15.2:0/1408010460 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78200035b0 con 0x7f7820003a30 2023-08-25T03:34:48.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.632+0000 7f7836d9d700 1 --2- 172.21.15.2:0/1408010460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7820004d90 0x7f7820005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:48.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.632+0000 7f783659c700 1 --2- 172.21.15.2:0/1408010460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7820003a30 0x7f7820003e50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f782800c210 tx=0x7f7820005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ffffee3ff702572 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:48.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.632+0000 7f783559a700 1 -- 172.21.15.2:0/1408010460 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782800cd70 con 0x7f7820003a30 2023-08-25T03:34:48.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.632+0000 7f783559a700 1 -- 172.21.15.2:0/1408010460 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7828005830 con 0x7f7820003a30 2023-08-25T03:34:48.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.632+0000 7f783559a700 1 -- 172.21.15.2:0/1408010460 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7828005bc0 con 0x7f7820003a30 2023-08-25T03:34:48.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.632+0000 7f783ceb5700 1 -- 172.21.15.2:0/1408010460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7820003a30 msgr2=0x7f7820003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:48.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.632+0000 7f783ceb5700 1 --2- 172.21.15.2:0/1408010460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7820003a30 0x7f7820003e50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f782800c210 tx=0x7f7820005880 comp rx=0 tx=0).stop 2023-08-25T03:34:48.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.632+0000 7f7836d9d700 1 -- 172.21.15.2:0/1408010460 reap_dead start 2023-08-25T03:34:48.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.632+0000 7f783ceb5700 1 -- 172.21.15.2:0/1408010460 shutdown_connections 2023-08-25T03:34:48.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.632+0000 7f783ceb5700 1 -- 172.21.15.2:0/1408010460 >> 172.21.15.2:0/1408010460 conn(0x7f783809a200 msgr2=0x7f78380d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:48.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.632+0000 7f783ceb5700 1 -- 172.21.15.2:0/1408010460 shutdown_connections 2023-08-25T03:34:48.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.632+0000 7f783ceb5700 1 -- 172.21.15.2:0/1408010460 wait complete. 2023-08-25T03:34:48.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.634+0000 7f783ceb5700 1 Processor -- start 2023-08-25T03:34:48.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.634+0000 7f783ceb5700 1 -- start start 2023-08-25T03:34:48.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.635+0000 7f783ceb5700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7820003a30 0x7f78380b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:48.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.635+0000 7f783ceb5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78380b5830 0x7f7838159530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:48.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.635+0000 7f783ceb5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78380cc310 0x7f78380cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:48.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.635+0000 7f783ceb5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78380cccd0 con 0x7f78380b5830 2023-08-25T03:34:48.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.635+0000 7f783ceb5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78380b3fd0 con 0x7f78380cc310 2023-08-25T03:34:48.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.635+0000 7f783ceb5700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f783806b6d0 con 0x7f7820003a30 2023-08-25T03:34:48.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.635+0000 7f7835d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78380b5830 0x7f7838159530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:48.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.635+0000 7f7835d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78380b5830 0x7f7838159530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60706/0 (socket says 172.21.15.2:60706) 2023-08-25T03:34:48.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.635+0000 7f7835d9b700 1 -- 172.21.15.2:0/2084387148 learned_addr learned my addr 172.21.15.2:0/2084387148 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:48.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.636+0000 7f783659c700 1 --2- 172.21.15.2:0/2084387148 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7820003a30 0x7f78380b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:48.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.636+0000 7f7836d9d700 1 --2- 172.21.15.2:0/2084387148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78380cc310 0x7f78380cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:48.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.636+0000 7f7835d9b700 1 -- 172.21.15.2:0/2084387148 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7820003a30 msgr2=0x7f78380b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:48.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.636+0000 7f7835d9b700 1 --2- 172.21.15.2:0/2084387148 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7820003a30 0x7f78380b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.636+0000 7f7835d9b700 1 -- 172.21.15.2:0/2084387148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78380cc310 msgr2=0x7f78380cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:48.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.636+0000 7f7835d9b700 1 --2- 172.21.15.2:0/2084387148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78380cc310 0x7f78380cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.636+0000 7f7835d9b700 1 -- 172.21.15.2:0/2084387148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f783806b9b0 con 0x7f78380b5830 2023-08-25T03:34:48.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.636+0000 7f7835d9b700 1 --2- 172.21.15.2:0/2084387148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78380b5830 0x7f7838159530 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f782c011aa0 tx=0x7f782c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:48.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.636+0000 7f7826ffd700 1 -- 172.21.15.2:0/2084387148 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782c00f980 con 0x7f78380b5830 2023-08-25T03:34:48.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.636+0000 7f7826ffd700 1 -- 172.21.15.2:0/2084387148 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782c00fae0 con 0x7f78380b5830 2023-08-25T03:34:48.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.636+0000 7f7826ffd700 1 -- 172.21.15.2:0/2084387148 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782c00fe90 con 0x7f78380b5830 2023-08-25T03:34:48.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.636+0000 7f783ceb5700 1 -- 172.21.15.2:0/2084387148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f783806bc90 con 0x7f78380b5830 2023-08-25T03:34:48.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.636+0000 7f783ceb5700 1 -- 172.21.15.2:0/2084387148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f783806c250 con 0x7f78380b5830 2023-08-25T03:34:48.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.637+0000 7f783ceb5700 1 -- 172.21.15.2:0/2084387148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f783806c5c0 con 0x7f78380b5830 2023-08-25T03:34:48.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.637+0000 7f7826ffd700 1 -- 172.21.15.2:0/2084387148 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f782c014440 con 0x7f78380b5830 2023-08-25T03:34:48.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.638+0000 7f7826ffd700 1 --2- 172.21.15.2:0/2084387148 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f781806a0e0 0x7f781806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:48.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.638+0000 7f7826ffd700 1 -- 172.21.15.2:0/2084387148 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f782c088cd0 con 0x7f78380b5830 2023-08-25T03:34:48.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.638+0000 7f783659c700 1 --2- 172.21.15.2:0/2084387148 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f781806a0e0 0x7f781806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:48.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.638+0000 7f783659c700 1 --2- 172.21.15.2:0/2084387148 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f781806a0e0 0x7f781806c5a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7828008fd0 tx=0x7f7828007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:48.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.642+0000 7f7826ffd700 1 -- 172.21.15.2:0/2084387148 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f782c054440 con 0x7f78380b5830 2023-08-25T03:34:48.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.842+0000 7f783ceb5700 1 -- 172.21.15.2:0/2084387148 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f78380746d0 con 0x7f78380b5830 2023-08-25T03:34:48.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.842+0000 7f7826ffd700 1 -- 172.21.15.2:0/2084387148 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f782c022020 con 0x7f78380b5830 2023-08-25T03:34:48.844 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 8 2023-08-25T03:34:48.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.844+0000 7f783ceb5700 1 -- 172.21.15.2:0/2084387148 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f781806a0e0 msgr2=0x7f781806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:48.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.844+0000 7f783ceb5700 1 --2- 172.21.15.2:0/2084387148 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f781806a0e0 0x7f781806c5a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7828008fd0 tx=0x7f7828007000 comp rx=0 tx=0).stop 2023-08-25T03:34:48.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.844+0000 7f783ceb5700 1 -- 172.21.15.2:0/2084387148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78380b5830 msgr2=0x7f7838159530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:48.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.844+0000 7f783ceb5700 1 --2- 172.21.15.2:0/2084387148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78380b5830 0x7f7838159530 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f782c011aa0 tx=0x7f782c011e60 comp rx=0 tx=0).stop 2023-08-25T03:34:48.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.844+0000 7f783ceb5700 1 -- 172.21.15.2:0/2084387148 shutdown_connections 2023-08-25T03:34:48.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.844+0000 7f783ceb5700 1 --2- 172.21.15.2:0/2084387148 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78380cc310 0x7f78380cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.844+0000 7f783ceb5700 1 --2- 172.21.15.2:0/2084387148 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7820003a30 0x7f78380b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.844+0000 7f783ceb5700 1 --2- 172.21.15.2:0/2084387148 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f781806a0e0 0x7f781806c5a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.844+0000 7f783ceb5700 1 --2- 172.21.15.2:0/2084387148 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78380b5830 0x7f7838159530 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.844+0000 7f783ceb5700 1 -- 172.21.15.2:0/2084387148 >> 172.21.15.2:0/2084387148 conn(0x7f783809a200 msgr2=0x7f78380b08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:48.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.844+0000 7f783ceb5700 1 -- 172.21.15.2:0/2084387148 shutdown_connections 2023-08-25T03:34:48.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.844+0000 7f783ceb5700 1 -- 172.21.15.2:0/2084387148 wait complete. 2023-08-25T03:34:48.858 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T03:34:48.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.985+0000 7fa1274bf700 1 Processor -- start 2023-08-25T03:34:48.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.986+0000 7fa1274bf700 1 -- start start 2023-08-25T03:34:48.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.986+0000 7fa1274bf700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa12006a660 0x7fa120072330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:48.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.986+0000 7fa1274bf700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa12006aa40 0x7fa12015f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:48.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.986+0000 7fa1274bf700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1200b1e30 con 0x7fa120066050 2023-08-25T03:34:48.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.986+0000 7fa1274bf700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1200b12b0 con 0x7fa12006aa40 2023-08-25T03:34:48.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.986+0000 7fa1274bf700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1200b1590 con 0x7fa12006a660 2023-08-25T03:34:48.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.987+0000 7fa125a5c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa12006aa40 0x7fa12015f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:48.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.987+0000 7fa124a5a700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fa120066050 0x7fa12015f200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47380/0 (socket says 172.21.15.2:47380) 2023-08-25T03:34:48.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.987+0000 7fa124a5a700 1 -- 172.21.15.2:0/3518949416 learned_addr learned my addr 172.21.15.2:0/3518949416 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:48.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.987+0000 7fa12525b700 1 --2- 172.21.15.2:0/3518949416 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa12006a660 0x7fa120072330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:48.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.987+0000 7fa125a5c700 1 -- 172.21.15.2:0/3518949416 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa12006a660 msgr2=0x7fa120072330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:48.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.987+0000 7fa125a5c700 1 --2- 172.21.15.2:0/3518949416 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa12006a660 0x7fa120072330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.987+0000 7fa125a5c700 1 -- 172.21.15.2:0/3518949416 >> v1:172.21.15.2:6789/0 conn(0x7fa120066050 legacy=0x7fa12015f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:48.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.987+0000 7fa125a5c700 1 -- 172.21.15.2:0/3518949416 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1200b1870 con 0x7fa12006aa40 2023-08-25T03:34:48.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.987+0000 7fa125a5c700 1 --2- 172.21.15.2:0/3518949416 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa12006aa40 0x7fa12015f910 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa118009d90 tx=0x7fa11800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a36d090c092e9de server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:48.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.987+0000 7fa117fff700 1 -- 172.21.15.2:0/3518949416 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa11800e070 con 0x7fa12006aa40 2023-08-25T03:34:48.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.987+0000 7fa117fff700 1 -- 172.21.15.2:0/3518949416 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa11800bbd0 con 0x7fa12006aa40 2023-08-25T03:34:48.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.987+0000 7fa117fff700 1 -- 172.21.15.2:0/3518949416 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa118012430 con 0x7fa12006aa40 2023-08-25T03:34:48.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.988+0000 7fa1274bf700 1 -- 172.21.15.2:0/3518949416 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa12006aa40 msgr2=0x7fa12015f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:48.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.988+0000 7fa1274bf700 1 --2- 172.21.15.2:0/3518949416 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa12006aa40 0x7fa12015f910 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa118009d90 tx=0x7fa11800b330 comp rx=0 tx=0).stop 2023-08-25T03:34:48.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.988+0000 7fa1274bf700 1 -- 172.21.15.2:0/3518949416 shutdown_connections 2023-08-25T03:34:48.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.988+0000 7fa1274bf700 1 --2- 172.21.15.2:0/3518949416 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa12006aa40 0x7fa12015f910 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.988+0000 7fa1274bf700 1 --2- 172.21.15.2:0/3518949416 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa12006a660 0x7fa120072330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.988+0000 7fa1274bf700 1 -- 172.21.15.2:0/3518949416 >> 172.21.15.2:0/3518949416 conn(0x7fa1200cf500 msgr2=0x7fa1200cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:48.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.988+0000 7fa1274bf700 1 -- 172.21.15.2:0/3518949416 shutdown_connections 2023-08-25T03:34:48.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.988+0000 7fa1274bf700 1 -- 172.21.15.2:0/3518949416 wait complete. 2023-08-25T03:34:48.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.989+0000 7fa1274bf700 1 Processor -- start 2023-08-25T03:34:48.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.989+0000 7fa1274bf700 1 -- start start 2023-08-25T03:34:48.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.990+0000 7fa1274bf700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa120066050 0x7fa12015b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:48.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.990+0000 7fa1274bf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa12006a660 0x7fa1200c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:48.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.990+0000 7fa1274bf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa12006aa40 0x7fa1200c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:48.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.990+0000 7fa1274bf700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1200c94f0 con 0x7fa12006aa40 2023-08-25T03:34:48.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.990+0000 7fa1274bf700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1200ad4a0 con 0x7fa12006a660 2023-08-25T03:34:48.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.990+0000 7fa1274bf700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1200ad780 con 0x7fa120066050 2023-08-25T03:34:48.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.990+0000 7fa125a5c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa12006aa40 0x7fa1200c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:48.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.990+0000 7fa125a5c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa12006aa40 0x7fa1200c8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60722/0 (socket says 172.21.15.2:60722) 2023-08-25T03:34:48.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.990+0000 7fa125a5c700 1 -- 172.21.15.2:0/1998500214 learned_addr learned my addr 172.21.15.2:0/1998500214 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:48.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.990+0000 7fa12525b700 1 --2- 172.21.15.2:0/1998500214 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa120066050 0x7fa12015b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:48.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.990+0000 7fa125a5c700 1 -- 172.21.15.2:0/1998500214 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa120066050 msgr2=0x7fa12015b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:48.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.990+0000 7fa125a5c700 1 --2- 172.21.15.2:0/1998500214 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa120066050 0x7fa12015b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.990+0000 7fa125a5c700 1 -- 172.21.15.2:0/1998500214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa12006a660 msgr2=0x7fa1200c8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:48.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.990+0000 7fa125a5c700 1 --2- 172.21.15.2:0/1998500214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa12006a660 0x7fa1200c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:48.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.990+0000 7fa125a5c700 1 -- 172.21.15.2:0/1998500214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa118009a70 con 0x7fa12006aa40 2023-08-25T03:34:48.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.990+0000 7fa125a5c700 1 --2- 172.21.15.2:0/1998500214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa12006aa40 0x7fa1200c8fb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa118014040 tx=0x7fa118004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:48.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.990+0000 7fa115ffb700 1 -- 172.21.15.2:0/1998500214 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa118012430 con 0x7fa12006aa40 2023-08-25T03:34:48.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.990+0000 7fa115ffb700 1 -- 172.21.15.2:0/1998500214 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa118020070 con 0x7fa12006aa40 2023-08-25T03:34:48.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.991+0000 7fa1274bf700 1 -- 172.21.15.2:0/1998500214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1200c2b30 con 0x7fa12006aa40 2023-08-25T03:34:48.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.991+0000 7fa115ffb700 1 -- 172.21.15.2:0/1998500214 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa118003ea0 con 0x7fa12006aa40 2023-08-25T03:34:48.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.991+0000 7fa1274bf700 1 -- 172.21.15.2:0/1998500214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1200ada60 con 0x7fa12006aa40 2023-08-25T03:34:48.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.992+0000 7fa1274bf700 1 -- 172.21.15.2:0/1998500214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1200c3050 con 0x7fa12006aa40 2023-08-25T03:34:48.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.992+0000 7fa115ffb700 1 -- 172.21.15.2:0/1998500214 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa118004280 con 0x7fa12006aa40 2023-08-25T03:34:48.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.992+0000 7fa115ffb700 1 --2- 172.21.15.2:0/1998500214 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa10406a0e0 0x7fa10406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:49.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.993+0000 7fa115ffb700 1 -- 172.21.15.2:0/1998500214 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa118022070 con 0x7fa12006aa40 2023-08-25T03:34:49.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.993+0000 7fa12525b700 1 --2- 172.21.15.2:0/1998500214 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa10406a0e0 0x7fa10406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:49.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.993+0000 7fa12525b700 1 --2- 172.21.15.2:0/1998500214 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa10406a0e0 0x7fa10406c5a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa120151c60 tx=0x7fa110008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:49.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:48.997+0000 7fa115ffb700 1 -- 172.21.15.2:0/1998500214 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa1200c3050 con 0x7fa12006aa40 2023-08-25T03:34:49.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.194+0000 7fa1274bf700 1 -- 172.21.15.2:0/1998500214 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa1200c3050 con 0x7fa12006aa40 2023-08-25T03:34:49.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.195+0000 7fa115ffb700 1 -- 172.21.15.2:0/1998500214 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa1200c3050 con 0x7fa12006aa40 2023-08-25T03:34:49.196 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-08-25T03:34:49.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.197+0000 7fa1274bf700 1 -- 172.21.15.2:0/1998500214 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa10406a0e0 msgr2=0x7fa10406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:49.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.197+0000 7fa1274bf700 1 --2- 172.21.15.2:0/1998500214 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa10406a0e0 0x7fa10406c5a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa120151c60 tx=0x7fa110008040 comp rx=0 tx=0).stop 2023-08-25T03:34:49.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.197+0000 7fa1274bf700 1 -- 172.21.15.2:0/1998500214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa12006aa40 msgr2=0x7fa1200c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:49.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.197+0000 7fa1274bf700 1 --2- 172.21.15.2:0/1998500214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa12006aa40 0x7fa1200c8fb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa118014040 tx=0x7fa118004ad0 comp rx=0 tx=0).stop 2023-08-25T03:34:49.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.197+0000 7fa1274bf700 1 -- 172.21.15.2:0/1998500214 shutdown_connections 2023-08-25T03:34:49.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.197+0000 7fa1274bf700 1 --2- 172.21.15.2:0/1998500214 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa10406a0e0 0x7fa10406c5a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:49.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.197+0000 7fa1274bf700 1 --2- 172.21.15.2:0/1998500214 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa12006aa40 0x7fa1200c8fb0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:49.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.197+0000 7fa1274bf700 1 --2- 172.21.15.2:0/1998500214 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa120066050 0x7fa12015b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:49.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.197+0000 7fa1274bf700 1 --2- 172.21.15.2:0/1998500214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa12006a660 0x7fa1200c8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:49.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.197+0000 7fa1274bf700 1 -- 172.21.15.2:0/1998500214 >> 172.21.15.2:0/1998500214 conn(0x7fa1200cf500 msgr2=0x7fa12006c210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:49.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.197+0000 7fa1274bf700 1 -- 172.21.15.2:0/1998500214 shutdown_connections 2023-08-25T03:34:49.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.197+0000 7fa1274bf700 1 -- 172.21.15.2:0/1998500214 wait complete. 2023-08-25T03:34:49.210 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T03:34:49.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.338+0000 7fae45070700 1 Processor -- start 2023-08-25T03:34:49.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.338+0000 7fae45070700 1 -- start start 2023-08-25T03:34:49.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.338+0000 7fae45070700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fae4015d360 0x7fae4006b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:49.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.338+0000 7fae45070700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fae400cfd80 0x7fae4006ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:49.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.338+0000 7fae45070700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae400d1b40 con 0x7fae4006bf80 2023-08-25T03:34:49.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.338+0000 7fae45070700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae4006e0e0 con 0x7fae4015d360 2023-08-25T03:34:49.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.339+0000 7fae45070700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae400d0620 con 0x7fae400cfd80 2023-08-25T03:34:49.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.339+0000 7fae3ed9d700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fae4015d360 0x7fae4006b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:49.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.339+0000 7fae3ed9d700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fae4015d360 0x7fae4006b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60726/0 (socket says 172.21.15.2:60726) 2023-08-25T03:34:49.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.339+0000 7fae3ed9d700 1 -- 172.21.15.2:0/822537470 learned_addr learned my addr 172.21.15.2:0/822537470 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:49.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.339+0000 7fae3e59c700 1 --2- 172.21.15.2:0/822537470 >> v2:172.21.15.31:3300/0 conn(0x7fae400cfd80 0x7fae4006ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:49.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.339+0000 7fae3ed9d700 1 -- 172.21.15.2:0/822537470 >> v2:172.21.15.31:3300/0 conn(0x7fae400cfd80 msgr2=0x7fae4006ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:49.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.339+0000 7fae3ed9d700 1 --2- 172.21.15.2:0/822537470 >> v2:172.21.15.31:3300/0 conn(0x7fae400cfd80 0x7fae4006ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:49.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.339+0000 7fae3ed9d700 1 -- 172.21.15.2:0/822537470 >> v1:172.21.15.31:6789/0 conn(0x7fae4006bf80 legacy=0x7fae40160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:49.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.339+0000 7fae3ed9d700 1 -- 172.21.15.2:0/822537470 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae400d0340 con 0x7fae4015d360 2023-08-25T03:34:49.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.339+0000 7fae3e59c700 1 --2- 172.21.15.2:0/822537470 >> v2:172.21.15.31:3300/0 conn(0x7fae400cfd80 0x7fae4006ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:49.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.339+0000 7fae3ed9d700 1 --2- 172.21.15.2:0/822537470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae4015d360 0x7fae4006b500 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fae40071300 tx=0x7fae30009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4dd9bce2ecc7be8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:49.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.339+0000 7fae3dd9b700 1 -- 172.21.15.2:0/822537470 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30005e30 con 0x7fae4015d360 2023-08-25T03:34:49.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.339+0000 7fae3dd9b700 1 -- 172.21.15.2:0/822537470 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae30006030 con 0x7fae4015d360 2023-08-25T03:34:49.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.339+0000 7fae3dd9b700 1 -- 172.21.15.2:0/822537470 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae300063e0 con 0x7fae4015d360 2023-08-25T03:34:49.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.340+0000 7fae45070700 1 -- 172.21.15.2:0/822537470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae4015d360 msgr2=0x7fae4006b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:49.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.340+0000 7fae45070700 1 --2- 172.21.15.2:0/822537470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae4015d360 0x7fae4006b500 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fae40071300 tx=0x7fae30009bd0 comp rx=0 tx=0).stop 2023-08-25T03:34:49.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.340+0000 7fae45070700 1 -- 172.21.15.2:0/822537470 shutdown_connections 2023-08-25T03:34:49.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.340+0000 7fae45070700 1 --2- 172.21.15.2:0/822537470 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae4015d360 0x7fae4006b500 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:49.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.340+0000 7fae45070700 1 --2- 172.21.15.2:0/822537470 >> v2:172.21.15.31:3300/0 conn(0x7fae400cfd80 0x7fae4006ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:49.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.340+0000 7fae45070700 1 -- 172.21.15.2:0/822537470 >> 172.21.15.2:0/822537470 conn(0x7fae4009a200 msgr2=0x7fae400d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:49.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.340+0000 7fae45070700 1 -- 172.21.15.2:0/822537470 shutdown_connections 2023-08-25T03:34:49.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.340+0000 7fae45070700 1 -- 172.21.15.2:0/822537470 wait complete. 2023-08-25T03:34:49.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.341+0000 7fae45070700 1 Processor -- start 2023-08-25T03:34:49.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.341+0000 7fae45070700 1 -- start start 2023-08-25T03:34:49.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.342+0000 7fae45070700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae4006bf80 0x7fae4015e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:49.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.342+0000 7fae45070700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fae400cfd80 0x7fae401565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:49.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.342+0000 7fae45070700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae4015d360 0x7fae40156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:49.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.342+0000 7fae45070700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4015fc70 con 0x7fae4015d360 2023-08-25T03:34:49.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.342+0000 7fae45070700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4015f060 con 0x7fae4006bf80 2023-08-25T03:34:49.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.342+0000 7fae45070700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae4015f340 con 0x7fae400cfd80 2023-08-25T03:34:49.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.342+0000 7fae3f59e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae4015d360 0x7fae40156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:49.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.342+0000 7fae3f59e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae4015d360 0x7fae40156ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60736/0 (socket says 172.21.15.2:60736) 2023-08-25T03:34:49.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.342+0000 7fae3f59e700 1 -- 172.21.15.2:0/1032476328 learned_addr learned my addr 172.21.15.2:0/1032476328 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:49.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.343+0000 7fae3ed9d700 1 --2- 172.21.15.2:0/1032476328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae4006bf80 0x7fae4015e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:49.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.343+0000 7fae3e59c700 1 --2- 172.21.15.2:0/1032476328 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fae400cfd80 0x7fae401565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:49.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.343+0000 7fae3f59e700 1 -- 172.21.15.2:0/1032476328 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fae400cfd80 msgr2=0x7fae401565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:49.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.343+0000 7fae3f59e700 1 --2- 172.21.15.2:0/1032476328 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fae400cfd80 0x7fae401565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:49.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.343+0000 7fae3f59e700 1 -- 172.21.15.2:0/1032476328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae4006bf80 msgr2=0x7fae4015e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:49.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.343+0000 7fae3f59e700 1 --2- 172.21.15.2:0/1032476328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae4006bf80 0x7fae4015e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:49.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.343+0000 7fae3f59e700 1 -- 172.21.15.2:0/1032476328 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae30009500 con 0x7fae4015d360 2023-08-25T03:34:49.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.343+0000 7fae3f59e700 1 --2- 172.21.15.2:0/1032476328 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae4015d360 0x7fae40156ae0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fae3800b970 tx=0x7fae3800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:49.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.343+0000 7fae277fe700 1 -- 172.21.15.2:0/1032476328 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3800c9b0 con 0x7fae4015d360 2023-08-25T03:34:49.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.343+0000 7fae277fe700 1 -- 172.21.15.2:0/1032476328 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3800cb10 con 0x7fae4015d360 2023-08-25T03:34:49.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.343+0000 7fae45070700 1 -- 172.21.15.2:0/1032476328 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae4015f620 con 0x7fae4015d360 2023-08-25T03:34:49.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.343+0000 7fae277fe700 1 -- 172.21.15.2:0/1032476328 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae38014440 con 0x7fae4015d360 2023-08-25T03:34:49.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.343+0000 7fae45070700 1 -- 172.21.15.2:0/1032476328 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae40160100 con 0x7fae4015d360 2023-08-25T03:34:49.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.344+0000 7fae45070700 1 -- 172.21.15.2:0/1032476328 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae4015fe20 con 0x7fae4015d360 2023-08-25T03:34:49.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.345+0000 7fae277fe700 1 -- 172.21.15.2:0/1032476328 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fae3801d3b0 con 0x7fae4015d360 2023-08-25T03:34:49.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.345+0000 7fae277fe700 1 --2- 172.21.15.2:0/1032476328 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fae2c06a0e0 0x7fae2c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:49.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.345+0000 7fae277fe700 1 -- 172.21.15.2:0/1032476328 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fae38088270 con 0x7fae4015d360 2023-08-25T03:34:49.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.345+0000 7fae3ed9d700 1 --2- 172.21.15.2:0/1032476328 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fae2c06a0e0 0x7fae2c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:49.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.346+0000 7fae3ed9d700 1 --2- 172.21.15.2:0/1032476328 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fae2c06a0e0 0x7fae2c06c5a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fae40071300 tx=0x7fae30008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:49.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.350+0000 7fae277fe700 1 -- 172.21.15.2:0/1032476328 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fae4015fe20 con 0x7fae4015d360 2023-08-25T03:34:49.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.547+0000 7fae45070700 1 -- 172.21.15.2:0/1032476328 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fae40160320 con 0x7fae4015d360 2023-08-25T03:34:49.549 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.548+0000 7fae277fe700 1 -- 172.21.15.2:0/1032476328 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fae38053890 con 0x7fae4015d360 2023-08-25T03:34:49.549 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-08-25T03:34:49.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.550+0000 7fae45070700 1 -- 172.21.15.2:0/1032476328 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fae2c06a0e0 msgr2=0x7fae2c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:49.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.550+0000 7fae45070700 1 --2- 172.21.15.2:0/1032476328 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fae2c06a0e0 0x7fae2c06c5a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fae40071300 tx=0x7fae30008070 comp rx=0 tx=0).stop 2023-08-25T03:34:49.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.550+0000 7fae45070700 1 -- 172.21.15.2:0/1032476328 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae4015d360 msgr2=0x7fae40156ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:49.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.550+0000 7fae45070700 1 --2- 172.21.15.2:0/1032476328 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae4015d360 0x7fae40156ae0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fae3800b970 tx=0x7fae3800bd30 comp rx=0 tx=0).stop 2023-08-25T03:34:49.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.550+0000 7fae45070700 1 -- 172.21.15.2:0/1032476328 shutdown_connections 2023-08-25T03:34:49.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.550+0000 7fae45070700 1 --2- 172.21.15.2:0/1032476328 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fae2c06a0e0 0x7fae2c06c5a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:49.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.550+0000 7fae45070700 1 --2- 172.21.15.2:0/1032476328 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fae4015d360 0x7fae40156ae0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:49.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.550+0000 7fae45070700 1 --2- 172.21.15.2:0/1032476328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fae4006bf80 0x7fae4015e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:49.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.550+0000 7fae45070700 1 --2- 172.21.15.2:0/1032476328 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fae400cfd80 0x7fae401565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:49.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.550+0000 7fae45070700 1 -- 172.21.15.2:0/1032476328 >> 172.21.15.2:0/1032476328 conn(0x7fae4009a200 msgr2=0x7fae400adef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:49.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.550+0000 7fae45070700 1 -- 172.21.15.2:0/1032476328 shutdown_connections 2023-08-25T03:34:49.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.550+0000 7fae45070700 1 -- 172.21.15.2:0/1032476328 wait complete. 2023-08-25T03:34:49.563 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T03:34:49.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.691+0000 7f9049fcc700 1 Processor -- start 2023-08-25T03:34:49.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.691+0000 7f9049fcc700 1 -- start start 2023-08-25T03:34:49.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.691+0000 7f9049fcc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f90440cdb70 0x7f904406b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:49.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.692+0000 7f9049fcc700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90440ce000 0x7f904406be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:49.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.692+0000 7f9049fcc700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9044161620 con 0x7f90440b1fb0 2023-08-25T03:34:49.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.692+0000 7f9049fcc700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f904406d450 con 0x7f90440cdb70 2023-08-25T03:34:49.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.692+0000 7f9049fcc700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f904406d730 con 0x7f90440ce000 2023-08-25T03:34:49.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.692+0000 7f9043fff700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90440ce000 0x7f904406be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:49.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.692+0000 7f9043fff700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90440ce000 0x7f904406be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.2:60988/0 (socket says 172.21.15.2:60988) 2023-08-25T03:34:49.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.692+0000 7f9043fff700 1 -- 172.21.15.2:0/1401420276 learned_addr learned my addr 172.21.15.2:0/1401420276 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:49.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.692+0000 7f9042ffd700 1 --2- 172.21.15.2:0/1401420276 >> v2:172.21.15.31:3300/0 conn(0x7f90440cdb70 0x7f904406b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:49.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.692+0000 7f9043fff700 1 -- 172.21.15.2:0/1401420276 >> v2:172.21.15.31:3300/0 conn(0x7f90440cdb70 msgr2=0x7f904406b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:49.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.692+0000 7f9043fff700 1 --2- 172.21.15.2:0/1401420276 >> v2:172.21.15.31:3300/0 conn(0x7f90440cdb70 0x7f904406b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:49.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.692+0000 7f9043fff700 1 -- 172.21.15.2:0/1401420276 >> v1:172.21.15.31:6789/0 conn(0x7f90440b1fb0 legacy=0x7f9044160f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:49.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.692+0000 7f9043fff700 1 -- 172.21.15.2:0/1401420276 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904406da10 con 0x7f90440ce000 2023-08-25T03:34:49.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.692+0000 7f9043fff700 1 --2- 172.21.15.2:0/1401420276 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90440ce000 0x7f904406be00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f903c009e50 tx=0x7f903c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa2f0f52c674611a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:49.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.693+0000 7f90427fc700 1 -- 172.21.15.2:0/1401420276 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c00e070 con 0x7f90440ce000 2023-08-25T03:34:49.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.693+0000 7f90427fc700 1 -- 172.21.15.2:0/1401420276 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903c00b910 con 0x7f90440ce000 2023-08-25T03:34:49.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.693+0000 7f90427fc700 1 -- 172.21.15.2:0/1401420276 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c00bcc0 con 0x7f90440ce000 2023-08-25T03:34:49.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.693+0000 7f9049fcc700 1 -- 172.21.15.2:0/1401420276 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90440ce000 msgr2=0x7f904406be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:49.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.693+0000 7f9049fcc700 1 --2- 172.21.15.2:0/1401420276 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90440ce000 0x7f904406be00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f903c009e50 tx=0x7f903c00b330 comp rx=0 tx=0).stop 2023-08-25T03:34:49.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.693+0000 7f9049fcc700 1 -- 172.21.15.2:0/1401420276 shutdown_connections 2023-08-25T03:34:49.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.693+0000 7f9049fcc700 1 --2- 172.21.15.2:0/1401420276 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90440ce000 0x7f904406be00 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:49.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.693+0000 7f9049fcc700 1 --2- 172.21.15.2:0/1401420276 >> v2:172.21.15.31:3300/0 conn(0x7f90440cdb70 0x7f904406b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:49.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.693+0000 7f9049fcc700 1 -- 172.21.15.2:0/1401420276 >> 172.21.15.2:0/1401420276 conn(0x7f90440cfea0 msgr2=0x7f90440d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:49.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.693+0000 7f9049fcc700 1 -- 172.21.15.2:0/1401420276 shutdown_connections 2023-08-25T03:34:49.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.693+0000 7f9049fcc700 1 -- 172.21.15.2:0/1401420276 wait complete. 2023-08-25T03:34:49.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.694+0000 7f9049fcc700 1 Processor -- start 2023-08-25T03:34:49.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.694+0000 7f9049fcc700 1 -- start start 2023-08-25T03:34:49.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.695+0000 7f9049fcc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90440b1fb0 0x7f90441f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:49.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.695+0000 7f9049fcc700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90440cdb70 0x7f90440aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:49.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.695+0000 7f9049fcc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90440ce000 0x7f90440b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:49.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.695+0000 7f9049fcc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90440af070 con 0x7f90440ce000 2023-08-25T03:34:49.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.695+0000 7f9049fcc700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90440af310 con 0x7f90440b1fb0 2023-08-25T03:34:49.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.695+0000 7f9049fcc700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90440af5f0 con 0x7f90440cdb70 2023-08-25T03:34:49.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.695+0000 7f9043fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90440ce000 0x7f90440b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:49.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.695+0000 7f9043fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90440ce000 0x7f90440b0e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60750/0 (socket says 172.21.15.2:60750) 2023-08-25T03:34:49.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.695+0000 7f9043fff700 1 -- 172.21.15.2:0/3482314750 learned_addr learned my addr 172.21.15.2:0/3482314750 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:49.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.695+0000 7f90437fe700 1 --2- 172.21.15.2:0/3482314750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90440b1fb0 0x7f90441f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:49.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.696+0000 7f9042ffd700 1 --2- 172.21.15.2:0/3482314750 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90440cdb70 0x7f90440aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:49.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.696+0000 7f9043fff700 1 -- 172.21.15.2:0/3482314750 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90440cdb70 msgr2=0x7f90440aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:49.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.696+0000 7f9043fff700 1 --2- 172.21.15.2:0/3482314750 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90440cdb70 0x7f90440aeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:49.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.696+0000 7f9043fff700 1 -- 172.21.15.2:0/3482314750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90440b1fb0 msgr2=0x7f90441f6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:49.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.696+0000 7f9043fff700 1 --2- 172.21.15.2:0/3482314750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90440b1fb0 0x7f90441f6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:49.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.696+0000 7f9043fff700 1 -- 172.21.15.2:0/3482314750 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f903c009b30 con 0x7f90440ce000 2023-08-25T03:34:49.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.696+0000 7f9042ffd700 1 --2- 172.21.15.2:0/3482314750 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90440cdb70 0x7f90440aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:49.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.696+0000 7f90437fe700 1 --2- 172.21.15.2:0/3482314750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90440b1fb0 0x7f90441f6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:49.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.696+0000 7f9043fff700 1 --2- 172.21.15.2:0/3482314750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90440ce000 0x7f90440b0e00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f904406e2c0 tx=0x7f903c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:49.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.696+0000 7f902bfff700 1 -- 172.21.15.2:0/3482314750 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c004960 con 0x7f90440ce000 2023-08-25T03:34:49.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.696+0000 7f902bfff700 1 -- 172.21.15.2:0/3482314750 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f903c0099b0 con 0x7f90440ce000 2023-08-25T03:34:49.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.696+0000 7f9049fcc700 1 -- 172.21.15.2:0/3482314750 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90440af8d0 con 0x7f90440ce000 2023-08-25T03:34:49.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.696+0000 7f902bfff700 1 -- 172.21.15.2:0/3482314750 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903c01c650 con 0x7f90440ce000 2023-08-25T03:34:49.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.696+0000 7f9049fcc700 1 -- 172.21.15.2:0/3482314750 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90440afe90 con 0x7f90440ce000 2023-08-25T03:34:49.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.697+0000 7f9049fcc700 1 -- 172.21.15.2:0/3482314750 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9044066f70 con 0x7f90440ce000 2023-08-25T03:34:49.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.698+0000 7f902bfff700 1 -- 172.21.15.2:0/3482314750 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f903c01c7b0 con 0x7f90440ce000 2023-08-25T03:34:49.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.698+0000 7f902bfff700 1 --2- 172.21.15.2:0/3482314750 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f903006a0e0 0x7f903006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:49.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.698+0000 7f902bfff700 1 -- 172.21.15.2:0/3482314750 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f903c08a280 con 0x7f90440ce000 2023-08-25T03:34:49.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.698+0000 7f90437fe700 1 --2- 172.21.15.2:0/3482314750 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f903006a0e0 0x7f903006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:49.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.698+0000 7f90437fe700 1 --2- 172.21.15.2:0/3482314750 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f903006a0e0 0x7f903006c5a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9034000f60 tx=0x7f9034009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:49.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.702+0000 7f902bfff700 1 -- 172.21.15.2:0/3482314750 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f903c0559c0 con 0x7f90440ce000 2023-08-25T03:34:49.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.920+0000 7f9049fcc700 1 -- 172.21.15.2:0/3482314750 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f90440746d0 con 0x7f90440ce000 2023-08-25T03:34:49.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.921+0000 7f902bfff700 1 -- 172.21.15.2:0/3482314750 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f903c058fe0 con 0x7f90440ce000 2023-08-25T03:34:49.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.923+0000 7f9049fcc700 1 -- 172.21.15.2:0/3482314750 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f903006a0e0 msgr2=0x7f903006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:49.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.923+0000 7f9049fcc700 1 --2- 172.21.15.2:0/3482314750 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f903006a0e0 0x7f903006c5a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9034000f60 tx=0x7f9034009040 comp rx=0 tx=0).stop 2023-08-25T03:34:49.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.923+0000 7f9049fcc700 1 -- 172.21.15.2:0/3482314750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90440ce000 msgr2=0x7f90440b0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:49.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.923+0000 7f9049fcc700 1 --2- 172.21.15.2:0/3482314750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90440ce000 0x7f90440b0e00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f904406e2c0 tx=0x7f903c004ad0 comp rx=0 tx=0).stop 2023-08-25T03:34:49.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.923+0000 7f9049fcc700 1 -- 172.21.15.2:0/3482314750 shutdown_connections 2023-08-25T03:34:49.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.923+0000 7f9049fcc700 1 --2- 172.21.15.2:0/3482314750 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f903006a0e0 0x7f903006c5a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:49.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.923+0000 7f9049fcc700 1 --2- 172.21.15.2:0/3482314750 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f90440ce000 0x7f90440b0e00 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:49.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.923+0000 7f9049fcc700 1 --2- 172.21.15.2:0/3482314750 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90440b1fb0 0x7f90441f6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:49.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.923+0000 7f9049fcc700 1 --2- 172.21.15.2:0/3482314750 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90440cdb70 0x7f90440aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:49.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.923+0000 7f9049fcc700 1 -- 172.21.15.2:0/3482314750 >> 172.21.15.2:0/3482314750 conn(0x7f90440cfea0 msgr2=0x7f904406a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:49.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.923+0000 7f9049fcc700 1 -- 172.21.15.2:0/3482314750 shutdown_connections 2023-08-25T03:34:49.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:49.923+0000 7f9049fcc700 1 -- 172.21.15.2:0/3482314750 wait complete. 2023-08-25T03:34:49.925 INFO:teuthology.orchestra.run.smithi002.stderr:listed 0 entries 2023-08-25T03:34:49.936 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-08-25T03:34:49.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:49.937 DEBUG:teuthology.orchestra.run.smithi002:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-08-25T03:34:49.983 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi002.front.sepia.ceph.com 2023-08-25T03:34:49.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:49.985 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-08-25T03:34:50.002 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:34:50.003 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:34:50.024 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:34:50.025 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:34:50.086 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:34:50.086 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:34:50.146 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:34:50.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.272+0000 7f28f3a55700 1 Processor -- start 2023-08-25T03:34:50.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.272+0000 7f28f3a55700 1 -- start start 2023-08-25T03:34:50.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.273+0000 7f28f3a55700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28ec15a230 0x7f28ec0c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:50.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.273+0000 7f28f3a55700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f28ec0ca130 0x7f28ec06c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:50.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.273+0000 7f28f3a55700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f28ec06c6a0 0x7f28ec0ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:50.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.273+0000 7f28f3a55700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28ec06d280 con 0x7f28ec06c6a0 2023-08-25T03:34:50.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.273+0000 7f28f3a55700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28ec06cac0 con 0x7f28ec0ca130 2023-08-25T03:34:50.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.273+0000 7f28f3a55700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec0711d0 con 0x7f28ec15a230 2023-08-25T03:34:50.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.273+0000 7f28f1ff2700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f28ec06c6a0 0x7f28ec0ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:50.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.273+0000 7f28f1ff2700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f28ec06c6a0 0x7f28ec0ccdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60758/0 (socket says 172.21.15.2:60758) 2023-08-25T03:34:50.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.273+0000 7f28f1ff2700 1 -- 172.21.15.2:0/4184871856 learned_addr learned my addr 172.21.15.2:0/4184871856 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:50.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.273+0000 7f28f17f1700 1 --2- 172.21.15.2:0/4184871856 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28ec15a230 0x7f28ec0c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:50.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.273+0000 7f28f0ff0700 1 --2- 172.21.15.2:0/4184871856 >> v2:172.21.15.31:3300/0 conn(0x7f28ec0ca130 0x7f28ec06c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:50.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.273+0000 7f28f1ff2700 1 -- 172.21.15.2:0/4184871856 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28ec15a230 msgr2=0x7f28ec0c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:50.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.273+0000 7f28f1ff2700 1 --2- 172.21.15.2:0/4184871856 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28ec15a230 0x7f28ec0c9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:50.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.273+0000 7f28f1ff2700 1 -- 172.21.15.2:0/4184871856 >> v2:172.21.15.31:3300/0 conn(0x7f28ec0ca130 msgr2=0x7f28ec06c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:50.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.273+0000 7f28f1ff2700 1 --2- 172.21.15.2:0/4184871856 >> v2:172.21.15.31:3300/0 conn(0x7f28ec0ca130 0x7f28ec06c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:50.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.273+0000 7f28f1ff2700 1 -- 172.21.15.2:0/4184871856 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ec0714b0 con 0x7f28ec06c6a0 2023-08-25T03:34:50.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.273+0000 7f28f0ff0700 1 --2- 172.21.15.2:0/4184871856 >> v2:172.21.15.31:3300/0 conn(0x7f28ec0ca130 0x7f28ec06c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:50.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.273+0000 7f28f1ff2700 1 --2- 172.21.15.2:0/4184871856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28ec06c6a0 0x7f28ec0ccdb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0b1d30 tx=0x7f28e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0ec1760946afe9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:50.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.273+0000 7f28e3fff700 1 -- 172.21.15.2:0/4184871856 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e400e070 con 0x7f28ec06c6a0 2023-08-25T03:34:50.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.273+0000 7f28e3fff700 1 -- 172.21.15.2:0/4184871856 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e400b9c0 con 0x7f28ec06c6a0 2023-08-25T03:34:50.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.273+0000 7f28e3fff700 1 -- 172.21.15.2:0/4184871856 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e400bd70 con 0x7f28ec06c6a0 2023-08-25T03:34:50.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.274+0000 7f28f3a55700 1 -- 172.21.15.2:0/4184871856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28ec06c6a0 msgr2=0x7f28ec0ccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:50.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.274+0000 7f28f3a55700 1 --2- 172.21.15.2:0/4184871856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28ec06c6a0 0x7f28ec0ccdb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0b1d30 tx=0x7f28e400b330 comp rx=0 tx=0).stop 2023-08-25T03:34:50.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.274+0000 7f28f3a55700 1 -- 172.21.15.2:0/4184871856 shutdown_connections 2023-08-25T03:34:50.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.274+0000 7f28f3a55700 1 --2- 172.21.15.2:0/4184871856 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28ec06c6a0 0x7f28ec0ccdb0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:50.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.274+0000 7f28f3a55700 1 --2- 172.21.15.2:0/4184871856 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28ec15a230 0x7f28ec0c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:50.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.274+0000 7f28f3a55700 1 --2- 172.21.15.2:0/4184871856 >> v2:172.21.15.31:3300/0 conn(0x7f28ec0ca130 0x7f28ec06c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:50.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.274+0000 7f28f3a55700 1 -- 172.21.15.2:0/4184871856 >> 172.21.15.2:0/4184871856 conn(0x7f28ec0cf1e0 msgr2=0x7f28ec0cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:50.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.274+0000 7f28f3a55700 1 -- 172.21.15.2:0/4184871856 shutdown_connections 2023-08-25T03:34:50.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.274+0000 7f28f3a55700 1 -- 172.21.15.2:0/4184871856 wait complete. 2023-08-25T03:34:50.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.276+0000 7f28f3a55700 1 Processor -- start 2023-08-25T03:34:50.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.276+0000 7f28f3a55700 1 -- start start 2023-08-25T03:34:50.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.277+0000 7f28f3a55700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28ec06c6a0 0x7f28ec158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:50.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.277+0000 7f28f3a55700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28ec0ca130 0x7f28ec06d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:50.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.277+0000 7f28f3a55700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28ec15a230 0x7f28ec06de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:50.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.277+0000 7f28f3a55700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec159180 con 0x7f28ec15a230 2023-08-25T03:34:50.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.277+0000 7f28f3a55700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec15d3f0 con 0x7f28ec0ca130 2023-08-25T03:34:50.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.277+0000 7f28f3a55700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec15c870 con 0x7f28ec06c6a0 2023-08-25T03:34:50.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.277+0000 7f28f1ff2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28ec15a230 0x7f28ec06de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:50.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.277+0000 7f28f1ff2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28ec15a230 0x7f28ec06de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60764/0 (socket says 172.21.15.2:60764) 2023-08-25T03:34:50.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.277+0000 7f28f1ff2700 1 -- 172.21.15.2:0/2309372589 learned_addr learned my addr 172.21.15.2:0/2309372589 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:50.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.277+0000 7f28f17f1700 1 --2- 172.21.15.2:0/2309372589 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28ec06c6a0 0x7f28ec158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:50.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.277+0000 7f28f0ff0700 1 --2- 172.21.15.2:0/2309372589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28ec0ca130 0x7f28ec06d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:50.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.277+0000 7f28f1ff2700 1 -- 172.21.15.2:0/2309372589 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28ec06c6a0 msgr2=0x7f28ec158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:50.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.277+0000 7f28f1ff2700 1 --2- 172.21.15.2:0/2309372589 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28ec06c6a0 0x7f28ec158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:50.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.277+0000 7f28f1ff2700 1 -- 172.21.15.2:0/2309372589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28ec0ca130 msgr2=0x7f28ec06d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:50.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.277+0000 7f28f1ff2700 1 --2- 172.21.15.2:0/2309372589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28ec0ca130 0x7f28ec06d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:50.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.277+0000 7f28f1ff2700 1 -- 172.21.15.2:0/2309372589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28e4009b30 con 0x7f28ec15a230 2023-08-25T03:34:50.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.277+0000 7f28f0ff0700 1 --2- 172.21.15.2:0/2309372589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28ec0ca130 0x7f28ec06d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:50.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.277+0000 7f28f1ff2700 1 --2- 172.21.15.2:0/2309372589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28ec15a230 0x7f28ec06de00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f28ec067340 tx=0x7f28e4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:50.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.278+0000 7f28e1ffb700 1 -- 172.21.15.2:0/2309372589 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e40099b0 con 0x7f28ec15a230 2023-08-25T03:34:50.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.278+0000 7f28e1ffb700 1 -- 172.21.15.2:0/2309372589 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28e4004250 con 0x7f28ec15a230 2023-08-25T03:34:50.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.278+0000 7f28f3a55700 1 -- 172.21.15.2:0/2309372589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28ec15cb50 con 0x7f28ec15a230 2023-08-25T03:34:50.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.278+0000 7f28e1ffb700 1 -- 172.21.15.2:0/2309372589 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28e4004600 con 0x7f28ec15a230 2023-08-25T03:34:50.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.278+0000 7f28f3a55700 1 -- 172.21.15.2:0/2309372589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28ec15d110 con 0x7f28ec15a230 2023-08-25T03:34:50.290 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.279+0000 7f28f3a55700 1 -- 172.21.15.2:0/2309372589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28ec1f0850 con 0x7f28ec15a230 2023-08-25T03:34:50.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.279+0000 7f28e1ffb700 1 -- 172.21.15.2:0/2309372589 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f28e4004760 con 0x7f28ec15a230 2023-08-25T03:34:50.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.280+0000 7f28e1ffb700 1 --2- 172.21.15.2:0/2309372589 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f28d006a0e0 0x7f28d006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:50.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.280+0000 7f28f17f1700 1 --2- 172.21.15.2:0/2309372589 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f28d006a0e0 0x7f28d006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:50.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.280+0000 7f28e1ffb700 1 -- 172.21.15.2:0/2309372589 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f28e40899d0 con 0x7f28ec15a230 2023-08-25T03:34:50.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.280+0000 7f28f17f1700 1 --2- 172.21.15.2:0/2309372589 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f28d006a0e0 0x7f28d006c5a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f28e8006b20 tx=0x7f28e8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:50.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.284+0000 7f28e1ffb700 1 -- 172.21.15.2:0/2309372589 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f28ec1f0850 con 0x7f28ec15a230 2023-08-25T03:34:50.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.485+0000 7f28f3a55700 1 -- 172.21.15.2:0/2309372589 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28ec1f0850 con 0x7f28ec15a230 2023-08-25T03:34:50.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.486+0000 7f28e1ffb700 1 -- 172.21.15.2:0/2309372589 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18230 (secure 0 0 0) 0x7f28ec1f0850 con 0x7f28ec15a230 2023-08-25T03:34:50.487 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:50.487 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:34:44.588865+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:34:00.869118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T03:34:01.987016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.f","mappings":[{"from":1,"to":5}]},{"pgid":"4.2b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:34:50.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.488+0000 7f28f3a55700 1 -- 172.21.15.2:0/2309372589 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f28d006a0e0 msgr2=0x7f28d006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:50.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.488+0000 7f28f3a55700 1 --2- 172.21.15.2:0/2309372589 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f28d006a0e0 0x7f28d006c5a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f28e8006b20 tx=0x7f28e8008040 comp rx=0 tx=0).stop 2023-08-25T03:34:50.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.488+0000 7f28f3a55700 1 -- 172.21.15.2:0/2309372589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28ec15a230 msgr2=0x7f28ec06de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:50.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.488+0000 7f28f3a55700 1 --2- 172.21.15.2:0/2309372589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28ec15a230 0x7f28ec06de00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f28ec067340 tx=0x7f28e4004ad0 comp rx=0 tx=0).stop 2023-08-25T03:34:50.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.488+0000 7f28f3a55700 1 -- 172.21.15.2:0/2309372589 shutdown_connections 2023-08-25T03:34:50.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.488+0000 7f28f3a55700 1 --2- 172.21.15.2:0/2309372589 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f28d006a0e0 0x7f28d006c5a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:50.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.488+0000 7f28f3a55700 1 --2- 172.21.15.2:0/2309372589 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f28ec15a230 0x7f28ec06de00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:50.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.488+0000 7f28f3a55700 1 --2- 172.21.15.2:0/2309372589 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28ec06c6a0 0x7f28ec158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:50.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.488+0000 7f28f3a55700 1 --2- 172.21.15.2:0/2309372589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28ec0ca130 0x7f28ec06d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:50.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.488+0000 7f28f3a55700 1 -- 172.21.15.2:0/2309372589 >> 172.21.15.2:0/2309372589 conn(0x7f28ec0cf1e0 msgr2=0x7f28ec0afb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:50.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.488+0000 7f28f3a55700 1 -- 172.21.15.2:0/2309372589 shutdown_connections 2023-08-25T03:34:50.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.488+0000 7f28f3a55700 1 -- 172.21.15.2:0/2309372589 wait complete. 2023-08-25T03:34:50.501 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T03:33:49.560643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T03:33:56.265184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T03:34:00.869118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T03:34:01.987016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T03:34:50.502 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T03:34:50.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.631+0000 7f17fbe67700 1 Processor -- start 2023-08-25T03:34:50.632 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.631+0000 7f17fbe67700 1 -- start start 2023-08-25T03:34:50.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.632+0000 7f17fbe67700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f17f40cfd80 0x7f17f406bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:50.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.632+0000 7f17fbe67700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f17f406e570 0x7f17f406e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:50.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.632+0000 7f17fbe67700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17f40d1b40 con 0x7f17f415d360 2023-08-25T03:34:50.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.632+0000 7f17fbe67700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17f406e0e0 con 0x7f17f406e570 2023-08-25T03:34:50.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.632+0000 7f17fbe67700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f40d0620 con 0x7f17f40cfd80 2023-08-25T03:34:50.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.632+0000 7f17fa404700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f17f406e570 0x7f17f406e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:50.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.632+0000 7f17f9402700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f17f40cfd80 0x7f17f406bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:50.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.632+0000 7f17fa404700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f17f406e570 0x7f17f406e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60772/0 (socket says 172.21.15.2:60772) 2023-08-25T03:34:50.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.632+0000 7f17fa404700 1 -- 172.21.15.2:0/2976601927 learned_addr learned my addr 172.21.15.2:0/2976601927 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:50.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.632+0000 7f17fa404700 1 -- 172.21.15.2:0/2976601927 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f17f40cfd80 msgr2=0x7f17f406bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:50.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.632+0000 7f17fa404700 1 --2- 172.21.15.2:0/2976601927 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f17f40cfd80 0x7f17f406bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:50.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.632+0000 7f17fa404700 1 -- 172.21.15.2:0/2976601927 >> v1:172.21.15.31:6789/0 conn(0x7f17f415d360 legacy=0x7f17f406b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:50.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.632+0000 7f17fa404700 1 -- 172.21.15.2:0/2976601927 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f40d0340 con 0x7f17f406e570 2023-08-25T03:34:50.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.633+0000 7f17fa404700 1 --2- 172.21.15.2:0/2976601927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17f406e570 0x7f17f406e9d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009e50 tx=0x7f17ec00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a2d770a3512abc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:50.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.633+0000 7f17f8c01700 1 -- 172.21.15.2:0/2976601927 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec00e070 con 0x7f17f406e570 2023-08-25T03:34:50.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.633+0000 7f17f8c01700 1 -- 172.21.15.2:0/2976601927 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17ec00b910 con 0x7f17f406e570 2023-08-25T03:34:50.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.633+0000 7f17f8c01700 1 -- 172.21.15.2:0/2976601927 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec00bcc0 con 0x7f17f406e570 2023-08-25T03:34:50.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.634+0000 7f17fbe67700 1 -- 172.21.15.2:0/2976601927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17f406e570 msgr2=0x7f17f406e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:50.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.634+0000 7f17fbe67700 1 --2- 172.21.15.2:0/2976601927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17f406e570 0x7f17f406e9d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f17ec009e50 tx=0x7f17ec00b330 comp rx=0 tx=0).stop 2023-08-25T03:34:50.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.634+0000 7f17fbe67700 1 -- 172.21.15.2:0/2976601927 shutdown_connections 2023-08-25T03:34:50.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.634+0000 7f17fbe67700 1 --2- 172.21.15.2:0/2976601927 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17f406e570 0x7f17f406e9d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:50.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.634+0000 7f17fbe67700 1 --2- 172.21.15.2:0/2976601927 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f17f40cfd80 0x7f17f406bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:50.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.634+0000 7f17fbe67700 1 -- 172.21.15.2:0/2976601927 >> 172.21.15.2:0/2976601927 conn(0x7f17f409a200 msgr2=0x7f17f40d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:50.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.634+0000 7f17fbe67700 1 -- 172.21.15.2:0/2976601927 shutdown_connections 2023-08-25T03:34:50.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.634+0000 7f17fbe67700 1 -- 172.21.15.2:0/2976601927 wait complete. 2023-08-25T03:34:50.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.635+0000 7f17fbe67700 1 Processor -- start 2023-08-25T03:34:50.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.635+0000 7f17fbe67700 1 -- start start 2023-08-25T03:34:50.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.636+0000 7f17fbe67700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17f406e570 0x7f17f40b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:50.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.636+0000 7f17fbe67700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f17f40cfd80 0x7f17f41f6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:50.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.636+0000 7f17fbe67700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17f415d360 0x7f17f4148df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:50.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.636+0000 7f17fbe67700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f40b1090 con 0x7f17f406e570 2023-08-25T03:34:50.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.636+0000 7f17fbe67700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f41524e0 con 0x7f17f415d360 2023-08-25T03:34:50.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.636+0000 7f17fbe67700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f40b3570 con 0x7f17f40cfd80 2023-08-25T03:34:50.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.636+0000 7f17f9c03700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17f406e570 0x7f17f40b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:50.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.636+0000 7f17f9c03700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17f406e570 0x7f17f40b0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60774/0 (socket says 172.21.15.2:60774) 2023-08-25T03:34:50.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.636+0000 7f17f9c03700 1 -- 172.21.15.2:0/1638770424 learned_addr learned my addr 172.21.15.2:0/1638770424 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:50.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.636+0000 7f17f9402700 1 --2- 172.21.15.2:0/1638770424 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f17f40cfd80 0x7f17f41f6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:50.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.636+0000 7f17fa404700 1 --2- 172.21.15.2:0/1638770424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17f415d360 0x7f17f4148df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:50.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.636+0000 7f17f9c03700 1 -- 172.21.15.2:0/1638770424 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f17f40cfd80 msgr2=0x7f17f41f6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:50.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.636+0000 7f17f9c03700 1 --2- 172.21.15.2:0/1638770424 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f17f40cfd80 0x7f17f41f6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:50.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.636+0000 7f17f9c03700 1 -- 172.21.15.2:0/1638770424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17f415d360 msgr2=0x7f17f4148df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:50.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.636+0000 7f17f9c03700 1 --2- 172.21.15.2:0/1638770424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17f415d360 0x7f17f4148df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:50.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.636+0000 7f17f9c03700 1 -- 172.21.15.2:0/1638770424 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ec009b30 con 0x7f17f406e570 2023-08-25T03:34:50.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.636+0000 7f17fa404700 1 --2- 172.21.15.2:0/1638770424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17f415d360 0x7f17f4148df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:50.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.636+0000 7f17f9c03700 1 --2- 172.21.15.2:0/1638770424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17f406e570 0x7f17f40b0b50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f17f000c4d0 tx=0x7f17f000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:50.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.637+0000 7f17ea7fc700 1 -- 172.21.15.2:0/1638770424 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f00052d0 con 0x7f17f406e570 2023-08-25T03:34:50.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.637+0000 7f17ea7fc700 1 -- 172.21.15.2:0/1638770424 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f0005430 con 0x7f17f406e570 2023-08-25T03:34:50.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.637+0000 7f17fbe67700 1 -- 172.21.15.2:0/1638770424 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17f4152f80 con 0x7f17f406e570 2023-08-25T03:34:50.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.637+0000 7f17ea7fc700 1 -- 172.21.15.2:0/1638770424 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f00038f0 con 0x7f17f406e570 2023-08-25T03:34:50.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.637+0000 7f17fbe67700 1 -- 172.21.15.2:0/1638770424 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f4153540 con 0x7f17f406e570 2023-08-25T03:34:50.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.638+0000 7f17fbe67700 1 -- 172.21.15.2:0/1638770424 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17f40b3850 con 0x7f17f406e570 2023-08-25T03:34:50.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.638+0000 7f17ea7fc700 1 -- 172.21.15.2:0/1638770424 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f17f0003a50 con 0x7f17f406e570 2023-08-25T03:34:50.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.639+0000 7f17ea7fc700 1 --2- 172.21.15.2:0/1638770424 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f17d806a0e0 0x7f17d806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:50.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.639+0000 7f17ea7fc700 1 -- 172.21.15.2:0/1638770424 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f17f0087a30 con 0x7f17f406e570 2023-08-25T03:34:50.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.639+0000 7f17f9402700 1 --2- 172.21.15.2:0/1638770424 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f17d806a0e0 0x7f17d806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:50.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.639+0000 7f17f9402700 1 --2- 172.21.15.2:0/1638770424 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f17d806a0e0 0x7f17d806c5a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f17f40d19a0 tx=0x7f17e4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:50.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.643+0000 7f17ea7fc700 1 -- 172.21.15.2:0/1638770424 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f17f40b3850 con 0x7f17f406e570 2023-08-25T03:34:50.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.841+0000 7f17fbe67700 1 -- 172.21.15.2:0/1638770424 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f17f40b3850 con 0x7f17f406e570 2023-08-25T03:34:50.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.841+0000 7f17ea7fc700 1 -- 172.21.15.2:0/1638770424 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f17f0017020 con 0x7f17f406e570 2023-08-25T03:34:50.842 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-08-25T03:34:50.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.843+0000 7f17fbe67700 1 -- 172.21.15.2:0/1638770424 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f17d806a0e0 msgr2=0x7f17d806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:50.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.843+0000 7f17fbe67700 1 --2- 172.21.15.2:0/1638770424 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f17d806a0e0 0x7f17d806c5a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f17f40d19a0 tx=0x7f17e4008040 comp rx=0 tx=0).stop 2023-08-25T03:34:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.843+0000 7f17fbe67700 1 -- 172.21.15.2:0/1638770424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17f406e570 msgr2=0x7f17f40b0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.843+0000 7f17fbe67700 1 --2- 172.21.15.2:0/1638770424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17f406e570 0x7f17f40b0b50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f17f000c4d0 tx=0x7f17f000c890 comp rx=0 tx=0).stop 2023-08-25T03:34:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.843+0000 7f17fbe67700 1 -- 172.21.15.2:0/1638770424 shutdown_connections 2023-08-25T03:34:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.843+0000 7f17fbe67700 1 --2- 172.21.15.2:0/1638770424 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f17d806a0e0 0x7f17d806c5a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.843+0000 7f17fbe67700 1 --2- 172.21.15.2:0/1638770424 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f17f406e570 0x7f17f40b0b50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:50.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.843+0000 7f17fbe67700 1 --2- 172.21.15.2:0/1638770424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f17f415d360 0x7f17f4148df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.843+0000 7f17fbe67700 1 --2- 172.21.15.2:0/1638770424 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f17f40cfd80 0x7f17f41f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.843+0000 7f17fbe67700 1 -- 172.21.15.2:0/1638770424 >> 172.21.15.2:0/1638770424 conn(0x7f17f409a200 msgr2=0x7f17f40afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.844+0000 7f17fbe67700 1 -- 172.21.15.2:0/1638770424 shutdown_connections 2023-08-25T03:34:50.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.844+0000 7f17fbe67700 1 -- 172.21.15.2:0/1638770424 wait complete. 2023-08-25T03:34:50.857 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T03:34:50.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.985+0000 7f89e2f2d700 1 Processor -- start 2023-08-25T03:34:50.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.985+0000 7f89e2f2d700 1 -- start start 2023-08-25T03:34:50.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.986+0000 7f89e2f2d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f89dc1607f0 0x7f89dc070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:50.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.986+0000 7f89e2f2d700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f89dc06a1e0 0x7f89dc06a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:50.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.986+0000 7f89e2f2d700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89dc160bd0 con 0x7f89dc0af180 2023-08-25T03:34:50.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.986+0000 7f89e2f2d700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89dc06abd0 con 0x7f89dc06a1e0 2023-08-25T03:34:50.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.986+0000 7f89e2f2d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89dc0713a0 con 0x7f89dc1607f0 2023-08-25T03:34:50.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.986+0000 7f89e14ca700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f89dc06a1e0 0x7f89dc06a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:50.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.986+0000 7f89dbfff700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f89dc0af180 0x7f89dc0ac8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47448/0 (socket says 172.21.15.2:47448) 2023-08-25T03:34:50.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.986+0000 7f89dbfff700 1 -- 172.21.15.2:0/3597653276 learned_addr learned my addr 172.21.15.2:0/3597653276 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:50.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.987+0000 7f89e0cc9700 1 --2- 172.21.15.2:0/3597653276 >> v2:172.21.15.31:3300/0 conn(0x7f89dc1607f0 0x7f89dc070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:50.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.987+0000 7f89e14ca700 1 -- 172.21.15.2:0/3597653276 >> v2:172.21.15.31:3300/0 conn(0x7f89dc1607f0 msgr2=0x7f89dc070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:50.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.987+0000 7f89e14ca700 1 --2- 172.21.15.2:0/3597653276 >> v2:172.21.15.31:3300/0 conn(0x7f89dc1607f0 0x7f89dc070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:50.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.987+0000 7f89e14ca700 1 -- 172.21.15.2:0/3597653276 >> v1:172.21.15.2:6789/0 conn(0x7f89dc0af180 legacy=0x7f89dc0ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:50.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.987+0000 7f89db7fe700 1 -- 172.21.15.2:0/3597653276 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4266109837 0 0) 0x7f89dc160bd0 con 0x7f89dc0af180 2023-08-25T03:34:50.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.987+0000 7f89e14ca700 1 -- 172.21.15.2:0/3597653276 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89dc0cdd00 con 0x7f89dc06a1e0 2023-08-25T03:34:50.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.987+0000 7f89e0cc9700 1 --2- 172.21.15.2:0/3597653276 >> v2:172.21.15.31:3300/0 conn(0x7f89dc1607f0 0x7f89dc070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:50.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.987+0000 7f89e14ca700 1 --2- 172.21.15.2:0/3597653276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89dc06a1e0 0x7f89dc06a690 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f89dc06df30 tx=0x7f89d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f1a7a6aca875fd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:50.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.987+0000 7f89db7fe700 1 -- 172.21.15.2:0/3597653276 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d400e070 con 0x7f89dc06a1e0 2023-08-25T03:34:50.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.987+0000 7f89db7fe700 1 -- 172.21.15.2:0/3597653276 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d400bbd0 con 0x7f89dc06a1e0 2023-08-25T03:34:50.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.987+0000 7f89db7fe700 1 -- 172.21.15.2:0/3597653276 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4012430 con 0x7f89dc06a1e0 2023-08-25T03:34:50.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.988+0000 7f89e2f2d700 1 -- 172.21.15.2:0/3597653276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89dc06a1e0 msgr2=0x7f89dc06a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:50.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.988+0000 7f89e2f2d700 1 --2- 172.21.15.2:0/3597653276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89dc06a1e0 0x7f89dc06a690 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f89dc06df30 tx=0x7f89d400b330 comp rx=0 tx=0).stop 2023-08-25T03:34:50.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.988+0000 7f89e2f2d700 1 -- 172.21.15.2:0/3597653276 shutdown_connections 2023-08-25T03:34:50.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.988+0000 7f89e2f2d700 1 --2- 172.21.15.2:0/3597653276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89dc06a1e0 0x7f89dc06a690 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:50.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.988+0000 7f89e2f2d700 1 --2- 172.21.15.2:0/3597653276 >> v2:172.21.15.31:3300/0 conn(0x7f89dc1607f0 0x7f89dc070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:50.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.988+0000 7f89e2f2d700 1 -- 172.21.15.2:0/3597653276 >> 172.21.15.2:0/3597653276 conn(0x7f89dc0cfd90 msgr2=0x7f89dc0d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:50.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.988+0000 7f89e2f2d700 1 -- 172.21.15.2:0/3597653276 shutdown_connections 2023-08-25T03:34:50.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.988+0000 7f89e2f2d700 1 -- 172.21.15.2:0/3597653276 wait complete. 2023-08-25T03:34:50.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.989+0000 7f89e2f2d700 1 Processor -- start 2023-08-25T03:34:50.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.989+0000 7f89e2f2d700 1 -- start start 2023-08-25T03:34:50.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.990+0000 7f89e2f2d700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f89dc06a1e0 0x7f89dc0c66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:50.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.990+0000 7f89e2f2d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89dc0af180 0x7f89dc0c7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:50.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.990+0000 7f89e2f2d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89dc1607f0 0x7f89dc0c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:50.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.990+0000 7f89e2f2d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc0c6c20 con 0x7f89dc1607f0 2023-08-25T03:34:50.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.990+0000 7f89e2f2d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc0c78c0 con 0x7f89dc0af180 2023-08-25T03:34:50.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.990+0000 7f89e2f2d700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89dc0c7ba0 con 0x7f89dc06a1e0 2023-08-25T03:34:50.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.990+0000 7f89e14ca700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89dc1607f0 0x7f89dc0c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:50.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.990+0000 7f89e14ca700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89dc1607f0 0x7f89dc0c9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60790/0 (socket says 172.21.15.2:60790) 2023-08-25T03:34:50.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.990+0000 7f89e14ca700 1 -- 172.21.15.2:0/3057838413 learned_addr learned my addr 172.21.15.2:0/3057838413 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:50.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.990+0000 7f89dbfff700 1 --2- 172.21.15.2:0/3057838413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89dc0af180 0x7f89dc0c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:50.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.990+0000 7f89e0cc9700 1 --2- 172.21.15.2:0/3057838413 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f89dc06a1e0 0x7f89dc0c66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:50.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.990+0000 7f89e14ca700 1 -- 172.21.15.2:0/3057838413 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f89dc06a1e0 msgr2=0x7f89dc0c66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:50.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.990+0000 7f89e14ca700 1 --2- 172.21.15.2:0/3057838413 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f89dc06a1e0 0x7f89dc0c66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:50.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.990+0000 7f89e14ca700 1 -- 172.21.15.2:0/3057838413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89dc0af180 msgr2=0x7f89dc0c7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:50.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.990+0000 7f89e14ca700 1 --2- 172.21.15.2:0/3057838413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89dc0af180 0x7f89dc0c7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:50.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.990+0000 7f89e14ca700 1 -- 172.21.15.2:0/3057838413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89d4009a70 con 0x7f89dc1607f0 2023-08-25T03:34:50.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.991+0000 7f89e14ca700 1 --2- 172.21.15.2:0/3057838413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89dc1607f0 0x7f89dc0c9440 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f89dc072990 tx=0x7f89d4004b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:50.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.991+0000 7f89d97fa700 1 -- 172.21.15.2:0/3057838413 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4012430 con 0x7f89dc1607f0 2023-08-25T03:34:51.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.991+0000 7f89d97fa700 1 -- 172.21.15.2:0/3057838413 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d4020070 con 0x7f89dc1607f0 2023-08-25T03:34:51.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.991+0000 7f89d97fa700 1 -- 172.21.15.2:0/3057838413 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d4004280 con 0x7f89dc1607f0 2023-08-25T03:34:51.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.991+0000 7f89e2f2d700 1 -- 172.21.15.2:0/3057838413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89dc0c7e80 con 0x7f89dc1607f0 2023-08-25T03:34:51.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.991+0000 7f89e2f2d700 1 -- 172.21.15.2:0/3057838413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89dc0c8440 con 0x7f89dc1607f0 2023-08-25T03:34:51.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.992+0000 7f89e2f2d700 1 -- 172.21.15.2:0/3057838413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89dc0746d0 con 0x7f89dc1607f0 2023-08-25T03:34:51.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.992+0000 7f89d97fa700 1 -- 172.21.15.2:0/3057838413 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f89d40043e0 con 0x7f89dc1607f0 2023-08-25T03:34:51.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.993+0000 7f89d97fa700 1 --2- 172.21.15.2:0/3057838413 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f89c006a0e0 0x7f89c006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:51.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.993+0000 7f89d97fa700 1 -- 172.21.15.2:0/3057838413 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f89d4022070 con 0x7f89dc1607f0 2023-08-25T03:34:51.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.993+0000 7f89e0cc9700 1 --2- 172.21.15.2:0/3057838413 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f89c006a0e0 0x7f89c006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:51.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.993+0000 7f89e0cc9700 1 --2- 172.21.15.2:0/3057838413 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f89c006a0e0 0x7f89c006c5a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f89d0006fd0 tx=0x7f89d0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:51.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:50.997+0000 7f89d97fa700 1 -- 172.21.15.2:0/3057838413 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f89d4055a30 con 0x7f89dc1607f0 2023-08-25T03:34:51.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.195+0000 7f89e2f2d700 1 -- 172.21.15.2:0/3057838413 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f89dc0ae800 con 0x7f89dc1607f0 2023-08-25T03:34:51.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.196+0000 7f89d97fa700 1 -- 172.21.15.2:0/3057838413 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f89d4059050 con 0x7f89dc1607f0 2023-08-25T03:34:51.197 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 8 2023-08-25T03:34:51.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.198+0000 7f89e2f2d700 1 -- 172.21.15.2:0/3057838413 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f89c006a0e0 msgr2=0x7f89c006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:51.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.198+0000 7f89e2f2d700 1 --2- 172.21.15.2:0/3057838413 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f89c006a0e0 0x7f89c006c5a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f89d0006fd0 tx=0x7f89d0008040 comp rx=0 tx=0).stop 2023-08-25T03:34:51.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.198+0000 7f89e2f2d700 1 -- 172.21.15.2:0/3057838413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89dc1607f0 msgr2=0x7f89dc0c9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:51.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.198+0000 7f89e2f2d700 1 --2- 172.21.15.2:0/3057838413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89dc1607f0 0x7f89dc0c9440 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f89dc072990 tx=0x7f89d4004b20 comp rx=0 tx=0).stop 2023-08-25T03:34:51.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.198+0000 7f89e2f2d700 1 -- 172.21.15.2:0/3057838413 shutdown_connections 2023-08-25T03:34:51.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.198+0000 7f89e2f2d700 1 --2- 172.21.15.2:0/3057838413 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f89c006a0e0 0x7f89c006c5a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:51.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.198+0000 7f89e2f2d700 1 --2- 172.21.15.2:0/3057838413 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f89dc1607f0 0x7f89dc0c9440 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:51.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.198+0000 7f89e2f2d700 1 --2- 172.21.15.2:0/3057838413 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f89dc06a1e0 0x7f89dc0c66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:51.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.198+0000 7f89e2f2d700 1 --2- 172.21.15.2:0/3057838413 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89dc0af180 0x7f89dc0c7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:51.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.198+0000 7f89e2f2d700 1 -- 172.21.15.2:0/3057838413 >> 172.21.15.2:0/3057838413 conn(0x7f89dc0cfd90 msgr2=0x7f89dc0d0170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:51.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.198+0000 7f89e2f2d700 1 -- 172.21.15.2:0/3057838413 shutdown_connections 2023-08-25T03:34:51.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.198+0000 7f89e2f2d700 1 -- 172.21.15.2:0/3057838413 wait complete. 2023-08-25T03:34:51.211 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T03:34:51.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.339+0000 7f4d635ac700 1 Processor -- start 2023-08-25T03:34:51.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.339+0000 7f4d635ac700 1 -- start start 2023-08-25T03:34:51.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.340+0000 7f4d635ac700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f4d5c15d360 0x7f4d5c06b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:51.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.340+0000 7f4d635ac700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f4d5c0cfd80 0x7f4d5c06ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:51.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.340+0000 7f4d635ac700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d5c0d1b40 con 0x7f4d5c06bf80 2023-08-25T03:34:51.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.340+0000 7f4d635ac700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d5c06e0e0 con 0x7f4d5c0cfd80 2023-08-25T03:34:51.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.340+0000 7f4d635ac700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d5c0d0620 con 0x7f4d5c15d360 2023-08-25T03:34:51.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.340+0000 7f4d60b47700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f4d5c0cfd80 0x7f4d5c06ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:51.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.340+0000 7f4d61b49700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f4d5c06bf80 0x7f4d5c160ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47464/0 (socket says 172.21.15.2:47464) 2023-08-25T03:34:51.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.340+0000 7f4d61b49700 1 -- 172.21.15.2:0/4049594445 learned_addr learned my addr 172.21.15.2:0/4049594445 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:51.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.340+0000 7f4d61348700 1 --2- 172.21.15.2:0/4049594445 >> v2:172.21.15.31:3300/0 conn(0x7f4d5c15d360 0x7f4d5c06b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:51.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.340+0000 7f4d53fff700 1 -- 172.21.15.2:0/4049594445 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 258757605 0 0) 0x7f4d5c0d1b40 con 0x7f4d5c06bf80 2023-08-25T03:34:51.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.340+0000 7f4d53fff700 1 -- 172.21.15.2:0/4049594445 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d480035b0 con 0x7f4d5c06bf80 2023-08-25T03:34:51.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.340+0000 7f4d60b47700 1 -- 172.21.15.2:0/4049594445 >> v2:172.21.15.31:3300/0 conn(0x7f4d5c15d360 msgr2=0x7f4d5c06b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:51.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.340+0000 7f4d60b47700 1 --2- 172.21.15.2:0/4049594445 >> v2:172.21.15.31:3300/0 conn(0x7f4d5c15d360 0x7f4d5c06b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:51.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.340+0000 7f4d60b47700 1 -- 172.21.15.2:0/4049594445 >> v1:172.21.15.2:6789/0 conn(0x7f4d5c06bf80 legacy=0x7f4d5c160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:51.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.340+0000 7f4d60b47700 1 -- 172.21.15.2:0/4049594445 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5c0d0340 con 0x7f4d5c0cfd80 2023-08-25T03:34:51.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.341+0000 7f4d60b47700 1 --2- 172.21.15.2:0/4049594445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d5c0cfd80 0x7f4d5c06ba40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c066e60 tx=0x7f4d4c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68b4f9825fff9adf server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:51.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.341+0000 7f4d53fff700 1 -- 172.21.15.2:0/4049594445 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c011830 con 0x7f4d5c0cfd80 2023-08-25T03:34:51.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.341+0000 7f4d53fff700 1 -- 172.21.15.2:0/4049594445 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d4c011990 con 0x7f4d5c0cfd80 2023-08-25T03:34:51.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.341+0000 7f4d53fff700 1 -- 172.21.15.2:0/4049594445 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4c011d20 con 0x7f4d5c0cfd80 2023-08-25T03:34:51.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.341+0000 7f4d635ac700 1 -- 172.21.15.2:0/4049594445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d5c0cfd80 msgr2=0x7f4d5c06ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:51.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.341+0000 7f4d635ac700 1 --2- 172.21.15.2:0/4049594445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d5c0cfd80 0x7f4d5c06ba40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c066e60 tx=0x7f4d4c009b20 comp rx=0 tx=0).stop 2023-08-25T03:34:51.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.341+0000 7f4d635ac700 1 -- 172.21.15.2:0/4049594445 shutdown_connections 2023-08-25T03:34:51.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.341+0000 7f4d635ac700 1 --2- 172.21.15.2:0/4049594445 >> v2:172.21.15.31:3300/0 conn(0x7f4d5c15d360 0x7f4d5c06b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:51.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.341+0000 7f4d635ac700 1 --2- 172.21.15.2:0/4049594445 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d5c0cfd80 0x7f4d5c06ba40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:51.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.341+0000 7f4d635ac700 1 -- 172.21.15.2:0/4049594445 >> 172.21.15.2:0/4049594445 conn(0x7f4d5c09a200 msgr2=0x7f4d5c0d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:51.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.341+0000 7f4d635ac700 1 -- 172.21.15.2:0/4049594445 shutdown_connections 2023-08-25T03:34:51.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.341+0000 7f4d635ac700 1 -- 172.21.15.2:0/4049594445 wait complete. 2023-08-25T03:34:51.356 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.343+0000 7f4d635ac700 1 Processor -- start 2023-08-25T03:34:51.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.343+0000 7f4d635ac700 1 -- start start 2023-08-25T03:34:51.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.343+0000 7f4d635ac700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d5c06bf80 0x7f4d5c155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:51.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.343+0000 7f4d635ac700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d5c0cfd80 0x7f4d5c15e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:51.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.343+0000 7f4d635ac700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4d5c15d360 0x7f4d5c153190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:51.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.343+0000 7f4d635ac700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c15fcc0 con 0x7f4d5c06bf80 2023-08-25T03:34:51.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.343+0000 7f4d635ac700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c15f0b0 con 0x7f4d5c0cfd80 2023-08-25T03:34:51.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.343+0000 7f4d635ac700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5c15f390 con 0x7f4d5c15d360 2023-08-25T03:34:51.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.343+0000 7f4d61348700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d5c06bf80 0x7f4d5c155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:51.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.343+0000 7f4d61348700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d5c06bf80 0x7f4d5c155220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60800/0 (socket says 172.21.15.2:60800) 2023-08-25T03:34:51.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.344+0000 7f4d61348700 1 -- 172.21.15.2:0/2328311457 learned_addr learned my addr 172.21.15.2:0/2328311457 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:51.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.344+0000 7f4d61348700 1 -- 172.21.15.2:0/2328311457 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4d5c15d360 msgr2=0x7f4d5c153190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:51.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.344+0000 7f4d61348700 1 --2- 172.21.15.2:0/2328311457 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4d5c15d360 0x7f4d5c153190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:51.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.344+0000 7f4d61348700 1 -- 172.21.15.2:0/2328311457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d5c0cfd80 msgr2=0x7f4d5c15e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:51.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.344+0000 7f4d60b47700 1 --2- 172.21.15.2:0/2328311457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d5c0cfd80 0x7f4d5c15e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:51.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.344+0000 7f4d61348700 1 --2- 172.21.15.2:0/2328311457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d5c0cfd80 0x7f4d5c15e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:51.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.344+0000 7f4d61348700 1 -- 172.21.15.2:0/2328311457 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4c0094a0 con 0x7f4d5c06bf80 2023-08-25T03:34:51.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.344+0000 7f4d61348700 1 --2- 172.21.15.2:0/2328311457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d5c06bf80 0x7f4d5c155220 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4d5800e730 tx=0x7f4d5800eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:51.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.344+0000 7f4d51ffb700 1 -- 172.21.15.2:0/2328311457 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58013610 con 0x7f4d5c06bf80 2023-08-25T03:34:51.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.344+0000 7f4d51ffb700 1 -- 172.21.15.2:0/2328311457 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d58013770 con 0x7f4d5c06bf80 2023-08-25T03:34:51.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.344+0000 7f4d635ac700 1 -- 172.21.15.2:0/2328311457 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d5c15f670 con 0x7f4d5c06bf80 2023-08-25T03:34:51.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.344+0000 7f4d51ffb700 1 -- 172.21.15.2:0/2328311457 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58013b20 con 0x7f4d5c06bf80 2023-08-25T03:34:51.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.344+0000 7f4d635ac700 1 -- 172.21.15.2:0/2328311457 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d5c153cf0 con 0x7f4d5c06bf80 2023-08-25T03:34:51.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.345+0000 7f4d635ac700 1 -- 172.21.15.2:0/2328311457 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d5c15ffa0 con 0x7f4d5c06bf80 2023-08-25T03:34:51.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.346+0000 7f4d51ffb700 1 -- 172.21.15.2:0/2328311457 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4d58013c80 con 0x7f4d5c06bf80 2023-08-25T03:34:51.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.346+0000 7f4d51ffb700 1 --2- 172.21.15.2:0/2328311457 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f4d4006a0e0 0x7f4d4006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:51.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.346+0000 7f4d51ffb700 1 -- 172.21.15.2:0/2328311457 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f4d58087f90 con 0x7f4d5c06bf80 2023-08-25T03:34:51.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.346+0000 7f4d60b47700 1 --2- 172.21.15.2:0/2328311457 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f4d4006a0e0 0x7f4d4006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:51.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.347+0000 7f4d60b47700 1 --2- 172.21.15.2:0/2328311457 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f4d4006a0e0 0x7f4d4006c5a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0b6c60 tx=0x7f4d4c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:51.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.351+0000 7f4d51ffb700 1 -- 172.21.15.2:0/2328311457 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4d58057030 con 0x7f4d5c06bf80 2023-08-25T03:34:51.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.550+0000 7f4d635ac700 1 -- 172.21.15.2:0/2328311457 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4d5c0746d0 con 0x7f4d5c06bf80 2023-08-25T03:34:51.551 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.550+0000 7f4d51ffb700 1 -- 172.21.15.2:0/2328311457 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4d58021020 con 0x7f4d5c06bf80 2023-08-25T03:34:51.551 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-08-25T03:34:51.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.552+0000 7f4d635ac700 1 -- 172.21.15.2:0/2328311457 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f4d4006a0e0 msgr2=0x7f4d4006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:51.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.552+0000 7f4d635ac700 1 --2- 172.21.15.2:0/2328311457 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f4d4006a0e0 0x7f4d4006c5a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c0b6c60 tx=0x7f4d4c008030 comp rx=0 tx=0).stop 2023-08-25T03:34:51.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.552+0000 7f4d635ac700 1 -- 172.21.15.2:0/2328311457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d5c06bf80 msgr2=0x7f4d5c155220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:51.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.553+0000 7f4d635ac700 1 --2- 172.21.15.2:0/2328311457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d5c06bf80 0x7f4d5c155220 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4d5800e730 tx=0x7f4d5800eaf0 comp rx=0 tx=0).stop 2023-08-25T03:34:51.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.553+0000 7f4d635ac700 1 -- 172.21.15.2:0/2328311457 shutdown_connections 2023-08-25T03:34:51.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.553+0000 7f4d635ac700 1 --2- 172.21.15.2:0/2328311457 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f4d4006a0e0 0x7f4d4006c5a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:51.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.553+0000 7f4d635ac700 1 --2- 172.21.15.2:0/2328311457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4d5c06bf80 0x7f4d5c155220 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:51.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.553+0000 7f4d635ac700 1 --2- 172.21.15.2:0/2328311457 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4d5c15d360 0x7f4d5c153190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:51.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.553+0000 7f4d635ac700 1 --2- 172.21.15.2:0/2328311457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d5c0cfd80 0x7f4d5c15e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:51.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.553+0000 7f4d635ac700 1 -- 172.21.15.2:0/2328311457 >> 172.21.15.2:0/2328311457 conn(0x7f4d5c09a200 msgr2=0x7f4d5c06c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:51.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.553+0000 7f4d635ac700 1 -- 172.21.15.2:0/2328311457 shutdown_connections 2023-08-25T03:34:51.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.553+0000 7f4d635ac700 1 -- 172.21.15.2:0/2328311457 wait complete. 2023-08-25T03:34:51.566 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T03:34:51.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.697+0000 7f2becbfa700 1 Processor -- start 2023-08-25T03:34:51.698 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.697+0000 7f2becbfa700 1 -- start start 2023-08-25T03:34:51.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.698+0000 7f2becbfa700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2be81607f0 0x7f2be8070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:51.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.698+0000 7f2becbfa700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f2be80af180 0x7f2be80ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:51.699 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.698+0000 7f2becbfa700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be8160bd0 con 0x7f2be806a1e0 2023-08-25T03:34:51.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.698+0000 7f2becbfa700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be80713a0 con 0x7f2be81607f0 2023-08-25T03:34:51.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.698+0000 7f2becbfa700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be80718a0 con 0x7f2be80af180 2023-08-25T03:34:51.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.698+0000 7f2be659c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2be81607f0 0x7f2be8070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:51.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.698+0000 7f2be6d9d700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f2be806a1e0 0x7f2be806ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47478/0 (socket says 172.21.15.2:47478) 2023-08-25T03:34:51.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.698+0000 7f2be659c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2be81607f0 0x7f2be8070e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60808/0 (socket says 172.21.15.2:60808) 2023-08-25T03:34:51.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.698+0000 7f2be6d9d700 1 -- 172.21.15.2:0/2329382093 learned_addr learned my addr 172.21.15.2:0/2329382093 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:51.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.698+0000 7f2be5d9b700 1 --2- 172.21.15.2:0/2329382093 >> v2:172.21.15.31:3300/0 conn(0x7f2be80af180 0x7f2be80ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:51.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.698+0000 7f2be659c700 1 -- 172.21.15.2:0/2329382093 >> v2:172.21.15.31:3300/0 conn(0x7f2be80af180 msgr2=0x7f2be80ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:51.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.698+0000 7f2be659c700 1 --2- 172.21.15.2:0/2329382093 >> v2:172.21.15.31:3300/0 conn(0x7f2be80af180 0x7f2be80ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:51.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.698+0000 7f2be659c700 1 -- 172.21.15.2:0/2329382093 >> v1:172.21.15.2:6789/0 conn(0x7f2be806a1e0 legacy=0x7f2be806ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:51.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.698+0000 7f2be659c700 1 -- 172.21.15.2:0/2329382093 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be80cdd00 con 0x7f2be81607f0 2023-08-25T03:34:51.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.698+0000 7f2be559a700 1 -- 172.21.15.2:0/2329382093 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 294211269 0 0) 0x7f2be8160bd0 con 0x7f2be806a1e0 2023-08-25T03:34:51.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.699+0000 7f2be659c700 1 --2- 172.21.15.2:0/2329382093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2be81607f0 0x7f2be8070e60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2be8071f20 tx=0x7f2bdc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7cdfc0c09a3bce server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:51.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.699+0000 7f2be559a700 1 -- 172.21.15.2:0/2329382093 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc011830 con 0x7f2be81607f0 2023-08-25T03:34:51.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.699+0000 7f2be559a700 1 -- 172.21.15.2:0/2329382093 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bdc011990 con 0x7f2be81607f0 2023-08-25T03:34:51.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.699+0000 7f2be559a700 1 -- 172.21.15.2:0/2329382093 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bdc011d20 con 0x7f2be81607f0 2023-08-25T03:34:51.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.699+0000 7f2becbfa700 1 -- 172.21.15.2:0/2329382093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2be81607f0 msgr2=0x7f2be8070e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:51.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.699+0000 7f2becbfa700 1 --2- 172.21.15.2:0/2329382093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2be81607f0 0x7f2be8070e60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2be8071f20 tx=0x7f2bdc009b20 comp rx=0 tx=0).stop 2023-08-25T03:34:51.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.699+0000 7f2becbfa700 1 -- 172.21.15.2:0/2329382093 shutdown_connections 2023-08-25T03:34:51.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.699+0000 7f2becbfa700 1 --2- 172.21.15.2:0/2329382093 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2be81607f0 0x7f2be8070e60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:51.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.699+0000 7f2becbfa700 1 --2- 172.21.15.2:0/2329382093 >> v2:172.21.15.31:3300/0 conn(0x7f2be80af180 0x7f2be80ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:51.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.699+0000 7f2becbfa700 1 -- 172.21.15.2:0/2329382093 >> 172.21.15.2:0/2329382093 conn(0x7f2be80cfd90 msgr2=0x7f2be80d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:51.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.699+0000 7f2becbfa700 1 -- 172.21.15.2:0/2329382093 shutdown_connections 2023-08-25T03:34:51.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.699+0000 7f2becbfa700 1 -- 172.21.15.2:0/2329382093 wait complete. 2023-08-25T03:34:51.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.701+0000 7f2becbfa700 1 Processor -- start 2023-08-25T03:34:51.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.701+0000 7f2becbfa700 1 -- start start 2023-08-25T03:34:51.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.701+0000 7f2becbfa700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be806a1e0 0x7f2be80c6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:51.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.702+0000 7f2becbfa700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2be80af180 0x7f2be80c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:51.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.702+0000 7f2becbfa700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2be81607f0 0x7f2be81f2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:51.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.702+0000 7f2becbfa700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be80c8de0 con 0x7f2be81607f0 2023-08-25T03:34:51.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.702+0000 7f2becbfa700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be81f3270 con 0x7f2be806a1e0 2023-08-25T03:34:51.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.702+0000 7f2becbfa700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be80c81f0 con 0x7f2be80af180 2023-08-25T03:34:51.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.702+0000 7f2be6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2be81607f0 0x7f2be81f2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:51.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.702+0000 7f2be6d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2be81607f0 0x7f2be81f2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60818/0 (socket says 172.21.15.2:60818) 2023-08-25T03:34:51.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.702+0000 7f2be6d9d700 1 -- 172.21.15.2:0/1304314145 learned_addr learned my addr 172.21.15.2:0/1304314145 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:51.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.702+0000 7f2be659c700 1 --2- 172.21.15.2:0/1304314145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be806a1e0 0x7f2be80c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:51.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.702+0000 7f2be5d9b700 1 --2- 172.21.15.2:0/1304314145 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2be80af180 0x7f2be80c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:51.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.702+0000 7f2be6d9d700 1 -- 172.21.15.2:0/1304314145 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2be80af180 msgr2=0x7f2be80c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:51.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.702+0000 7f2be6d9d700 1 --2- 172.21.15.2:0/1304314145 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2be80af180 0x7f2be80c4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:51.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.702+0000 7f2be6d9d700 1 -- 172.21.15.2:0/1304314145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be806a1e0 msgr2=0x7f2be80c6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:51.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.702+0000 7f2be6d9d700 1 --2- 172.21.15.2:0/1304314145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be806a1e0 0x7f2be80c6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:51.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.702+0000 7f2be6d9d700 1 -- 172.21.15.2:0/1304314145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bdc0094a0 con 0x7f2be81607f0 2023-08-25T03:34:51.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.702+0000 7f2be659c700 1 --2- 172.21.15.2:0/1304314145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be806a1e0 0x7f2be80c6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:51.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.702+0000 7f2be6d9d700 1 --2- 172.21.15.2:0/1304314145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2be81607f0 0x7f2be81f2d30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2be00015b0 tx=0x7f2be000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:51.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.702+0000 7f2bd6ffd700 1 -- 172.21.15.2:0/1304314145 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be000ab00 con 0x7f2be81607f0 2023-08-25T03:34:51.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.702+0000 7f2bd6ffd700 1 -- 172.21.15.2:0/1304314145 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2be000ac60 con 0x7f2be81607f0 2023-08-25T03:34:51.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.702+0000 7f2becbfa700 1 -- 172.21.15.2:0/1304314145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be80c8440 con 0x7f2be81607f0 2023-08-25T03:34:51.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.702+0000 7f2bd6ffd700 1 -- 172.21.15.2:0/1304314145 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2be0012400 con 0x7f2be81607f0 2023-08-25T03:34:51.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.702+0000 7f2becbfa700 1 -- 172.21.15.2:0/1304314145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be80c8a00 con 0x7f2be81607f0 2023-08-25T03:34:51.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.703+0000 7f2becbfa700 1 -- 172.21.15.2:0/1304314145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be80c8f20 con 0x7f2be81607f0 2023-08-25T03:34:51.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.704+0000 7f2bd6ffd700 1 -- 172.21.15.2:0/1304314145 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2be0007d50 con 0x7f2be81607f0 2023-08-25T03:34:51.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.705+0000 7f2bd6ffd700 1 --2- 172.21.15.2:0/1304314145 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2bc806a0e0 0x7f2bc806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:51.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.705+0000 7f2bd6ffd700 1 -- 172.21.15.2:0/1304314145 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f2be0088370 con 0x7f2be81607f0 2023-08-25T03:34:51.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.705+0000 7f2be659c700 1 --2- 172.21.15.2:0/1304314145 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2bc806a0e0 0x7f2bc806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:51.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.705+0000 7f2be659c700 1 --2- 172.21.15.2:0/1304314145 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2bc806a0e0 0x7f2bc806c5a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc0090b0 tx=0x7f2bdc01c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:51.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.709+0000 7f2bd6ffd700 1 -- 172.21.15.2:0/1304314145 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2be0053b90 con 0x7f2be81607f0 2023-08-25T03:34:51.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.909+0000 7f2becbfa700 1 -- 172.21.15.2:0/1304314145 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2be80746d0 con 0x7f2be81607f0 2023-08-25T03:34:51.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.910+0000 7f2bd6ffd700 1 -- 172.21.15.2:0/1304314145 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2be00571b0 con 0x7f2be81607f0 2023-08-25T03:34:51.911 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-08-25T03:34:51.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.912+0000 7f2becbfa700 1 -- 172.21.15.2:0/1304314145 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2bc806a0e0 msgr2=0x7f2bc806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:51.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.912+0000 7f2becbfa700 1 --2- 172.21.15.2:0/1304314145 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2bc806a0e0 0x7f2bc806c5a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc0090b0 tx=0x7f2bdc01c040 comp rx=0 tx=0).stop 2023-08-25T03:34:51.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.912+0000 7f2becbfa700 1 -- 172.21.15.2:0/1304314145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2be81607f0 msgr2=0x7f2be81f2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:51.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.912+0000 7f2becbfa700 1 --2- 172.21.15.2:0/1304314145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2be81607f0 0x7f2be81f2d30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2be00015b0 tx=0x7f2be000cee0 comp rx=0 tx=0).stop 2023-08-25T03:34:51.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.912+0000 7f2becbfa700 1 -- 172.21.15.2:0/1304314145 shutdown_connections 2023-08-25T03:34:51.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.912+0000 7f2becbfa700 1 --2- 172.21.15.2:0/1304314145 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2bc806a0e0 0x7f2bc806c5a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:51.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.912+0000 7f2becbfa700 1 --2- 172.21.15.2:0/1304314145 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2be81607f0 0x7f2be81f2d30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:51.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.912+0000 7f2becbfa700 1 --2- 172.21.15.2:0/1304314145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2be806a1e0 0x7f2be80c6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:51.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.912+0000 7f2becbfa700 1 --2- 172.21.15.2:0/1304314145 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2be80af180 0x7f2be80c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:51.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.912+0000 7f2becbfa700 1 -- 172.21.15.2:0/1304314145 >> 172.21.15.2:0/1304314145 conn(0x7f2be80cfd90 msgr2=0x7f2be80adc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:51.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.912+0000 7f2becbfa700 1 -- 172.21.15.2:0/1304314145 shutdown_connections 2023-08-25T03:34:51.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:51.912+0000 7f2becbfa700 1 -- 172.21.15.2:0/1304314145 wait complete. 2023-08-25T03:34:51.926 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T03:34:52.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.051+0000 7f16d98f7700 1 Processor -- start 2023-08-25T03:34:52.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.051+0000 7f16d98f7700 1 -- start start 2023-08-25T03:34:52.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.052+0000 7f16d98f7700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f16d415d370 0x7f16d40d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:52.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.052+0000 7f16d98f7700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16d40ce4f0 0x7f16d40b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:52.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.052+0000 7f16d98f7700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d415d750 con 0x7f16d40b8aa0 2023-08-25T03:34:52.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.052+0000 7f16d98f7700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16d40d1aa0 con 0x7f16d415d370 2023-08-25T03:34:52.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.052+0000 7f16d98f7700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d40d1d80 con 0x7f16d40ce4f0 2023-08-25T03:34:52.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.052+0000 7f16d27fc700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f16d40b8aa0 0x7f16d40cdde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47488/0 (socket says 172.21.15.2:47488) 2023-08-25T03:34:52.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.052+0000 7f16d27fc700 1 -- 172.21.15.2:0/980536978 learned_addr learned my addr 172.21.15.2:0/980536978 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:52.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.052+0000 7f16d2ffd700 1 --2- 172.21.15.2:0/980536978 >> v2:172.21.15.31:3300/0 conn(0x7f16d415d370 0x7f16d40d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:52.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.052+0000 7f16d37fe700 1 --2- 172.21.15.2:0/980536978 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16d40ce4f0 0x7f16d40b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:52.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.052+0000 7f16d1ffb700 1 -- 172.21.15.2:0/980536978 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3511576798 0 0) 0x7f16d415d750 con 0x7f16d40b8aa0 2023-08-25T03:34:52.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.052+0000 7f16d1ffb700 1 -- 172.21.15.2:0/980536978 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16bc0035b0 con 0x7f16d40b8aa0 2023-08-25T03:34:52.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.053+0000 7f16d1ffb700 1 -- 172.21.15.2:0/980536978 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1483345358 0 0) 0x7f16bc0035b0 con 0x7f16d40b8aa0 2023-08-25T03:34:52.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.053+0000 7f16d1ffb700 1 -- 172.21.15.2:0/980536978 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16d40ce4f0 msgr2=0x7f16d40b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:52.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.053+0000 7f16d1ffb700 1 --2- 172.21.15.2:0/980536978 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16d40ce4f0 0x7f16d40b62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:52.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.053+0000 7f16d1ffb700 1 -- 172.21.15.2:0/980536978 >> v2:172.21.15.31:3300/0 conn(0x7f16d415d370 msgr2=0x7f16d40d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:52.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.053+0000 7f16d1ffb700 1 --2- 172.21.15.2:0/980536978 >> v2:172.21.15.31:3300/0 conn(0x7f16d415d370 0x7f16d40d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:52.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.053+0000 7f16d1ffb700 1 -- 172.21.15.2:0/980536978 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d40d2060 con 0x7f16d40b8aa0 2023-08-25T03:34:52.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.053+0000 7f16d1ffb700 1 -- 172.21.15.2:0/980536978 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f16c4001670 con 0x7f16d40b8aa0 2023-08-25T03:34:52.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.053+0000 7f16d1ffb700 1 -- 172.21.15.2:0/980536978 >> v1:172.21.15.2:6789/0 conn(0x7f16d40b8aa0 legacy=0x7f16d40cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:52.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.053+0000 7f16d1ffb700 1 --2- 172.21.15.2:0/980536978 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16bc003a00 0x7f16bc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:52.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.053+0000 7f16d1ffb700 1 --2- 172.21.15.2:0/980536978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16bc004850 0x7f16bc004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:52.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.053+0000 7f16d1ffb700 1 --2- 172.21.15.2:0/980536978 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16d40ce4f0 0x7f16bc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:52.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.053+0000 7f16d1ffb700 1 -- 172.21.15.2:0/980536978 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d40d1aa0 con 0x7f16bc003a00 2023-08-25T03:34:52.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.053+0000 7f16d1ffb700 1 -- 172.21.15.2:0/980536978 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d40d1d80 con 0x7f16bc004850 2023-08-25T03:34:52.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.053+0000 7f16d1ffb700 1 -- 172.21.15.2:0/980536978 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc00ba40 con 0x7f16d40ce4f0 2023-08-25T03:34:52.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.053+0000 7f16d2ffd700 1 --2- 172.21.15.2:0/980536978 >> v2:172.21.15.31:3300/0 conn(0x7f16d415d370 0x7f16d40d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:34:52.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.053+0000 7f16d2ffd700 1 --2- 172.21.15.2:0/980536978 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16bc003a00 0x7f16bc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:52.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.053+0000 7f16d27fc700 1 --2- 172.21.15.2:0/980536978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16bc004850 0x7f16bc004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:52.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.053+0000 7f16d37fe700 1 --2- 172.21.15.2:0/980536978 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16d40ce4f0 0x7f16bc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:52.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.053+0000 7f16d2ffd700 1 -- 172.21.15.2:0/980536978 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16d40ce4f0 msgr2=0x7f16bc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:52.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.053+0000 7f16d2ffd700 1 --2- 172.21.15.2:0/980536978 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16d40ce4f0 0x7f16bc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:52.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.053+0000 7f16d2ffd700 1 -- 172.21.15.2:0/980536978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16bc004850 msgr2=0x7f16bc004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:52.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.053+0000 7f16d2ffd700 1 --2- 172.21.15.2:0/980536978 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16bc004850 0x7f16bc004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:52.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.053+0000 7f16d2ffd700 1 -- 172.21.15.2:0/980536978 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16bc0035b0 con 0x7f16bc003a00 2023-08-25T03:34:52.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.053+0000 7f16d2ffd700 1 --2- 172.21.15.2:0/980536978 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16bc003a00 0x7f16bc003e20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f16c8009d10 tx=0x7f16bc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eade59b5ac4592a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:52.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.054+0000 7f16d1ffb700 1 -- 172.21.15.2:0/980536978 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c8009e60 con 0x7f16bc003a00 2023-08-25T03:34:52.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.054+0000 7f16d1ffb700 1 -- 172.21.15.2:0/980536978 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c8004030 con 0x7f16bc003a00 2023-08-25T03:34:52.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.054+0000 7f16d1ffb700 1 -- 172.21.15.2:0/980536978 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c80043a0 con 0x7f16bc003a00 2023-08-25T03:34:52.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.054+0000 7f16d98f7700 1 -- 172.21.15.2:0/980536978 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16bc003a00 msgr2=0x7f16bc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:52.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.054+0000 7f16d98f7700 1 --2- 172.21.15.2:0/980536978 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16bc003a00 0x7f16bc003e20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f16c8009d10 tx=0x7f16bc005880 comp rx=0 tx=0).stop 2023-08-25T03:34:52.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.054+0000 7f16d37fe700 1 -- 172.21.15.2:0/980536978 reap_dead start 2023-08-25T03:34:52.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.054+0000 7f16d98f7700 1 -- 172.21.15.2:0/980536978 shutdown_connections 2023-08-25T03:34:52.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.054+0000 7f16d98f7700 1 -- 172.21.15.2:0/980536978 >> 172.21.15.2:0/980536978 conn(0x7f16d406ab50 msgr2=0x7f16d406af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:52.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.054+0000 7f16d98f7700 1 -- 172.21.15.2:0/980536978 shutdown_connections 2023-08-25T03:34:52.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.054+0000 7f16d98f7700 1 -- 172.21.15.2:0/980536978 wait complete. 2023-08-25T03:34:52.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.055+0000 7f16d98f7700 1 Processor -- start 2023-08-25T03:34:52.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.056+0000 7f16d98f7700 1 -- start start 2023-08-25T03:34:52.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.056+0000 7f16d98f7700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16bc003a00 0x7f16d41f2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:52.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.056+0000 7f16d98f7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16d40c9810 0x7f16d41f5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:52.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.056+0000 7f16d98f7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16d414fbd0 0x7f16d4153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:52.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.056+0000 7f16d98f7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d41f5aa0 con 0x7f16d414fbd0 2023-08-25T03:34:52.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.056+0000 7f16d98f7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d40c7120 con 0x7f16d40c9810 2023-08-25T03:34:52.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.056+0000 7f16d98f7700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d4152bf0 con 0x7f16bc003a00 2023-08-25T03:34:52.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.056+0000 7f16d37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16d414fbd0 0x7f16d4153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:52.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.056+0000 7f16d37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16d414fbd0 0x7f16d4153b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60836/0 (socket says 172.21.15.2:60836) 2023-08-25T03:34:52.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.056+0000 7f16d37fe700 1 -- 172.21.15.2:0/1304295019 learned_addr learned my addr 172.21.15.2:0/1304295019 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:52.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.057+0000 7f16d27fc700 1 --2- 172.21.15.2:0/1304295019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16d40c9810 0x7f16d41f5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:52.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.057+0000 7f16d2ffd700 1 --2- 172.21.15.2:0/1304295019 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16bc003a00 0x7f16d41f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:52.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.057+0000 7f16d37fe700 1 -- 172.21.15.2:0/1304295019 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16bc003a00 msgr2=0x7f16d41f2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:52.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.057+0000 7f16d37fe700 1 --2- 172.21.15.2:0/1304295019 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16bc003a00 0x7f16d41f2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:52.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.057+0000 7f16d37fe700 1 -- 172.21.15.2:0/1304295019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16d40c9810 msgr2=0x7f16d41f5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:52.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.057+0000 7f16d37fe700 1 --2- 172.21.15.2:0/1304295019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16d40c9810 0x7f16d41f5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:52.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.057+0000 7f16d37fe700 1 -- 172.21.15.2:0/1304295019 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d4152ed0 con 0x7f16d414fbd0 2023-08-25T03:34:52.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.057+0000 7f16d27fc700 1 --2- 172.21.15.2:0/1304295019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16d40c9810 0x7f16d41f5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:52.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.057+0000 7f16d37fe700 1 --2- 172.21.15.2:0/1304295019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16d414fbd0 0x7f16d4153b50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f16d40ca190 tx=0x7f16cc011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:52.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.057+0000 7f16bbfff700 1 -- 172.21.15.2:0/1304295019 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc00e040 con 0x7f16d414fbd0 2023-08-25T03:34:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.057+0000 7f16bbfff700 1 -- 172.21.15.2:0/1304295019 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16cc00f040 con 0x7f16d414fbd0 2023-08-25T03:34:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.057+0000 7f16d98f7700 1 -- 172.21.15.2:0/1304295019 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d40c6ad0 con 0x7f16d414fbd0 2023-08-25T03:34:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.057+0000 7f16bbfff700 1 -- 172.21.15.2:0/1304295019 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16cc00fdb0 con 0x7f16d414fbd0 2023-08-25T03:34:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.057+0000 7f16d98f7700 1 -- 172.21.15.2:0/1304295019 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d4152790 con 0x7f16d414fbd0 2023-08-25T03:34:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.058+0000 7f16d98f7700 1 -- 172.21.15.2:0/1304295019 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d41529b0 con 0x7f16d414fbd0 2023-08-25T03:34:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.059+0000 7f16bbfff700 1 -- 172.21.15.2:0/1304295019 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f16cc022460 con 0x7f16d414fbd0 2023-08-25T03:34:52.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.059+0000 7f16bbfff700 1 --2- 172.21.15.2:0/1304295019 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f16c006a0e0 0x7f16c006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:52.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.059+0000 7f16bbfff700 1 -- 172.21.15.2:0/1304295019 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f16cc088b40 con 0x7f16d414fbd0 2023-08-25T03:34:52.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.059+0000 7f16d2ffd700 1 --2- 172.21.15.2:0/1304295019 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f16c006a0e0 0x7f16c006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:52.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.060+0000 7f16d2ffd700 1 --2- 172.21.15.2:0/1304295019 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f16c006a0e0 0x7f16c006c5a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f16c80090b0 tx=0x7f16c8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:52.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.064+0000 7f16bbfff700 1 -- 172.21.15.2:0/1304295019 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f16d41529b0 con 0x7f16d414fbd0 2023-08-25T03:34:52.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.279+0000 7f16d98f7700 1 -- 172.21.15.2:0/1304295019 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f16d41529b0 con 0x7f16d414fbd0 2023-08-25T03:34:52.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.280+0000 7f16bbfff700 1 -- 172.21.15.2:0/1304295019 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f16d41529b0 con 0x7f16d414fbd0 2023-08-25T03:34:52.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.281+0000 7f16d98f7700 1 -- 172.21.15.2:0/1304295019 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f16c006a0e0 msgr2=0x7f16c006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:52.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.281+0000 7f16d98f7700 1 --2- 172.21.15.2:0/1304295019 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f16c006a0e0 0x7f16c006c5a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f16c80090b0 tx=0x7f16c8008030 comp rx=0 tx=0).stop 2023-08-25T03:34:52.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.281+0000 7f16d98f7700 1 -- 172.21.15.2:0/1304295019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16d414fbd0 msgr2=0x7f16d4153b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:52.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.281+0000 7f16d98f7700 1 --2- 172.21.15.2:0/1304295019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16d414fbd0 0x7f16d4153b50 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f16d40ca190 tx=0x7f16cc011f80 comp rx=0 tx=0).stop 2023-08-25T03:34:52.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.281+0000 7f16d98f7700 1 -- 172.21.15.2:0/1304295019 shutdown_connections 2023-08-25T03:34:52.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.281+0000 7f16d98f7700 1 --2- 172.21.15.2:0/1304295019 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f16c006a0e0 0x7f16c006c5a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:52.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.281+0000 7f16d98f7700 1 --2- 172.21.15.2:0/1304295019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16d414fbd0 0x7f16d4153b50 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:52.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.281+0000 7f16d98f7700 1 --2- 172.21.15.2:0/1304295019 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16bc003a00 0x7f16d41f2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:52.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.281+0000 7f16d98f7700 1 --2- 172.21.15.2:0/1304295019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16d40c9810 0x7f16d41f5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:52.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.281+0000 7f16d98f7700 1 -- 172.21.15.2:0/1304295019 >> 172.21.15.2:0/1304295019 conn(0x7f16d406ab50 msgr2=0x7f16d406af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:52.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.282+0000 7f16d98f7700 1 -- 172.21.15.2:0/1304295019 shutdown_connections 2023-08-25T03:34:52.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.282+0000 7f16d98f7700 1 -- 172.21.15.2:0/1304295019 wait complete. 2023-08-25T03:34:52.287 INFO:teuthology.orchestra.run.smithi002.stderr:listed 0 entries 2023-08-25T03:34:52.295 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-08-25T03:34:52.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:52.296 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-08-25T03:34:52.344 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-08-25T03:34:52.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:34:52.345 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-08-25T03:34:52.364 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-08-25T03:34:52.365 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T03:34:52.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.489+0000 7f3e5bb22700 1 Processor -- start 2023-08-25T03:34:52.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.490+0000 7f3e5bb22700 1 -- start start 2023-08-25T03:34:52.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.490+0000 7f3e5bb22700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e540ccd50 0x7f3e540cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:52.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.490+0000 7f3e5bb22700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e54071dc0 con 0x7f3e5415a410 2023-08-25T03:34:52.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.490+0000 7f3e5bb22700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e540720a0 con 0x7f3e5415a030 2023-08-25T03:34:52.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.490+0000 7f3e5bb22700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e540cd720 con 0x7f3e540ccd50 2023-08-25T03:34:52.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.490+0000 7f3e590bd700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f3e5415a410 0x7f3e540ce170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47508/0 (socket says 172.21.15.2:47508) 2023-08-25T03:34:52.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.490+0000 7f3e590bd700 1 -- 172.21.15.2:0/898391639 learned_addr learned my addr 172.21.15.2:0/898391639 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:52.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.491+0000 7f3e5a0bf700 1 --2- 172.21.15.2:0/898391639 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e540ccd50 0x7f3e540cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:52.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.491+0000 7f3e588bc700 1 -- 172.21.15.2:0/898391639 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206235186 0 0) 0x7f3e54071dc0 con 0x7f3e5415a410 2023-08-25T03:34:52.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.491+0000 7f3e588bc700 1 -- 172.21.15.2:0/898391639 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e400035b0 con 0x7f3e5415a410 2023-08-25T03:34:52.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.491+0000 7f3e588bc700 1 -- 172.21.15.2:0/898391639 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2097742922 0 0) 0x7f3e540720a0 con 0x7f3e5415a030 2023-08-25T03:34:52.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.491+0000 7f3e588bc700 1 -- 172.21.15.2:0/898391639 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e54071dc0 con 0x7f3e5415a030 2023-08-25T03:34:52.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.491+0000 7f3e588bc700 1 -- 172.21.15.2:0/898391639 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2904801281 0 0) 0x7f3e400035b0 con 0x7f3e5415a410 2023-08-25T03:34:52.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.491+0000 7f3e588bc700 1 -- 172.21.15.2:0/898391639 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e540ccd50 msgr2=0x7f3e540cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:52.494 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.491+0000 7f3e588bc700 1 --2- 172.21.15.2:0/898391639 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e540ccd50 0x7f3e540cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:52.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.491+0000 7f3e588bc700 1 -- 172.21.15.2:0/898391639 >> v1:172.21.15.31:6789/0 conn(0x7f3e5415a030 legacy=0x7f3e540cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:52.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.491+0000 7f3e588bc700 1 -- 172.21.15.2:0/898391639 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e540cb4b0 con 0x7f3e5415a410 2023-08-25T03:34:52.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.491+0000 7f3e588bc700 1 -- 172.21.15.2:0/898391639 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f3e48001670 con 0x7f3e5415a410 2023-08-25T03:34:52.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.491+0000 7f3e588bc700 1 -- 172.21.15.2:0/898391639 >> v1:172.21.15.2:6789/0 conn(0x7f3e5415a410 legacy=0x7f3e540ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:52.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.491+0000 7f3e588bc700 1 --2- 172.21.15.2:0/898391639 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e540ccd50 0x7f3e40003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:52.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.491+0000 7f3e588bc700 1 --2- 172.21.15.2:0/898391639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e40004120 0x7f3e40004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:52.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.491+0000 7f3e588bc700 1 --2- 172.21.15.2:0/898391639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e40004d90 0x7f3e40005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:52.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.491+0000 7f3e588bc700 1 -- 172.21.15.2:0/898391639 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e540cd720 con 0x7f3e40004120 2023-08-25T03:34:52.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.491+0000 7f3e588bc700 1 -- 172.21.15.2:0/898391639 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4000bba0 con 0x7f3e40004d90 2023-08-25T03:34:52.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.491+0000 7f3e588bc700 1 -- 172.21.15.2:0/898391639 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e4000bd30 con 0x7f3e540ccd50 2023-08-25T03:34:52.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.491+0000 7f3e590bd700 1 --2- 172.21.15.2:0/898391639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e40004120 0x7f3e40004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:52.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.491+0000 7f3e598be700 1 --2- 172.21.15.2:0/898391639 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e540ccd50 0x7f3e40003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:52.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.491+0000 7f3e5a0bf700 1 --2- 172.21.15.2:0/898391639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e40004d90 0x7f3e40005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:52.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.492+0000 7f3e590bd700 1 -- 172.21.15.2:0/898391639 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e540ccd50 msgr2=0x7f3e40003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:52.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.492+0000 7f3e590bd700 1 --2- 172.21.15.2:0/898391639 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e540ccd50 0x7f3e40003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:52.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.492+0000 7f3e590bd700 1 -- 172.21.15.2:0/898391639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e40004d90 msgr2=0x7f3e40005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:52.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.492+0000 7f3e590bd700 1 --2- 172.21.15.2:0/898391639 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e40004d90 0x7f3e40005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:52.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.492+0000 7f3e590bd700 1 -- 172.21.15.2:0/898391639 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e400035b0 con 0x7f3e40004120 2023-08-25T03:34:52.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.492+0000 7f3e590bd700 1 --2- 172.21.15.2:0/898391639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e40004120 0x7f3e40004850 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3e4800c210 tx=0x7f3e40005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b92c06fc8e50059e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:52.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.492+0000 7f3e588bc700 1 -- 172.21.15.2:0/898391639 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4800cd50 con 0x7f3e40004120 2023-08-25T03:34:52.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.492+0000 7f3e588bc700 1 -- 172.21.15.2:0/898391639 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e48005960 con 0x7f3e40004120 2023-08-25T03:34:52.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.492+0000 7f3e588bc700 1 -- 172.21.15.2:0/898391639 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e48005d10 con 0x7f3e40004120 2023-08-25T03:34:52.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.492+0000 7f3e5bb22700 1 -- 172.21.15.2:0/898391639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e40004120 msgr2=0x7f3e40004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:52.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.492+0000 7f3e5bb22700 1 --2- 172.21.15.2:0/898391639 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e40004120 0x7f3e40004850 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f3e4800c210 tx=0x7f3e40005880 comp rx=0 tx=0).stop 2023-08-25T03:34:52.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.492+0000 7f3e5a0bf700 1 -- 172.21.15.2:0/898391639 reap_dead start 2023-08-25T03:34:52.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.493+0000 7f3e5bb22700 1 -- 172.21.15.2:0/898391639 shutdown_connections 2023-08-25T03:34:52.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.493+0000 7f3e5bb22700 1 -- 172.21.15.2:0/898391639 >> 172.21.15.2:0/898391639 conn(0x7f3e540b1cf0 msgr2=0x7f3e540b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:52.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.493+0000 7f3e5bb22700 1 -- 172.21.15.2:0/898391639 shutdown_connections 2023-08-25T03:34:52.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.493+0000 7f3e5bb22700 1 -- 172.21.15.2:0/898391639 wait complete. 2023-08-25T03:34:52.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.494+0000 7f3e5bb22700 1 Processor -- start 2023-08-25T03:34:52.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.494+0000 7f3e5bb22700 1 -- start start 2023-08-25T03:34:52.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.495+0000 7f3e5bb22700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e40004120 0x7f3e5414d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:52.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.495+0000 7f3e5bb22700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e541f6140 0x7f3e5414b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:52.501 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.495+0000 7f3e5bb22700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e5415b460 0x7f3e5406a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:52.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.495+0000 7f3e5bb22700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e540ac8e0 con 0x7f3e5415b460 2023-08-25T03:34:52.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.495+0000 7f3e5bb22700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e540acb80 con 0x7f3e40004120 2023-08-25T03:34:52.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.495+0000 7f3e5bb22700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e541f6900 con 0x7f3e541f6140 2023-08-25T03:34:52.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.495+0000 7f3e5a0bf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e5415b460 0x7f3e5406a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:52.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.495+0000 7f3e5a0bf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e5415b460 0x7f3e5406a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60858/0 (socket says 172.21.15.2:60858) 2023-08-25T03:34:52.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.495+0000 7f3e5a0bf700 1 -- 172.21.15.2:0/2240837081 learned_addr learned my addr 172.21.15.2:0/2240837081 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:52.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.495+0000 7f3e598be700 1 --2- 172.21.15.2:0/2240837081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e40004120 0x7f3e5414d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:52.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.495+0000 7f3e590bd700 1 --2- 172.21.15.2:0/2240837081 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e541f6140 0x7f3e5414b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:52.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.495+0000 7f3e5a0bf700 1 -- 172.21.15.2:0/2240837081 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e541f6140 msgr2=0x7f3e5414b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:52.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.495+0000 7f3e5a0bf700 1 --2- 172.21.15.2:0/2240837081 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e541f6140 0x7f3e5414b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:52.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.495+0000 7f3e5a0bf700 1 -- 172.21.15.2:0/2240837081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e40004120 msgr2=0x7f3e5414d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:52.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.495+0000 7f3e5a0bf700 1 --2- 172.21.15.2:0/2240837081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e40004120 0x7f3e5414d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:52.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.495+0000 7f3e5a0bf700 1 -- 172.21.15.2:0/2240837081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e541f6be0 con 0x7f3e5415b460 2023-08-25T03:34:52.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.495+0000 7f3e5a0bf700 1 --2- 172.21.15.2:0/2240837081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e5415b460 0x7f3e5406a1e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3e540b60d0 tx=0x7f3e4c011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:52.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.495+0000 7f3e467fc700 1 -- 172.21.15.2:0/2240837081 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c00e040 con 0x7f3e5415b460 2023-08-25T03:34:52.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.495+0000 7f3e467fc700 1 -- 172.21.15.2:0/2240837081 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4c00f040 con 0x7f3e5415b460 2023-08-25T03:34:52.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.495+0000 7f3e5bb22700 1 -- 172.21.15.2:0/2240837081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e541f6ec0 con 0x7f3e5415b460 2023-08-25T03:34:52.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.495+0000 7f3e467fc700 1 -- 172.21.15.2:0/2240837081 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c00fde0 con 0x7f3e5415b460 2023-08-25T03:34:52.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.495+0000 7f3e5bb22700 1 -- 172.21.15.2:0/2240837081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e5414b840 con 0x7f3e5415b460 2023-08-25T03:34:52.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.496+0000 7f3e5bb22700 1 -- 172.21.15.2:0/2240837081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e54066f70 con 0x7f3e5415b460 2023-08-25T03:34:52.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.497+0000 7f3e467fc700 1 -- 172.21.15.2:0/2240837081 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3e4c015430 con 0x7f3e5415b460 2023-08-25T03:34:52.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.497+0000 7f3e467fc700 1 --2- 172.21.15.2:0/2240837081 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3e3806a0e0 0x7f3e3806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:52.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.497+0000 7f3e467fc700 1 -- 172.21.15.2:0/2240837081 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f3e4c088d60 con 0x7f3e5415b460 2023-08-25T03:34:52.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.497+0000 7f3e598be700 1 --2- 172.21.15.2:0/2240837081 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3e3806a0e0 0x7f3e3806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:52.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.498+0000 7f3e598be700 1 --2- 172.21.15.2:0/2240837081 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3e3806a0e0 0x7f3e3806c5a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3e5414d070 tx=0x7f3e50009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:52.505 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.502+0000 7f3e467fc700 1 -- 172.21.15.2:0/2240837081 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3e4c0544a0 con 0x7f3e5415b460 2023-08-25T03:34:52.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.748+0000 7f3e5bb22700 1 -- 172.21.15.2:0/2240837081 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e541f70b0 con 0x7f3e5415b460 2023-08-25T03:34:52.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.748+0000 7f3e467fc700 1 -- 172.21.15.2:0/2240837081 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4857 (secure 0 0 0) 0x7f3e4c057ac0 con 0x7f3e5415b460 2023-08-25T03:34:52.749 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:52.750 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2074312983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2074312983},{"type":"v1","addr":"172.21.15.2:6837","nonce":2074312983}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4059180922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4059180922},{"type":"v1","addr":"172.21.15.31:6835","nonce":4059180922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/922384230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":922384230},{"type":"v1","addr":"172.21.15.31:6834","nonce":922384230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T03:34:03.009109+0000","modified":"2023-08-25T03:34:06.613200+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.2:6835/193990892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":193990892},{"type":"v1","addr":"172.21.15.2:6835","nonce":193990892}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T03:34:52.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.750+0000 7f3e5bb22700 1 -- 172.21.15.2:0/2240837081 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3e3806a0e0 msgr2=0x7f3e3806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:52.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.750+0000 7f3e5bb22700 1 --2- 172.21.15.2:0/2240837081 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3e3806a0e0 0x7f3e3806c5a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3e5414d070 tx=0x7f3e50009040 comp rx=0 tx=0).stop 2023-08-25T03:34:52.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.750+0000 7f3e5bb22700 1 -- 172.21.15.2:0/2240837081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e5415b460 msgr2=0x7f3e5406a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:52.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.750+0000 7f3e5bb22700 1 --2- 172.21.15.2:0/2240837081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e5415b460 0x7f3e5406a1e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f3e540b60d0 tx=0x7f3e4c011f80 comp rx=0 tx=0).stop 2023-08-25T03:34:52.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.750+0000 7f3e5bb22700 1 -- 172.21.15.2:0/2240837081 shutdown_connections 2023-08-25T03:34:52.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.750+0000 7f3e5bb22700 1 --2- 172.21.15.2:0/2240837081 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3e3806a0e0 0x7f3e3806c5a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:52.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.750+0000 7f3e5bb22700 1 --2- 172.21.15.2:0/2240837081 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3e5415b460 0x7f3e5406a1e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:52.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.750+0000 7f3e5bb22700 1 --2- 172.21.15.2:0/2240837081 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e40004120 0x7f3e5414d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:52.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.750+0000 7f3e5bb22700 1 --2- 172.21.15.2:0/2240837081 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e541f6140 0x7f3e5414b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:52.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.750+0000 7f3e5bb22700 1 -- 172.21.15.2:0/2240837081 >> 172.21.15.2:0/2240837081 conn(0x7f3e540b1cf0 msgr2=0x7f3e5414e600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:52.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.750+0000 7f3e5bb22700 1 -- 172.21.15.2:0/2240837081 shutdown_connections 2023-08-25T03:34:52.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.750+0000 7f3e5bb22700 1 -- 172.21.15.2:0/2240837081 wait complete. 2023-08-25T03:34:52.753 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 8 2023-08-25T03:34:52.763 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:34:52.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.887+0000 7f97110b2700 1 Processor -- start 2023-08-25T03:34:52.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.887+0000 7f97110b2700 1 -- start start 2023-08-25T03:34:52.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.888+0000 7f97110b2700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f970c06b770 0x7f970c06bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:52.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.888+0000 7f97110b2700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f970c0b1be0 0x7f970c0b2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:52.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.888+0000 7f97110b2700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f970c0d0560 con 0x7f970c0cca80 2023-08-25T03:34:52.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.888+0000 7f97110b2700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f970c06c090 con 0x7f970c0b1be0 2023-08-25T03:34:52.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.888+0000 7f97110b2700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f970c06c280 con 0x7f970c06b770 2023-08-25T03:34:52.890 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.888+0000 7f970a59c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f970c0b1be0 0x7f970c0b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:52.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.888+0000 7f970a59c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f970c0b1be0 0x7f970c0b2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60862/0 (socket says 172.21.15.2:60862) 2023-08-25T03:34:52.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.889+0000 7f970a59c700 1 -- 172.21.15.2:0/2871061945 learned_addr learned my addr 172.21.15.2:0/2871061945 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:52.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.889+0000 7f970ad9d700 1 --2- 172.21.15.2:0/2871061945 >> v2:172.21.15.31:3300/0 conn(0x7f970c06b770 0x7f970c06bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:52.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.889+0000 7f970a59c700 1 -- 172.21.15.2:0/2871061945 >> v2:172.21.15.31:3300/0 conn(0x7f970c06b770 msgr2=0x7f970c06bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:52.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.889+0000 7f970a59c700 1 --2- 172.21.15.2:0/2871061945 >> v2:172.21.15.31:3300/0 conn(0x7f970c06b770 0x7f970c06bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:52.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.889+0000 7f970a59c700 1 -- 172.21.15.2:0/2871061945 >> v1:172.21.15.31:6789/0 conn(0x7f970c0cca80 legacy=0x7f970c0cd7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:52.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.889+0000 7f970a59c700 1 -- 172.21.15.2:0/2871061945 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970c06c560 con 0x7f970c0b1be0 2023-08-25T03:34:52.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.889+0000 7f970a59c700 1 --2- 172.21.15.2:0/2871061945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f970c0b1be0 0x7f970c0b2050 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f970c071640 tx=0x7f96f4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29104ebf37032db server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:52.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.889+0000 7f9709d9b700 1 -- 172.21.15.2:0/2871061945 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f4004230 con 0x7f970c0b1be0 2023-08-25T03:34:52.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.889+0000 7f9709d9b700 1 -- 172.21.15.2:0/2871061945 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f4004390 con 0x7f970c0b1be0 2023-08-25T03:34:52.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.889+0000 7f9709d9b700 1 -- 172.21.15.2:0/2871061945 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f40036a0 con 0x7f970c0b1be0 2023-08-25T03:34:52.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.890+0000 7f97110b2700 1 -- 172.21.15.2:0/2871061945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f970c0b1be0 msgr2=0x7f970c0b2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:52.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.890+0000 7f97110b2700 1 --2- 172.21.15.2:0/2871061945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f970c0b1be0 0x7f970c0b2050 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f970c071640 tx=0x7f96f4009bd0 comp rx=0 tx=0).stop 2023-08-25T03:34:52.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.890+0000 7f97110b2700 1 -- 172.21.15.2:0/2871061945 shutdown_connections 2023-08-25T03:34:52.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.890+0000 7f97110b2700 1 --2- 172.21.15.2:0/2871061945 >> v2:172.21.15.31:3300/0 conn(0x7f970c06b770 0x7f970c06bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:52.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.890+0000 7f97110b2700 1 --2- 172.21.15.2:0/2871061945 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f970c0b1be0 0x7f970c0b2050 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:52.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.890+0000 7f97110b2700 1 -- 172.21.15.2:0/2871061945 >> 172.21.15.2:0/2871061945 conn(0x7f970c09a2b0 msgr2=0x7f970c0d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:52.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.890+0000 7f97110b2700 1 -- 172.21.15.2:0/2871061945 shutdown_connections 2023-08-25T03:34:52.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.890+0000 7f97110b2700 1 -- 172.21.15.2:0/2871061945 wait complete. 2023-08-25T03:34:52.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.891+0000 7f97110b2700 1 Processor -- start 2023-08-25T03:34:52.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.891+0000 7f97110b2700 1 -- start start 2023-08-25T03:34:52.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.892+0000 7f97110b2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f970c06b770 0x7f970c14bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:52.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.892+0000 7f97110b2700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f970c0b1be0 0x7f970c0aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:52.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.892+0000 7f97110b2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f970c0cca80 0x7f970c14c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:52.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.892+0000 7f97110b2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970c0aea30 con 0x7f970c06b770 2023-08-25T03:34:52.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.892+0000 7f97110b2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970c0adba0 con 0x7f970c0cca80 2023-08-25T03:34:52.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.892+0000 7f97110b2700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f970c0b6a00 con 0x7f970c0b1be0 2023-08-25T03:34:52.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.892+0000 7f970ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f970c06b770 0x7f970c14bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:52.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.892+0000 7f970ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f970c06b770 0x7f970c14bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60868/0 (socket says 172.21.15.2:60868) 2023-08-25T03:34:52.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.892+0000 7f970ad9d700 1 -- 172.21.15.2:0/316729506 learned_addr learned my addr 172.21.15.2:0/316729506 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:52.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.892+0000 7f970b59e700 1 --2- 172.21.15.2:0/316729506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f970c0cca80 0x7f970c14c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:52.898 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.892+0000 7f970a59c700 1 --2- 172.21.15.2:0/316729506 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f970c0b1be0 0x7f970c0aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:52.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.892+0000 7f970ad9d700 1 -- 172.21.15.2:0/316729506 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f970c0b1be0 msgr2=0x7f970c0aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:52.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.892+0000 7f970ad9d700 1 --2- 172.21.15.2:0/316729506 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f970c0b1be0 0x7f970c0aee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:52.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.892+0000 7f970ad9d700 1 -- 172.21.15.2:0/316729506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f970c0cca80 msgr2=0x7f970c14c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:52.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.892+0000 7f970ad9d700 1 --2- 172.21.15.2:0/316729506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f970c0cca80 0x7f970c14c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:52.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.892+0000 7f970ad9d700 1 -- 172.21.15.2:0/316729506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f4009550 con 0x7f970c06b770 2023-08-25T03:34:52.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.893+0000 7f970a59c700 1 --2- 172.21.15.2:0/316729506 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f970c0b1be0 0x7f970c0aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:52.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.893+0000 7f970ad9d700 1 --2- 172.21.15.2:0/316729506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f970c06b770 0x7f970c14bd20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00e730 tx=0x7f96fc00eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:52.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.893+0000 7f97037fe700 1 -- 172.21.15.2:0/316729506 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc013610 con 0x7f970c06b770 2023-08-25T03:34:52.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.893+0000 7f97037fe700 1 -- 172.21.15.2:0/316729506 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96fc013770 con 0x7f970c06b770 2023-08-25T03:34:52.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.893+0000 7f97110b2700 1 -- 172.21.15.2:0/316729506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f970c14c260 con 0x7f970c06b770 2023-08-25T03:34:52.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.893+0000 7f97037fe700 1 -- 172.21.15.2:0/316729506 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc013b20 con 0x7f970c06b770 2023-08-25T03:34:52.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.893+0000 7f97110b2700 1 -- 172.21.15.2:0/316729506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f970c0ae090 con 0x7f970c06b770 2023-08-25T03:34:52.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.894+0000 7f97110b2700 1 -- 172.21.15.2:0/316729506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f970c0746d0 con 0x7f970c06b770 2023-08-25T03:34:52.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.894+0000 7f97037fe700 1 -- 172.21.15.2:0/316729506 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f96fc013c80 con 0x7f970c06b770 2023-08-25T03:34:52.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.895+0000 7f97037fe700 1 --2- 172.21.15.2:0/316729506 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f96ec06a0e0 0x7f96ec06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:52.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.895+0000 7f97037fe700 1 -- 172.21.15.2:0/316729506 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f96fc087eb0 con 0x7f970c06b770 2023-08-25T03:34:52.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.895+0000 7f970a59c700 1 --2- 172.21.15.2:0/316729506 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f96ec06a0e0 0x7f96ec06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:52.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.895+0000 7f970a59c700 1 --2- 172.21.15.2:0/316729506 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f96ec06a0e0 0x7f96ec06c5a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f970c1f5b00 tx=0x7f96f4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:52.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:52.899+0000 7f97037fe700 1 -- 172.21.15.2:0/316729506 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f96fc056ec0 con 0x7f970c06b770 2023-08-25T03:34:53.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.101+0000 7f97110b2700 1 -- 172.21.15.2:0/316729506 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f970c14c260 con 0x7f970c06b770 2023-08-25T03:34:53.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.102+0000 7f97037fe700 1 -- 172.21.15.2:0/316729506 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18230 (secure 0 0 0) 0x7f970c14c260 con 0x7f970c06b770 2023-08-25T03:34:53.103 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:53.104 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:34:44.588865+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:34:00.869118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T03:34:01.987016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.f","mappings":[{"from":1,"to":5}]},{"pgid":"4.2b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:34:53.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.104+0000 7f97110b2700 1 -- 172.21.15.2:0/316729506 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f96ec06a0e0 msgr2=0x7f96ec06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:53.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.104+0000 7f97110b2700 1 --2- 172.21.15.2:0/316729506 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f96ec06a0e0 0x7f96ec06c5a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f970c1f5b00 tx=0x7f96f4008030 comp rx=0 tx=0).stop 2023-08-25T03:34:53.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.104+0000 7f97110b2700 1 -- 172.21.15.2:0/316729506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f970c06b770 msgr2=0x7f970c14bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:53.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.104+0000 7f97110b2700 1 --2- 172.21.15.2:0/316729506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f970c06b770 0x7f970c14bd20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f96fc00e730 tx=0x7f96fc00eaf0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.105+0000 7f97110b2700 1 -- 172.21.15.2:0/316729506 shutdown_connections 2023-08-25T03:34:53.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.105+0000 7f97110b2700 1 --2- 172.21.15.2:0/316729506 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f96ec06a0e0 0x7f96ec06c5a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.105+0000 7f97110b2700 1 --2- 172.21.15.2:0/316729506 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f970c06b770 0x7f970c14bd20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.105+0000 7f97110b2700 1 --2- 172.21.15.2:0/316729506 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f970c0cca80 0x7f970c14c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.105+0000 7f97110b2700 1 --2- 172.21.15.2:0/316729506 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f970c0b1be0 0x7f970c0aee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.105+0000 7f97110b2700 1 -- 172.21.15.2:0/316729506 >> 172.21.15.2:0/316729506 conn(0x7f970c09a2b0 msgr2=0x7f970c160690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:53.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.105+0000 7f97110b2700 1 -- 172.21.15.2:0/316729506 shutdown_connections 2023-08-25T03:34:53.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.105+0000 7f97110b2700 1 -- 172.21.15.2:0/316729506 wait complete. 2023-08-25T03:34:53.118 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T03:33:49.560643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T03:33:56.265184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T03:34:00.869118+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T03:34:01.987016+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T03:34:53.118 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T03:34:53.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.245+0000 7f53c3fcd700 1 Processor -- start 2023-08-25T03:34:53.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.246+0000 7f53c3fcd700 1 -- start start 2023-08-25T03:34:53.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.246+0000 7f53c3fcd700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53bc0af180 0x7f53bc070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:53.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.246+0000 7f53c3fcd700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53bc160bd0 con 0x7f53bc1607f0 2023-08-25T03:34:53.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.246+0000 7f53c3fcd700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53bc0cdd00 con 0x7f53bc06a1e0 2023-08-25T03:34:53.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.246+0000 7f53c3fcd700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc0713a0 con 0x7f53bc0af180 2023-08-25T03:34:53.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.247+0000 7f53c1d69700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f53bc1607f0 0x7f53bc0ac8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47542/0 (socket says 172.21.15.2:47542) 2023-08-25T03:34:53.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.247+0000 7f53c1d69700 1 -- 172.21.15.2:0/2861745248 learned_addr learned my addr 172.21.15.2:0/2861745248 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:53.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.247+0000 7f53c1568700 1 --2- 172.21.15.2:0/2861745248 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53bc0af180 0x7f53bc070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:53.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.247+0000 7f53c0d67700 1 -- 172.21.15.2:0/2861745248 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 903006252 0 0) 0x7f53bc160bd0 con 0x7f53bc1607f0 2023-08-25T03:34:53.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.247+0000 7f53c0d67700 1 -- 172.21.15.2:0/2861745248 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53a80035b0 con 0x7f53bc1607f0 2023-08-25T03:34:53.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.247+0000 7f53c0d67700 1 -- 172.21.15.2:0/2861745248 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3875472188 0 0) 0x7f53bc0cdd00 con 0x7f53bc06a1e0 2023-08-25T03:34:53.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.247+0000 7f53c0d67700 1 -- 172.21.15.2:0/2861745248 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53bc160bd0 con 0x7f53bc06a1e0 2023-08-25T03:34:53.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.247+0000 7f53c0d67700 1 -- 172.21.15.2:0/2861745248 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 751131023 0 0) 0x7f53a80035b0 con 0x7f53bc1607f0 2023-08-25T03:34:53.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.247+0000 7f53c0d67700 1 -- 172.21.15.2:0/2861745248 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53bc0af180 msgr2=0x7f53bc070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:53.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.247+0000 7f53c0d67700 1 --2- 172.21.15.2:0/2861745248 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53bc0af180 0x7f53bc070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.247+0000 7f53c0d67700 1 -- 172.21.15.2:0/2861745248 >> v1:172.21.15.31:6789/0 conn(0x7f53bc06a1e0 legacy=0x7f53bc06ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:53.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.247+0000 7f53c0d67700 1 -- 172.21.15.2:0/2861745248 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53bc0cda00 con 0x7f53bc1607f0 2023-08-25T03:34:53.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.247+0000 7f53c0d67700 1 -- 172.21.15.2:0/2861745248 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f53b8001670 con 0x7f53bc1607f0 2023-08-25T03:34:53.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.247+0000 7f53c0d67700 1 -- 172.21.15.2:0/2861745248 >> v1:172.21.15.2:6789/0 conn(0x7f53bc1607f0 legacy=0x7f53bc0ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:53.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.247+0000 7f53c0d67700 1 --2- 172.21.15.2:0/2861745248 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53bc0af180 0x7f53a8003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:53.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.247+0000 7f53c0d67700 1 --2- 172.21.15.2:0/2861745248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53a8004120 0x7f53a8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.247+0000 7f53c0d67700 1 --2- 172.21.15.2:0/2861745248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53a8004d90 0x7f53a8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.247+0000 7f53c0d67700 1 -- 172.21.15.2:0/2861745248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a800bac0 con 0x7f53a8004120 2023-08-25T03:34:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.247+0000 7f53c0d67700 1 -- 172.21.15.2:0/2861745248 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a800bc30 con 0x7f53a8004d90 2023-08-25T03:34:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.247+0000 7f53c0d67700 1 -- 172.21.15.2:0/2861745248 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a800bdf0 con 0x7f53bc0af180 2023-08-25T03:34:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.247+0000 7f53c1568700 1 --2- 172.21.15.2:0/2861745248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53a8004120 0x7f53a8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.247+0000 7f53c1d69700 1 --2- 172.21.15.2:0/2861745248 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53bc0af180 0x7f53a8003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.248+0000 7f53c256a700 1 --2- 172.21.15.2:0/2861745248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53a8004d90 0x7f53a8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:53.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.248+0000 7f53c1568700 1 -- 172.21.15.2:0/2861745248 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53bc0af180 msgr2=0x7f53a8003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.248+0000 7f53c1568700 1 --2- 172.21.15.2:0/2861745248 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53bc0af180 0x7f53a8003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.248+0000 7f53c1568700 1 -- 172.21.15.2:0/2861745248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53a8004d90 msgr2=0x7f53a8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.248+0000 7f53c1568700 1 --2- 172.21.15.2:0/2861745248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53a8004d90 0x7f53a8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.248+0000 7f53c1568700 1 -- 172.21.15.2:0/2861745248 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a80035b0 con 0x7f53a8004120 2023-08-25T03:34:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.248+0000 7f53c1568700 1 --2- 172.21.15.2:0/2861745248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53a8004120 0x7f53a8004850 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f53b0009b60 tx=0x7f53a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4123dbba2e034b8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.248+0000 7f53c0d67700 1 -- 172.21.15.2:0/2861745248 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b000e6c0 con 0x7f53a8004120 2023-08-25T03:34:53.254 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.248+0000 7f53c0d67700 1 -- 172.21.15.2:0/2861745248 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b000e820 con 0x7f53a8004120 2023-08-25T03:34:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.248+0000 7f53c0d67700 1 -- 172.21.15.2:0/2861745248 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b000eb90 con 0x7f53a8004120 2023-08-25T03:34:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.249+0000 7f53c3fcd700 1 -- 172.21.15.2:0/2861745248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53a8004120 msgr2=0x7f53a8004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.249+0000 7f53c3fcd700 1 --2- 172.21.15.2:0/2861745248 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53a8004120 0x7f53a8004850 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f53b0009b60 tx=0x7f53a8005880 comp rx=0 tx=0).stop 2023-08-25T03:34:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.249+0000 7f53c256a700 1 -- 172.21.15.2:0/2861745248 reap_dead start 2023-08-25T03:34:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.249+0000 7f53c3fcd700 1 -- 172.21.15.2:0/2861745248 shutdown_connections 2023-08-25T03:34:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.249+0000 7f53c3fcd700 1 -- 172.21.15.2:0/2861745248 >> 172.21.15.2:0/2861745248 conn(0x7f53bc0cfd90 msgr2=0x7f53bc0d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.249+0000 7f53c3fcd700 1 -- 172.21.15.2:0/2861745248 shutdown_connections 2023-08-25T03:34:53.255 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.249+0000 7f53c3fcd700 1 -- 172.21.15.2:0/2861745248 wait complete. 2023-08-25T03:34:53.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.250+0000 7f53c3fcd700 1 Processor -- start 2023-08-25T03:34:53.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.250+0000 7f53c3fcd700 1 -- start start 2023-08-25T03:34:53.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.251+0000 7f53c3fcd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53a8004120 0x7f53bc0ca270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:53.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.251+0000 7f53c3fcd700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53bc0c8a60 0x7f53bc15b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:53.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.251+0000 7f53c3fcd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53bc0c72f0 0x7f53bc0c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:53.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.251+0000 7f53c3fcd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc0c6e60 con 0x7f53bc0c72f0 2023-08-25T03:34:53.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.251+0000 7f53c3fcd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc0c8ee0 con 0x7f53a8004120 2023-08-25T03:34:53.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.251+0000 7f53c3fcd700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53bc0ca970 con 0x7f53bc0c8a60 2023-08-25T03:34:53.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.251+0000 7f53c256a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53bc0c72f0 0x7f53bc0c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:53.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.251+0000 7f53c256a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53bc0c72f0 0x7f53bc0c77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60892/0 (socket says 172.21.15.2:60892) 2023-08-25T03:34:53.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.251+0000 7f53c256a700 1 -- 172.21.15.2:0/140574530 learned_addr learned my addr 172.21.15.2:0/140574530 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:53.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.251+0000 7f53c1568700 1 --2- 172.21.15.2:0/140574530 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53bc0c8a60 0x7f53bc15b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:53.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.251+0000 7f53c1d69700 1 --2- 172.21.15.2:0/140574530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53a8004120 0x7f53bc0ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:53.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.251+0000 7f53c256a700 1 -- 172.21.15.2:0/140574530 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53bc0c8a60 msgr2=0x7f53bc15b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:53.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.251+0000 7f53c256a700 1 --2- 172.21.15.2:0/140574530 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53bc0c8a60 0x7f53bc15b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.251+0000 7f53c256a700 1 -- 172.21.15.2:0/140574530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53a8004120 msgr2=0x7f53bc0ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:53.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.251+0000 7f53c256a700 1 --2- 172.21.15.2:0/140574530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53a8004120 0x7f53bc0ca270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.251+0000 7f53c256a700 1 -- 172.21.15.2:0/140574530 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53bc0cac50 con 0x7f53bc0c72f0 2023-08-25T03:34:53.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.251+0000 7f53c256a700 1 --2- 172.21.15.2:0/140574530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53bc0c72f0 0x7f53bc0c77a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f53bc06cbd0 tx=0x7f53b400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:53.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.251+0000 7f53ae7fc700 1 -- 172.21.15.2:0/140574530 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b4003c60 con 0x7f53bc0c72f0 2023-08-25T03:34:53.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.251+0000 7f53ae7fc700 1 -- 172.21.15.2:0/140574530 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b4003dc0 con 0x7f53bc0c72f0 2023-08-25T03:34:53.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.251+0000 7f53ae7fc700 1 -- 172.21.15.2:0/140574530 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b400db20 con 0x7f53bc0c72f0 2023-08-25T03:34:53.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.251+0000 7f53c3fcd700 1 -- 172.21.15.2:0/140574530 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53bc0caf30 con 0x7f53bc0c72f0 2023-08-25T03:34:53.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.252+0000 7f53c3fcd700 1 -- 172.21.15.2:0/140574530 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53bc15b9a0 con 0x7f53bc0c72f0 2023-08-25T03:34:53.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.252+0000 7f53c3fcd700 1 -- 172.21.15.2:0/140574530 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53bc15bb30 con 0x7f53bc0c72f0 2023-08-25T03:34:53.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.253+0000 7f53ae7fc700 1 -- 172.21.15.2:0/140574530 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f53b400dc80 con 0x7f53bc0c72f0 2023-08-25T03:34:53.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.253+0000 7f53ae7fc700 1 --2- 172.21.15.2:0/140574530 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f53a006a0e0 0x7f53a006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:53.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.253+0000 7f53ae7fc700 1 -- 172.21.15.2:0/140574530 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f53b4088740 con 0x7f53bc0c72f0 2023-08-25T03:34:53.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.253+0000 7f53c1d69700 1 --2- 172.21.15.2:0/140574530 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f53a006a0e0 0x7f53a006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:53.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.254+0000 7f53c1d69700 1 --2- 172.21.15.2:0/140574530 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f53a006a0e0 0x7f53a006c5a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f53b8000c00 tx=0x7f53b8001e20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:53.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.258+0000 7f53ae7fc700 1 -- 172.21.15.2:0/140574530 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f53b4053df0 con 0x7f53bc0c72f0 2023-08-25T03:34:53.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.457+0000 7f53c3fcd700 1 -- 172.21.15.2:0/140574530 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f53bc0746d0 con 0x7f53bc0c72f0 2023-08-25T03:34:53.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.457+0000 7f53ae7fc700 1 -- 172.21.15.2:0/140574530 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f53b4057410 con 0x7f53bc0c72f0 2023-08-25T03:34:53.458 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-08-25T03:34:53.460 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.459+0000 7f53c3fcd700 1 -- 172.21.15.2:0/140574530 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f53a006a0e0 msgr2=0x7f53a006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:53.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.459+0000 7f53c3fcd700 1 --2- 172.21.15.2:0/140574530 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f53a006a0e0 0x7f53a006c5a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f53b8000c00 tx=0x7f53b8001e20 comp rx=0 tx=0).stop 2023-08-25T03:34:53.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.459+0000 7f53c3fcd700 1 -- 172.21.15.2:0/140574530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53bc0c72f0 msgr2=0x7f53bc0c77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:53.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.459+0000 7f53c3fcd700 1 --2- 172.21.15.2:0/140574530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53bc0c72f0 0x7f53bc0c77a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f53bc06cbd0 tx=0x7f53b400d430 comp rx=0 tx=0).stop 2023-08-25T03:34:53.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.459+0000 7f53c3fcd700 1 -- 172.21.15.2:0/140574530 shutdown_connections 2023-08-25T03:34:53.461 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.459+0000 7f53c3fcd700 1 --2- 172.21.15.2:0/140574530 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f53a006a0e0 0x7f53a006c5a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.459+0000 7f53c3fcd700 1 --2- 172.21.15.2:0/140574530 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53bc0c72f0 0x7f53bc0c77a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.459+0000 7f53c3fcd700 1 --2- 172.21.15.2:0/140574530 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53a8004120 0x7f53bc0ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.459+0000 7f53c3fcd700 1 --2- 172.21.15.2:0/140574530 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53bc0c8a60 0x7f53bc15b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.459+0000 7f53c3fcd700 1 -- 172.21.15.2:0/140574530 >> 172.21.15.2:0/140574530 conn(0x7f53bc0cfd90 msgr2=0x7f53bc15de50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:53.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.460+0000 7f53c3fcd700 1 -- 172.21.15.2:0/140574530 shutdown_connections 2023-08-25T03:34:53.462 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.460+0000 7f53c3fcd700 1 -- 172.21.15.2:0/140574530 wait complete. 2023-08-25T03:34:53.473 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T03:34:53.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.602+0000 7f94741bd700 1 Processor -- start 2023-08-25T03:34:53.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.603+0000 7f94741bd700 1 -- start start 2023-08-25T03:34:53.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.603+0000 7f94741bd700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f946c0cacf0 0x7f946c0cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:53.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.603+0000 7f94741bd700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946c06eaf0 con 0x7f946c159f10 2023-08-25T03:34:53.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.603+0000 7f94741bd700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f946c15e2c0 con 0x7f946c0cedf0 2023-08-25T03:34:53.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.603+0000 7f94741bd700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946c06e850 con 0x7f946c0cacf0 2023-08-25T03:34:53.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.604+0000 7f9471f59700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f946c159f10 0x7f946c15dae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47562/0 (socket says 172.21.15.2:47562) 2023-08-25T03:34:53.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.604+0000 7f9471f59700 1 -- 172.21.15.2:0/1208331107 learned_addr learned my addr 172.21.15.2:0/1208331107 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:53.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.604+0000 7f947275a700 1 --2- 172.21.15.2:0/1208331107 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f946c0cacf0 0x7f946c0cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:53.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.604+0000 7f9470f57700 1 -- 172.21.15.2:0/1208331107 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1970984305 0 0) 0x7f946c06eaf0 con 0x7f946c159f10 2023-08-25T03:34:53.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.604+0000 7f9470f57700 1 -- 172.21.15.2:0/1208331107 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94580035b0 con 0x7f946c159f10 2023-08-25T03:34:53.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.604+0000 7f9470f57700 1 -- 172.21.15.2:0/1208331107 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3565628455 0 0) 0x7f946c15e2c0 con 0x7f946c0cedf0 2023-08-25T03:34:53.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.604+0000 7f9470f57700 1 -- 172.21.15.2:0/1208331107 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f946c06eaf0 con 0x7f946c0cedf0 2023-08-25T03:34:53.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.604+0000 7f947275a700 1 -- 172.21.15.2:0/1208331107 >> v1:172.21.15.31:6789/0 conn(0x7f946c0cedf0 legacy=0x7f946c06ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:53.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.604+0000 7f947275a700 1 -- 172.21.15.2:0/1208331107 >> v1:172.21.15.2:6789/0 conn(0x7f946c159f10 legacy=0x7f946c15dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:53.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.604+0000 7f947275a700 1 -- 172.21.15.2:0/1208331107 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946c15e5a0 con 0x7f946c0cacf0 2023-08-25T03:34:53.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.604+0000 7f947275a700 1 --2- 172.21.15.2:0/1208331107 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f946c0cacf0 0x7f946c0cb1a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9464009dc0 tx=0x7f946400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=561d751b77724b61 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:53.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.604+0000 7f9470f57700 1 -- 172.21.15.2:0/1208331107 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946400e070 con 0x7f946c0cacf0 2023-08-25T03:34:53.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.605+0000 7f9470f57700 1 -- 172.21.15.2:0/1208331107 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946400bb20 con 0x7f946c0cacf0 2023-08-25T03:34:53.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.605+0000 7f9470f57700 1 -- 172.21.15.2:0/1208331107 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9464012430 con 0x7f946c0cacf0 2023-08-25T03:34:53.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.605+0000 7f94741bd700 1 -- 172.21.15.2:0/1208331107 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f946c0cacf0 msgr2=0x7f946c0cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:53.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.605+0000 7f94741bd700 1 --2- 172.21.15.2:0/1208331107 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f946c0cacf0 0x7f946c0cb1a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9464009dc0 tx=0x7f946400b330 comp rx=0 tx=0).stop 2023-08-25T03:34:53.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.605+0000 7f94741bd700 1 -- 172.21.15.2:0/1208331107 shutdown_connections 2023-08-25T03:34:53.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.605+0000 7f94741bd700 1 --2- 172.21.15.2:0/1208331107 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f946c0cacf0 0x7f946c0cb1a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.605+0000 7f94741bd700 1 -- 172.21.15.2:0/1208331107 >> 172.21.15.2:0/1208331107 conn(0x7f946c0cfd90 msgr2=0x7f946c0d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:53.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.605+0000 7f94741bd700 1 -- 172.21.15.2:0/1208331107 shutdown_connections 2023-08-25T03:34:53.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.605+0000 7f94741bd700 1 -- 172.21.15.2:0/1208331107 wait complete. 2023-08-25T03:34:53.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.606+0000 7f94741bd700 1 Processor -- start 2023-08-25T03:34:53.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.606+0000 7f94741bd700 1 -- start start 2023-08-25T03:34:53.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.607+0000 7f94741bd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f946c0cacf0 0x7f946c14f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:53.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.607+0000 7f94741bd700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f946c0cedf0 0x7f946c0c27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:53.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.607+0000 7f94741bd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f946c159f10 0x7f946c070e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:53.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.607+0000 7f94741bd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c15ba70 con 0x7f946c0cacf0 2023-08-25T03:34:53.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.607+0000 7f94741bd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946c150a60 con 0x7f946c159f10 2023-08-25T03:34:53.614 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.607+0000 7f94741bd700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946c15bcc0 con 0x7f946c0cedf0 2023-08-25T03:34:53.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.607+0000 7f9471f59700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f946c0cacf0 0x7f946c14f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:53.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.607+0000 7f9471f59700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f946c0cacf0 0x7f946c14f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60902/0 (socket says 172.21.15.2:60902) 2023-08-25T03:34:53.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.607+0000 7f9471f59700 1 -- 172.21.15.2:0/1433253891 learned_addr learned my addr 172.21.15.2:0/1433253891 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:53.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.607+0000 7f9471758700 1 --2- 172.21.15.2:0/1433253891 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f946c0cedf0 0x7f946c0c27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:53.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.607+0000 7f947275a700 1 --2- 172.21.15.2:0/1433253891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f946c159f10 0x7f946c070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:53.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.608+0000 7f9471f59700 1 -- 172.21.15.2:0/1433253891 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f946c0cedf0 msgr2=0x7f946c0c27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:53.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.608+0000 7f9471f59700 1 --2- 172.21.15.2:0/1433253891 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f946c0cedf0 0x7f946c0c27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.608+0000 7f9471f59700 1 -- 172.21.15.2:0/1433253891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f946c159f10 msgr2=0x7f946c070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:53.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.608+0000 7f9471f59700 1 --2- 172.21.15.2:0/1433253891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f946c159f10 0x7f946c070e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.608+0000 7f9471f59700 1 -- 172.21.15.2:0/1433253891 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9464009a70 con 0x7f946c0cacf0 2023-08-25T03:34:53.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.608+0000 7f9471f59700 1 --2- 172.21.15.2:0/1433253891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f946c0cacf0 0x7f946c14f290 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9460001eb0 tx=0x7f946000c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:53.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.608+0000 7f945e7fc700 1 -- 172.21.15.2:0/1433253891 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9460005d70 con 0x7f946c0cacf0 2023-08-25T03:34:53.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.608+0000 7f945e7fc700 1 -- 172.21.15.2:0/1433253891 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94600070d0 con 0x7f946c0cacf0 2023-08-25T03:34:53.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.608+0000 7f94741bd700 1 -- 172.21.15.2:0/1433253891 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f946c15bfa0 con 0x7f946c0cacf0 2023-08-25T03:34:53.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.608+0000 7f945e7fc700 1 -- 172.21.15.2:0/1433253891 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9460007460 con 0x7f946c0cacf0 2023-08-25T03:34:53.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.608+0000 7f94741bd700 1 -- 172.21.15.2:0/1433253891 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f946c15c560 con 0x7f946c0cacf0 2023-08-25T03:34:53.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.609+0000 7f94741bd700 1 -- 172.21.15.2:0/1433253891 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f946c0746d0 con 0x7f946c0cacf0 2023-08-25T03:34:53.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.609+0000 7f945e7fc700 1 -- 172.21.15.2:0/1433253891 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f94600075c0 con 0x7f946c0cacf0 2023-08-25T03:34:53.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.610+0000 7f945e7fc700 1 --2- 172.21.15.2:0/1433253891 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f945006a0e0 0x7f945006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:53.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.610+0000 7f945e7fc700 1 -- 172.21.15.2:0/1433253891 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f9460087930 con 0x7f946c0cacf0 2023-08-25T03:34:53.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.610+0000 7f9471758700 1 --2- 172.21.15.2:0/1433253891 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f945006a0e0 0x7f945006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:53.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.610+0000 7f9471758700 1 --2- 172.21.15.2:0/1433253891 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f945006a0e0 0x7f945006c5a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f946c14fe20 tx=0x7f9468009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:53.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.614+0000 7f945e7fc700 1 -- 172.21.15.2:0/1433253891 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9460053070 con 0x7f946c0cacf0 2023-08-25T03:34:53.817 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.816+0000 7f94741bd700 1 -- 172.21.15.2:0/1433253891 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f946c15e880 con 0x7f946c0cacf0 2023-08-25T03:34:53.818 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.817+0000 7f945e7fc700 1 -- 172.21.15.2:0/1433253891 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v26) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9460056690 con 0x7f946c0cacf0 2023-08-25T03:34:53.818 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 8 2023-08-25T03:34:53.820 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.819+0000 7f94741bd700 1 -- 172.21.15.2:0/1433253891 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f945006a0e0 msgr2=0x7f945006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:53.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.819+0000 7f94741bd700 1 --2- 172.21.15.2:0/1433253891 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f945006a0e0 0x7f945006c5a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f946c14fe20 tx=0x7f9468009040 comp rx=0 tx=0).stop 2023-08-25T03:34:53.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.819+0000 7f94741bd700 1 -- 172.21.15.2:0/1433253891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f946c0cacf0 msgr2=0x7f946c14f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:53.821 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.819+0000 7f94741bd700 1 --2- 172.21.15.2:0/1433253891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f946c0cacf0 0x7f946c14f290 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9460001eb0 tx=0x7f946000c7b0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.819+0000 7f94741bd700 1 -- 172.21.15.2:0/1433253891 shutdown_connections 2023-08-25T03:34:53.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.819+0000 7f94741bd700 1 --2- 172.21.15.2:0/1433253891 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f945006a0e0 0x7f945006c5a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.819+0000 7f94741bd700 1 --2- 172.21.15.2:0/1433253891 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f946c0cacf0 0x7f946c14f290 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.819+0000 7f94741bd700 1 --2- 172.21.15.2:0/1433253891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f946c159f10 0x7f946c070e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.819+0000 7f94741bd700 1 --2- 172.21.15.2:0/1433253891 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f946c0cedf0 0x7f946c0c27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.819+0000 7f94741bd700 1 -- 172.21.15.2:0/1433253891 >> 172.21.15.2:0/1433253891 conn(0x7f946c0cfd90 msgr2=0x7f946c0af7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:53.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.819+0000 7f94741bd700 1 -- 172.21.15.2:0/1433253891 shutdown_connections 2023-08-25T03:34:53.824 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.819+0000 7f94741bd700 1 -- 172.21.15.2:0/1433253891 wait complete. 2023-08-25T03:34:53.835 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T03:34:53.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.963+0000 7f1810310700 1 Processor -- start 2023-08-25T03:34:53.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.963+0000 7f1810310700 1 -- start start 2023-08-25T03:34:53.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.964+0000 7f1810310700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18081607f0 0x7f1808070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:53.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.964+0000 7f1810310700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f180806a1e0 0x7f180806a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:53.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.964+0000 7f1810310700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f180806ab70 con 0x7f18080af180 2023-08-25T03:34:53.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.964+0000 7f1810310700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18080713a0 con 0x7f180806a1e0 2023-08-25T03:34:53.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.964+0000 7f1810310700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18080718a0 con 0x7f18081607f0 2023-08-25T03:34:53.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.964+0000 7f180d8ab700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f18080af180 0x7f18080ac8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47576/0 (socket says 172.21.15.2:47576) 2023-08-25T03:34:53.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.964+0000 7f180d8ab700 1 -- 172.21.15.2:0/2293814459 learned_addr learned my addr 172.21.15.2:0/2293814459 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:53.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.964+0000 7f180e8ad700 1 --2- 172.21.15.2:0/2293814459 >> v2:172.21.15.31:3300/0 conn(0x7f180806a1e0 0x7f180806a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:53.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.964+0000 7f180e0ac700 1 --2- 172.21.15.2:0/2293814459 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18081607f0 0x7f1808070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:53.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.965+0000 7f180d0aa700 1 -- 172.21.15.2:0/2293814459 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 999559602 0 0) 0x7f17fc001110 con 0x7f18080af180 2023-08-25T03:34:53.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.965+0000 7f180d0aa700 1 -- 172.21.15.2:0/2293814459 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17f40035b0 con 0x7f18080af180 2023-08-25T03:34:53.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.965+0000 7f180e8ad700 1 -- 172.21.15.2:0/2293814459 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18081607f0 msgr2=0x7f1808070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:53.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.965+0000 7f180e8ad700 1 --2- 172.21.15.2:0/2293814459 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18081607f0 0x7f1808070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.965+0000 7f180e8ad700 1 -- 172.21.15.2:0/2293814459 >> v1:172.21.15.2:6789/0 conn(0x7f18080af180 legacy=0x7f18080ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:53.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.965+0000 7f180e8ad700 1 -- 172.21.15.2:0/2293814459 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1808160bd0 con 0x7f180806a1e0 2023-08-25T03:34:53.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.965+0000 7f180e0ac700 1 --2- 172.21.15.2:0/2293814459 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18081607f0 0x7f1808070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:34:53.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.965+0000 7f180e8ad700 1 --2- 172.21.15.2:0/2293814459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f180806a1e0 0x7f180806a630 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1800009d90 tx=0x7f180000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eaab87c51c12d0cb server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:53.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.965+0000 7f180d0aa700 1 -- 172.21.15.2:0/2293814459 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180000e070 con 0x7f180806a1e0 2023-08-25T03:34:53.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.965+0000 7f180d0aa700 1 -- 172.21.15.2:0/2293814459 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180000bb20 con 0x7f180806a1e0 2023-08-25T03:34:53.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.965+0000 7f180d0aa700 1 -- 172.21.15.2:0/2293814459 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1800012430 con 0x7f180806a1e0 2023-08-25T03:34:53.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.966+0000 7f1810310700 1 -- 172.21.15.2:0/2293814459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f180806a1e0 msgr2=0x7f180806a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:53.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.966+0000 7f1810310700 1 --2- 172.21.15.2:0/2293814459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f180806a1e0 0x7f180806a630 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1800009d90 tx=0x7f180000b330 comp rx=0 tx=0).stop 2023-08-25T03:34:53.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.966+0000 7f1810310700 1 -- 172.21.15.2:0/2293814459 shutdown_connections 2023-08-25T03:34:53.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.966+0000 7f1810310700 1 --2- 172.21.15.2:0/2293814459 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f180806a1e0 0x7f180806a630 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.966+0000 7f1810310700 1 --2- 172.21.15.2:0/2293814459 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18081607f0 0x7f1808070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.966+0000 7f1810310700 1 -- 172.21.15.2:0/2293814459 >> 172.21.15.2:0/2293814459 conn(0x7f18080cfd90 msgr2=0x7f18080d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:53.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.966+0000 7f1810310700 1 -- 172.21.15.2:0/2293814459 shutdown_connections 2023-08-25T03:34:53.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.966+0000 7f1810310700 1 -- 172.21.15.2:0/2293814459 wait complete. 2023-08-25T03:34:53.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.967+0000 7f1810310700 1 Processor -- start 2023-08-25T03:34:53.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.967+0000 7f1810310700 1 -- start start 2023-08-25T03:34:53.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.968+0000 7f1810310700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f180806a1e0 0x7f18080c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:53.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.968+0000 7f1810310700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18080af180 0x7f18080c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:53.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.968+0000 7f1810310700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18081607f0 0x7f18080c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:53.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.968+0000 7f1810310700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180815bec0 con 0x7f18080af180 2023-08-25T03:34:53.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.968+0000 7f1810310700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18080c6690 con 0x7f180806a1e0 2023-08-25T03:34:53.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.968+0000 7f1810310700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1808071160 con 0x7f18081607f0 2023-08-25T03:34:53.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.968+0000 7f180d8ab700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18080af180 0x7f18080c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:53.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.968+0000 7f180d8ab700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18080af180 0x7f18080c5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60916/0 (socket says 172.21.15.2:60916) 2023-08-25T03:34:53.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.968+0000 7f180d8ab700 1 -- 172.21.15.2:0/175030381 learned_addr learned my addr 172.21.15.2:0/175030381 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:53.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.968+0000 7f180e0ac700 1 --2- 172.21.15.2:0/175030381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f180806a1e0 0x7f18080c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:53.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.968+0000 7f180e8ad700 1 --2- 172.21.15.2:0/175030381 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18081607f0 0x7f18080c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:53.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.968+0000 7f180d8ab700 1 -- 172.21.15.2:0/175030381 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18081607f0 msgr2=0x7f18080c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:53.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.968+0000 7f180d8ab700 1 --2- 172.21.15.2:0/175030381 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18081607f0 0x7f18080c6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.968+0000 7f180d8ab700 1 -- 172.21.15.2:0/175030381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f180806a1e0 msgr2=0x7f18080c95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:53.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.968+0000 7f180d8ab700 1 --2- 172.21.15.2:0/175030381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f180806a1e0 0x7f18080c95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:53.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.968+0000 7f180d8ab700 1 -- 172.21.15.2:0/175030381 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18040094a0 con 0x7f18080af180 2023-08-25T03:34:53.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.969+0000 7f180d8ab700 1 --2- 172.21.15.2:0/175030381 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18080af180 0x7f18080c5c10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f17fc007fd0 tx=0x7f17fc00ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:53.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.969+0000 7f17faffd700 1 -- 172.21.15.2:0/175030381 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc005d70 con 0x7f18080af180 2023-08-25T03:34:53.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.969+0000 7f17faffd700 1 -- 172.21.15.2:0/175030381 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17fc0070d0 con 0x7f18080af180 2023-08-25T03:34:53.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.969+0000 7f1810310700 1 -- 172.21.15.2:0/175030381 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1800009a70 con 0x7f18080af180 2023-08-25T03:34:53.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.969+0000 7f17faffd700 1 -- 172.21.15.2:0/175030381 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc007460 con 0x7f18080af180 2023-08-25T03:34:53.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.969+0000 7f1810310700 1 -- 172.21.15.2:0/175030381 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180815b520 con 0x7f18080af180 2023-08-25T03:34:53.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.970+0000 7f1810310700 1 -- 172.21.15.2:0/175030381 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18080746d0 con 0x7f18080af180 2023-08-25T03:34:53.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.971+0000 7f17faffd700 1 -- 172.21.15.2:0/175030381 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f17fc0075c0 con 0x7f18080af180 2023-08-25T03:34:53.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.971+0000 7f17faffd700 1 --2- 172.21.15.2:0/175030381 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f17ec06a0e0 0x7f17ec06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:53.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.971+0000 7f17faffd700 1 -- 172.21.15.2:0/175030381 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f17fc087d80 con 0x7f18080af180 2023-08-25T03:34:53.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.971+0000 7f180e0ac700 1 --2- 172.21.15.2:0/175030381 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f17ec06a0e0 0x7f17ec06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:53.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.971+0000 7f180e0ac700 1 --2- 172.21.15.2:0/175030381 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f17ec06a0e0 0x7f17ec06c5a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f180815d030 tx=0x7f1804008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:53.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:53.975+0000 7f17faffd700 1 -- 172.21.15.2:0/175030381 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f17fc053510 con 0x7f18080af180 2023-08-25T03:34:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.173+0000 7f1810310700 1 -- 172.21.15.2:0/175030381 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f180806ab70 con 0x7f18080af180 2023-08-25T03:34:54.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.173+0000 7f17faffd700 1 -- 172.21.15.2:0/175030381 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f180806ab70 con 0x7f18080af180 2023-08-25T03:34:54.175 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-08-25T03:34:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.175+0000 7f1810310700 1 -- 172.21.15.2:0/175030381 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f17ec06a0e0 msgr2=0x7f17ec06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:54.177 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.175+0000 7f1810310700 1 --2- 172.21.15.2:0/175030381 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f17ec06a0e0 0x7f17ec06c5a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f180815d030 tx=0x7f1804008040 comp rx=0 tx=0).stop 2023-08-25T03:34:54.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.175+0000 7f1810310700 1 -- 172.21.15.2:0/175030381 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18080af180 msgr2=0x7f18080c5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:54.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.175+0000 7f1810310700 1 --2- 172.21.15.2:0/175030381 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18080af180 0x7f18080c5c10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f17fc007fd0 tx=0x7f17fc00ca90 comp rx=0 tx=0).stop 2023-08-25T03:34:54.178 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.175+0000 7f1810310700 1 -- 172.21.15.2:0/175030381 shutdown_connections 2023-08-25T03:34:54.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.175+0000 7f1810310700 1 --2- 172.21.15.2:0/175030381 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18081607f0 0x7f18080c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:54.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.175+0000 7f1810310700 1 --2- 172.21.15.2:0/175030381 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f180806a1e0 0x7f18080c95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:54.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.175+0000 7f1810310700 1 --2- 172.21.15.2:0/175030381 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f17ec06a0e0 0x7f17ec06c5a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:54.179 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.175+0000 7f1810310700 1 --2- 172.21.15.2:0/175030381 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f18080af180 0x7f18080c5c10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:54.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.175+0000 7f1810310700 1 -- 172.21.15.2:0/175030381 >> 172.21.15.2:0/175030381 conn(0x7f18080cfd90 msgr2=0x7f180806eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:54.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.176+0000 7f1810310700 1 -- 172.21.15.2:0/175030381 shutdown_connections 2023-08-25T03:34:54.180 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.176+0000 7f1810310700 1 -- 172.21.15.2:0/175030381 wait complete. 2023-08-25T03:34:54.190 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T03:34:54.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.319+0000 7f7871d49700 1 Processor -- start 2023-08-25T03:34:54.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.319+0000 7f7871d49700 1 -- start start 2023-08-25T03:34:54.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.319+0000 7f7871d49700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f786c06a830 0x7f786c0d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:54.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.319+0000 7f7871d49700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f786c0cc470 con 0x7f786c071330 2023-08-25T03:34:54.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.319+0000 7f7871d49700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f786c0cbe90 con 0x7f786c06a450 2023-08-25T03:34:54.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.319+0000 7f7871d49700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f786c071740 con 0x7f786c06a830 2023-08-25T03:34:54.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.320+0000 7f786affd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f786c06a830 0x7f786c0d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:54.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.320+0000 7f786bfff700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f786c071330 0x7f786c06d610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47588/0 (socket says 172.21.15.2:47588) 2023-08-25T03:34:54.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.320+0000 7f786bfff700 1 -- 172.21.15.2:0/1513357374 learned_addr learned my addr 172.21.15.2:0/1513357374 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:54.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.320+0000 7f786affd700 1 -- 172.21.15.2:0/1513357374 >> v1:172.21.15.31:6789/0 conn(0x7f786c06a450 legacy=0x7f786c06cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:54.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.320+0000 7f786affd700 1 -- 172.21.15.2:0/1513357374 >> v1:172.21.15.2:6789/0 conn(0x7f786c071330 legacy=0x7f786c06d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:54.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.320+0000 7f786affd700 1 -- 172.21.15.2:0/1513357374 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c06dd20 con 0x7f786c06a830 2023-08-25T03:34:54.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.320+0000 7f786affd700 1 --2- 172.21.15.2:0/1513357374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f786c06a830 0x7f786c0d08b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f785c009840 tx=0x7f785c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b901baf15d4e849 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:54.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.320+0000 7f786a7fc700 1 -- 172.21.15.2:0/1513357374 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c011830 con 0x7f786c06a830 2023-08-25T03:34:54.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.320+0000 7f786a7fc700 1 -- 172.21.15.2:0/1513357374 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785c011990 con 0x7f786c06a830 2023-08-25T03:34:54.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.320+0000 7f786a7fc700 1 -- 172.21.15.2:0/1513357374 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c011d20 con 0x7f786c06a830 2023-08-25T03:34:54.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.321+0000 7f7871d49700 1 -- 172.21.15.2:0/1513357374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f786c06a830 msgr2=0x7f786c0d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:54.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.321+0000 7f7871d49700 1 --2- 172.21.15.2:0/1513357374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f786c06a830 0x7f786c0d08b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f785c009840 tx=0x7f785c009b50 comp rx=0 tx=0).stop 2023-08-25T03:34:54.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.321+0000 7f7871d49700 1 -- 172.21.15.2:0/1513357374 shutdown_connections 2023-08-25T03:34:54.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.321+0000 7f7871d49700 1 --2- 172.21.15.2:0/1513357374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f786c06a830 0x7f786c0d08b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:54.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.321+0000 7f7871d49700 1 -- 172.21.15.2:0/1513357374 >> 172.21.15.2:0/1513357374 conn(0x7f786c0cfd90 msgr2=0x7f786c0d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:54.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.321+0000 7f7871d49700 1 -- 172.21.15.2:0/1513357374 shutdown_connections 2023-08-25T03:34:54.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.321+0000 7f7871d49700 1 -- 172.21.15.2:0/1513357374 wait complete. 2023-08-25T03:34:54.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.322+0000 7f7871d49700 1 Processor -- start 2023-08-25T03:34:54.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.322+0000 7f7871d49700 1 -- start start 2023-08-25T03:34:54.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.323+0000 7f7871d49700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f786c06a450 0x7f786c0c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:54.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.323+0000 7f7871d49700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f786c06a830 0x7f786c0c42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:54.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.323+0000 7f7871d49700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f786c071330 0x7f786c0c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:54.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.323+0000 7f7871d49700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c0c5730 con 0x7f786c071330 2023-08-25T03:34:54.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.323+0000 7f7871d49700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c0c6530 con 0x7f786c06a450 2023-08-25T03:34:54.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.323+0000 7f7871d49700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786c0c5920 con 0x7f786c06a830 2023-08-25T03:34:54.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.323+0000 7f786bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f786c071330 0x7f786c0c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:54.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.323+0000 7f786bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f786c071330 0x7f786c0c7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60932/0 (socket says 172.21.15.2:60932) 2023-08-25T03:34:54.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.323+0000 7f786bfff700 1 -- 172.21.15.2:0/3482029168 learned_addr learned my addr 172.21.15.2:0/3482029168 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:54.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.323+0000 7f786b7fe700 1 --2- 172.21.15.2:0/3482029168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f786c06a450 0x7f786c0c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:54.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.323+0000 7f786affd700 1 --2- 172.21.15.2:0/3482029168 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f786c06a830 0x7f786c0c42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:54.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.323+0000 7f786bfff700 1 -- 172.21.15.2:0/3482029168 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f786c06a830 msgr2=0x7f786c0c42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:54.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.323+0000 7f786bfff700 1 --2- 172.21.15.2:0/3482029168 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f786c06a830 0x7f786c0c42b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:54.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.323+0000 7f786bfff700 1 -- 172.21.15.2:0/3482029168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f786c06a450 msgr2=0x7f786c0c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:54.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.323+0000 7f786bfff700 1 --2- 172.21.15.2:0/3482029168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f786c06a450 0x7f786c0c51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:54.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.323+0000 7f786bfff700 1 -- 172.21.15.2:0/3482029168 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785c0094a0 con 0x7f786c071330 2023-08-25T03:34:54.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.324+0000 7f786bfff700 1 --2- 172.21.15.2:0/3482029168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f786c071330 0x7f786c0c7bc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f786400cb10 tx=0x7f786400ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:54.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.324+0000 7f7853fff700 1 -- 172.21.15.2:0/3482029168 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786400a9e0 con 0x7f786c071330 2023-08-25T03:34:54.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.324+0000 7f7853fff700 1 -- 172.21.15.2:0/3482029168 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786400ab40 con 0x7f786c071330 2023-08-25T03:34:54.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.324+0000 7f7853fff700 1 -- 172.21.15.2:0/3482029168 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7864012400 con 0x7f786c071330 2023-08-25T03:34:54.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.324+0000 7f7871d49700 1 -- 172.21.15.2:0/3482029168 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f786c0c5c00 con 0x7f786c071330 2023-08-25T03:34:54.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.324+0000 7f7871d49700 1 -- 172.21.15.2:0/3482029168 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786c0c61c0 con 0x7f786c071330 2023-08-25T03:34:54.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.325+0000 7f7871d49700 1 -- 172.21.15.2:0/3482029168 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786c0746d0 con 0x7f786c071330 2023-08-25T03:34:54.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.325+0000 7f7853fff700 1 -- 172.21.15.2:0/3482029168 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7864007d50 con 0x7f786c071330 2023-08-25T03:34:54.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.326+0000 7f7853fff700 1 --2- 172.21.15.2:0/3482029168 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f785806a0e0 0x7f785806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:54.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.326+0000 7f7853fff700 1 -- 172.21.15.2:0/3482029168 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f7864087e80 con 0x7f786c071330 2023-08-25T03:34:54.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.326+0000 7f786b7fe700 1 --2- 172.21.15.2:0/3482029168 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f785806a0e0 0x7f785806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:54.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.326+0000 7f786b7fe700 1 --2- 172.21.15.2:0/3482029168 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f785806a0e0 0x7f785806c5a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7860007fd0 tx=0x7f7860009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:54.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.330+0000 7f7853fff700 1 -- 172.21.15.2:0/3482029168 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7864053530 con 0x7f786c071330 2023-08-25T03:34:54.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.529+0000 7f7871d49700 1 -- 172.21.15.2:0/3482029168 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f786c0c6670 con 0x7f786c071330 2023-08-25T03:34:54.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.529+0000 7f7853fff700 1 -- 172.21.15.2:0/3482029168 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f78640170a0 con 0x7f786c071330 2023-08-25T03:34:54.531 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-08-25T03:34:54.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.531+0000 7f7871d49700 1 -- 172.21.15.2:0/3482029168 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f785806a0e0 msgr2=0x7f785806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:54.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.531+0000 7f7871d49700 1 --2- 172.21.15.2:0/3482029168 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f785806a0e0 0x7f785806c5a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7860007fd0 tx=0x7f7860009040 comp rx=0 tx=0).stop 2023-08-25T03:34:54.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.531+0000 7f7871d49700 1 -- 172.21.15.2:0/3482029168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f786c071330 msgr2=0x7f786c0c7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:54.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.531+0000 7f7871d49700 1 --2- 172.21.15.2:0/3482029168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f786c071330 0x7f786c0c7bc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f786400cb10 tx=0x7f786400ce20 comp rx=0 tx=0).stop 2023-08-25T03:34:54.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.531+0000 7f7871d49700 1 -- 172.21.15.2:0/3482029168 shutdown_connections 2023-08-25T03:34:54.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.531+0000 7f7871d49700 1 --2- 172.21.15.2:0/3482029168 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f785806a0e0 0x7f785806c5a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:54.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.531+0000 7f7871d49700 1 --2- 172.21.15.2:0/3482029168 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f786c071330 0x7f786c0c7bc0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:54.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.531+0000 7f7871d49700 1 --2- 172.21.15.2:0/3482029168 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f786c06a450 0x7f786c0c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:54.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.531+0000 7f7871d49700 1 --2- 172.21.15.2:0/3482029168 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f786c06a830 0x7f786c0c42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:54.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.531+0000 7f7871d49700 1 -- 172.21.15.2:0/3482029168 >> 172.21.15.2:0/3482029168 conn(0x7f786c0cfd90 msgr2=0x7f786c0d03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:54.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.532+0000 7f7871d49700 1 -- 172.21.15.2:0/3482029168 shutdown_connections 2023-08-25T03:34:54.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.532+0000 7f7871d49700 1 -- 172.21.15.2:0/3482029168 wait complete. 2023-08-25T03:34:54.545 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T03:34:54.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.671+0000 7fa9be550700 1 Processor -- start 2023-08-25T03:34:54.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.671+0000 7fa9be550700 1 -- start start 2023-08-25T03:34:54.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.672+0000 7fa9be550700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9b80b22b0 0x7fa9b806b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:54.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.672+0000 7fa9be550700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9b80cff50 con 0x7fa9b815d6c0 2023-08-25T03:34:54.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.672+0000 7fa9be550700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9b80d0230 con 0x7fa9b806bb60 2023-08-25T03:34:54.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.672+0000 7fa9be550700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b806bff0 con 0x7fa9b80b22b0 2023-08-25T03:34:54.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.672+0000 7fa9b7fff700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fa9b815d6c0 0x7fa9b80d18f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47602/0 (socket says 172.21.15.2:47602) 2023-08-25T03:34:54.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.672+0000 7fa9b7fff700 1 -- 172.21.15.2:0/3000275977 learned_addr learned my addr 172.21.15.2:0/3000275977 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:54.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.672+0000 7fa9b77fe700 1 --2- 172.21.15.2:0/3000275977 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9b80b22b0 0x7fa9b806b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:54.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.672+0000 7fa9b6ffd700 1 -- 172.21.15.2:0/3000275977 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3858127653 0 0) 0x7fa9b80cff50 con 0x7fa9b815d6c0 2023-08-25T03:34:54.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.672+0000 7fa9b6ffd700 1 -- 172.21.15.2:0/3000275977 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9a00035b0 con 0x7fa9b815d6c0 2023-08-25T03:34:54.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.672+0000 7fa9b6ffd700 1 -- 172.21.15.2:0/3000275977 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 282662624 0 0) 0x7fa9b80d0230 con 0x7fa9b806bb60 2023-08-25T03:34:54.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.672+0000 7fa9b6ffd700 1 -- 172.21.15.2:0/3000275977 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9b80cff50 con 0x7fa9b806bb60 2023-08-25T03:34:54.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.672+0000 7fa9b77fe700 1 -- 172.21.15.2:0/3000275977 >> v1:172.21.15.31:6789/0 conn(0x7fa9b806bb60 legacy=0x7fa9b80cbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:54.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.672+0000 7fa9b6ffd700 1 -- 172.21.15.2:0/3000275977 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2848266402 0 0) 0x7fa9a00035b0 con 0x7fa9b815d6c0 2023-08-25T03:34:54.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.672+0000 7fa9b77fe700 1 -- 172.21.15.2:0/3000275977 >> v1:172.21.15.2:6789/0 conn(0x7fa9b815d6c0 legacy=0x7fa9b80d18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:54.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.672+0000 7fa9b77fe700 1 -- 172.21.15.2:0/3000275977 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b80d0510 con 0x7fa9b80b22b0 2023-08-25T03:34:54.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.672+0000 7fa9b77fe700 1 --2- 172.21.15.2:0/3000275977 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9b80b22b0 0x7fa9b806b620 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009870 tx=0x7fa9ac009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42d5b79aaebc6d50 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:54.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.673+0000 7fa9b6ffd700 1 -- 172.21.15.2:0/3000275977 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8001670 con 0x7fa9b80b22b0 2023-08-25T03:34:54.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.673+0000 7fa9b6ffd700 1 -- 172.21.15.2:0/3000275977 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ac004030 con 0x7fa9b80b22b0 2023-08-25T03:34:54.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.673+0000 7fa9b6ffd700 1 -- 172.21.15.2:0/3000275977 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac0043c0 con 0x7fa9b80b22b0 2023-08-25T03:34:54.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.673+0000 7fa9be550700 1 -- 172.21.15.2:0/3000275977 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9b80b22b0 msgr2=0x7fa9b806b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:54.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.673+0000 7fa9be550700 1 --2- 172.21.15.2:0/3000275977 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9b80b22b0 0x7fa9b806b620 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009870 tx=0x7fa9ac009b80 comp rx=0 tx=0).stop 2023-08-25T03:34:54.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.673+0000 7fa9be550700 1 -- 172.21.15.2:0/3000275977 shutdown_connections 2023-08-25T03:34:54.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.673+0000 7fa9be550700 1 --2- 172.21.15.2:0/3000275977 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9b80b22b0 0x7fa9b806b620 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:54.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.673+0000 7fa9be550700 1 -- 172.21.15.2:0/3000275977 >> 172.21.15.2:0/3000275977 conn(0x7fa9b809a2b0 msgr2=0x7fa9b80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:54.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.673+0000 7fa9be550700 1 -- 172.21.15.2:0/3000275977 shutdown_connections 2023-08-25T03:34:54.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.673+0000 7fa9be550700 1 -- 172.21.15.2:0/3000275977 wait complete. 2023-08-25T03:34:54.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.674+0000 7fa9be550700 1 Processor -- start 2023-08-25T03:34:54.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.675+0000 7fa9be550700 1 -- start start 2023-08-25T03:34:54.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.675+0000 7fa9be550700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9b806bb60 0x7fa9b81e71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:54.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.675+0000 7fa9be550700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9b80b22b0 0x7fa9b80ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:54.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.675+0000 7fa9be550700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9b815d6c0 0x7fa9b81503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:54.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.675+0000 7fa9be550700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b81e53d0 con 0x7fa9b806bb60 2023-08-25T03:34:54.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.675+0000 7fa9be550700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b81e7d50 con 0x7fa9b80b22b0 2023-08-25T03:34:54.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.675+0000 7fa9be550700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b81e7790 con 0x7fa9b815d6c0 2023-08-25T03:34:54.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.675+0000 7fa9b7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9b806bb60 0x7fa9b81e71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:54.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.675+0000 7fa9b7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9b806bb60 0x7fa9b81e71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60942/0 (socket says 172.21.15.2:60942) 2023-08-25T03:34:54.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.676+0000 7fa9b7fff700 1 -- 172.21.15.2:0/2593815013 learned_addr learned my addr 172.21.15.2:0/2593815013 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:54.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.676+0000 7fa9b77fe700 1 --2- 172.21.15.2:0/2593815013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9b80b22b0 0x7fa9b80ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:54.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.676+0000 7fa9bcaed700 1 --2- 172.21.15.2:0/2593815013 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9b815d6c0 0x7fa9b81503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:54.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.676+0000 7fa9b7fff700 1 -- 172.21.15.2:0/2593815013 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9b815d6c0 msgr2=0x7fa9b81503d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:54.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.676+0000 7fa9b7fff700 1 --2- 172.21.15.2:0/2593815013 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9b815d6c0 0x7fa9b81503d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:54.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.676+0000 7fa9b7fff700 1 -- 172.21.15.2:0/2593815013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9b80b22b0 msgr2=0x7fa9b80ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:54.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.676+0000 7fa9b7fff700 1 --2- 172.21.15.2:0/2593815013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9b80b22b0 0x7fa9b80ca4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:54.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.676+0000 7fa9b7fff700 1 -- 172.21.15.2:0/2593815013 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ac0094a0 con 0x7fa9b806bb60 2023-08-25T03:34:54.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.676+0000 7fa9b7fff700 1 --2- 172.21.15.2:0/2593815013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9b806bb60 0x7fa9b81e71b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800c6b0 tx=0x7fa9a800ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:54.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.676+0000 7fa9b4ff9700 1 -- 172.21.15.2:0/2593815013 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8004c80 con 0x7fa9b806bb60 2023-08-25T03:34:54.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.676+0000 7fa9b4ff9700 1 -- 172.21.15.2:0/2593815013 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a8004e00 con 0x7fa9b806bb60 2023-08-25T03:34:54.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.676+0000 7fa9be550700 1 -- 172.21.15.2:0/2593815013 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b81e7a70 con 0x7fa9b806bb60 2023-08-25T03:34:54.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.676+0000 7fa9b4ff9700 1 -- 172.21.15.2:0/2593815013 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8007310 con 0x7fa9b806bb60 2023-08-25T03:34:54.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.676+0000 7fa9be550700 1 -- 172.21.15.2:0/2593815013 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b80caa10 con 0x7fa9b806bb60 2023-08-25T03:34:54.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.677+0000 7fa9be550700 1 -- 172.21.15.2:0/2593815013 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9b80ce580 con 0x7fa9b806bb60 2023-08-25T03:34:54.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.678+0000 7fa9b4ff9700 1 -- 172.21.15.2:0/2593815013 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa9a8007470 con 0x7fa9b806bb60 2023-08-25T03:34:54.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.678+0000 7fa9b4ff9700 1 --2- 172.21.15.2:0/2593815013 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa9a406a0e0 0x7fa9a406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:54.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.678+0000 7fa9b4ff9700 1 -- 172.21.15.2:0/2593815013 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fa9a8088400 con 0x7fa9b806bb60 2023-08-25T03:34:54.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.678+0000 7fa9b77fe700 1 --2- 172.21.15.2:0/2593815013 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa9a406a0e0 0x7fa9a406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:54.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.679+0000 7fa9b77fe700 1 --2- 172.21.15.2:0/2593815013 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa9a406a0e0 0x7fa9a406c5a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0090b0 tx=0x7fa9ac008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:54.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.683+0000 7fa9b4ff9700 1 -- 172.21.15.2:0/2593815013 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa9a8053ab0 con 0x7fa9b806bb60 2023-08-25T03:34:54.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.930+0000 7fa9be550700 1 -- 172.21.15.2:0/2593815013 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9b81e2c00 con 0x7fa9b806bb60 2023-08-25T03:34:54.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.931+0000 7fa9b4ff9700 1 -- 172.21.15.2:0/2593815013 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4857 (secure 0 0 0) 0x7fa9a80570d0 con 0x7fa9b806bb60 2023-08-25T03:34:54.932 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:54.932 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2074312983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2074312983},{"type":"v1","addr":"172.21.15.2:6837","nonce":2074312983}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4059180922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4059180922},{"type":"v1","addr":"172.21.15.31:6835","nonce":4059180922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/922384230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":922384230},{"type":"v1","addr":"172.21.15.31:6834","nonce":922384230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T03:34:03.009109+0000","modified":"2023-08-25T03:34:06.613200+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.2:6835/193990892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":193990892},{"type":"v1","addr":"172.21.15.2:6835","nonce":193990892}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T03:34:54.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.933+0000 7fa9be550700 1 -- 172.21.15.2:0/2593815013 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa9a406a0e0 msgr2=0x7fa9a406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:54.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.933+0000 7fa9be550700 1 --2- 172.21.15.2:0/2593815013 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa9a406a0e0 0x7fa9a406c5a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0090b0 tx=0x7fa9ac008030 comp rx=0 tx=0).stop 2023-08-25T03:34:54.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.933+0000 7fa9be550700 1 -- 172.21.15.2:0/2593815013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9b806bb60 msgr2=0x7fa9b81e71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:54.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.933+0000 7fa9be550700 1 --2- 172.21.15.2:0/2593815013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9b806bb60 0x7fa9b81e71b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fa9a800c6b0 tx=0x7fa9a800ca70 comp rx=0 tx=0).stop 2023-08-25T03:34:54.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.933+0000 7fa9be550700 1 -- 172.21.15.2:0/2593815013 shutdown_connections 2023-08-25T03:34:54.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.933+0000 7fa9be550700 1 --2- 172.21.15.2:0/2593815013 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa9a406a0e0 0x7fa9a406c5a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:54.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.933+0000 7fa9be550700 1 --2- 172.21.15.2:0/2593815013 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa9b806bb60 0x7fa9b81e71b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:54.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.933+0000 7fa9be550700 1 --2- 172.21.15.2:0/2593815013 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9b815d6c0 0x7fa9b81503d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:54.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.933+0000 7fa9be550700 1 --2- 172.21.15.2:0/2593815013 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9b80b22b0 0x7fa9b80ca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:54.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.933+0000 7fa9be550700 1 -- 172.21.15.2:0/2593815013 >> 172.21.15.2:0/2593815013 conn(0x7fa9b809a2b0 msgr2=0x7fa9b80cda50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:54.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.933+0000 7fa9be550700 1 -- 172.21.15.2:0/2593815013 shutdown_connections 2023-08-25T03:34:54.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:54.933+0000 7fa9be550700 1 -- 172.21.15.2:0/2593815013 wait complete. 2023-08-25T03:34:54.937 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 8 2023-08-25T03:34:54.946 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:34:55.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.071+0000 7f7145e61700 1 Processor -- start 2023-08-25T03:34:55.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.071+0000 7f7145e61700 1 -- start start 2023-08-25T03:34:55.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.072+0000 7f7145e61700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f714015d400 0x7f71400cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:55.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.072+0000 7f7145e61700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f71400d04e0 0x7f71400b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:55.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.072+0000 7f7145e61700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71400cf940 con 0x7f71400cba10 2023-08-25T03:34:55.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.072+0000 7f7145e61700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71400cf330 con 0x7f71400d04e0 2023-08-25T03:34:55.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.072+0000 7f7145e61700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f714006c3f0 con 0x7f714015d400 2023-08-25T03:34:55.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.072+0000 7f713ffff700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f71400d04e0 0x7f71400b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:55.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.072+0000 7f713ffff700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f71400d04e0 0x7f71400b26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60954/0 (socket says 172.21.15.2:60954) 2023-08-25T03:34:55.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.072+0000 7f713ffff700 1 -- 172.21.15.2:0/3344702415 learned_addr learned my addr 172.21.15.2:0/3344702415 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:55.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.072+0000 7f713f7fe700 1 --2- 172.21.15.2:0/3344702415 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f714015d400 0x7f71400cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:55.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.072+0000 7f713ffff700 1 -- 172.21.15.2:0/3344702415 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f714015d400 msgr2=0x7f71400cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:55.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.072+0000 7f713ffff700 1 --2- 172.21.15.2:0/3344702415 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f714015d400 0x7f71400cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:55.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.072+0000 7f713ffff700 1 -- 172.21.15.2:0/3344702415 >> v1:172.21.15.31:6789/0 conn(0x7f71400cba10 legacy=0x7f71400cfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:55.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.072+0000 7f713ffff700 1 -- 172.21.15.2:0/3344702415 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f714006c6d0 con 0x7f71400d04e0 2023-08-25T03:34:55.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.072+0000 7f713f7fe700 1 --2- 172.21.15.2:0/3344702415 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f714015d400 0x7f71400cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:55.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.072+0000 7f713ffff700 1 --2- 172.21.15.2:0/3344702415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71400d04e0 0x7f71400b26b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7138009e50 tx=0x7f713800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af3e3bdd80aceb1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:55.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.072+0000 7f713e7fc700 1 -- 172.21.15.2:0/3344702415 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713800e070 con 0x7f71400d04e0 2023-08-25T03:34:55.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.072+0000 7f713e7fc700 1 -- 172.21.15.2:0/3344702415 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713800b9c0 con 0x7f71400d04e0 2023-08-25T03:34:55.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.072+0000 7f713e7fc700 1 -- 172.21.15.2:0/3344702415 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713800bd70 con 0x7f71400d04e0 2023-08-25T03:34:55.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.073+0000 7f7145e61700 1 -- 172.21.15.2:0/3344702415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71400d04e0 msgr2=0x7f71400b26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:55.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.073+0000 7f7145e61700 1 --2- 172.21.15.2:0/3344702415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71400d04e0 0x7f71400b26b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7138009e50 tx=0x7f713800b330 comp rx=0 tx=0).stop 2023-08-25T03:34:55.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.073+0000 7f7145e61700 1 -- 172.21.15.2:0/3344702415 shutdown_connections 2023-08-25T03:34:55.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.073+0000 7f7145e61700 1 --2- 172.21.15.2:0/3344702415 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f71400d04e0 0x7f71400b26b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:55.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.073+0000 7f7145e61700 1 --2- 172.21.15.2:0/3344702415 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f714015d400 0x7f71400cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:55.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.073+0000 7f7145e61700 1 -- 172.21.15.2:0/3344702415 >> 172.21.15.2:0/3344702415 conn(0x7f714009a270 msgr2=0x7f71400d0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:55.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.073+0000 7f7145e61700 1 -- 172.21.15.2:0/3344702415 shutdown_connections 2023-08-25T03:34:55.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.073+0000 7f7145e61700 1 -- 172.21.15.2:0/3344702415 wait complete. 2023-08-25T03:34:55.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.074+0000 7f7145e61700 1 Processor -- start 2023-08-25T03:34:55.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.074+0000 7f7145e61700 1 -- start start 2023-08-25T03:34:55.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.075+0000 7f7145e61700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f71400cba10 0x7f71400c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:55.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.075+0000 7f7145e61700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71400d04e0 0x7f7140157030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:55.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.075+0000 7f7145e61700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f714015d400 0x7f7140157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:55.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.075+0000 7f7145e61700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71400c5440 con 0x7f714015d400 2023-08-25T03:34:55.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.075+0000 7f7145e61700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71400c5720 con 0x7f71400d04e0 2023-08-25T03:34:55.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.075+0000 7f7145e61700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71400c5a00 con 0x7f71400cba10 2023-08-25T03:34:55.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.075+0000 7f713ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f714015d400 0x7f7140157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:55.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.075+0000 7f713ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f714015d400 0x7f7140157650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60958/0 (socket says 172.21.15.2:60958) 2023-08-25T03:34:55.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.075+0000 7f713ffff700 1 -- 172.21.15.2:0/3480903911 learned_addr learned my addr 172.21.15.2:0/3480903911 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:55.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.075+0000 7f713f7fe700 1 --2- 172.21.15.2:0/3480903911 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f71400cba10 0x7f71400c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:55.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.075+0000 7f713effd700 1 --2- 172.21.15.2:0/3480903911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71400d04e0 0x7f7140157030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:55.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.076+0000 7f713ffff700 1 -- 172.21.15.2:0/3480903911 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f71400cba10 msgr2=0x7f71400c6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:55.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.076+0000 7f713ffff700 1 --2- 172.21.15.2:0/3480903911 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f71400cba10 0x7f71400c6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:55.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.076+0000 7f713ffff700 1 -- 172.21.15.2:0/3480903911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71400d04e0 msgr2=0x7f7140157030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:55.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.076+0000 7f713ffff700 1 --2- 172.21.15.2:0/3480903911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71400d04e0 0x7f7140157030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:55.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.076+0000 7f713ffff700 1 -- 172.21.15.2:0/3480903911 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7138009b30 con 0x7f714015d400 2023-08-25T03:34:55.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.076+0000 7f713ffff700 1 --2- 172.21.15.2:0/3480903911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f714015d400 0x7f7140157650 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7138012000 tx=0x7f7138004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:55.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.076+0000 7f7127fff700 1 -- 172.21.15.2:0/3480903911 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7138004800 con 0x7f714015d400 2023-08-25T03:34:55.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.076+0000 7f7127fff700 1 -- 172.21.15.2:0/3480903911 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71380092e0 con 0x7f714015d400 2023-08-25T03:34:55.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.076+0000 7f7145e61700 1 -- 172.21.15.2:0/3480903911 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71400c5ce0 con 0x7f714015d400 2023-08-25T03:34:55.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.076+0000 7f7127fff700 1 -- 172.21.15.2:0/3480903911 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713801c680 con 0x7f714015d400 2023-08-25T03:34:55.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.076+0000 7f7145e61700 1 -- 172.21.15.2:0/3480903911 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7140157d30 con 0x7f714015d400 2023-08-25T03:34:55.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.077+0000 7f7145e61700 1 -- 172.21.15.2:0/3480903911 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71400c5f20 con 0x7f714015d400 2023-08-25T03:34:55.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.078+0000 7f7127fff700 1 -- 172.21.15.2:0/3480903911 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f713801c7e0 con 0x7f714015d400 2023-08-25T03:34:55.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.078+0000 7f7127fff700 1 --2- 172.21.15.2:0/3480903911 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f712c06a110 0x7f712c06c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:55.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.078+0000 7f7127fff700 1 -- 172.21.15.2:0/3480903911 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f713808a250 con 0x7f714015d400 2023-08-25T03:34:55.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.078+0000 7f713f7fe700 1 --2- 172.21.15.2:0/3480903911 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f712c06a110 0x7f712c06c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:55.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.079+0000 7f713f7fe700 1 --2- 172.21.15.2:0/3480903911 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f712c06a110 0x7f712c06c5d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f713000abe0 tx=0x7f7130008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:55.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.082+0000 7f7127fff700 1 -- 172.21.15.2:0/3480903911 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f71400c5f20 con 0x7f714015d400 2023-08-25T03:34:55.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.283+0000 7f7145e61700 1 -- 172.21.15.2:0/3480903911 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71400c5f20 con 0x7f714015d400 2023-08-25T03:34:55.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.284+0000 7f7127fff700 1 -- 172.21.15.2:0/3480903911 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18230 (secure 0 0 0) 0x7f71400c5f20 con 0x7f714015d400 2023-08-25T03:34:55.285 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:55.285 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:34:44.588865+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:34:00.869118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T03:34:01.987016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.f","mappings":[{"from":1,"to":5}]},{"pgid":"4.2b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:34:55.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.286+0000 7f7145e61700 1 -- 172.21.15.2:0/3480903911 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f712c06a110 msgr2=0x7f712c06c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:55.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.286+0000 7f7145e61700 1 --2- 172.21.15.2:0/3480903911 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f712c06a110 0x7f712c06c5d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f713000abe0 tx=0x7f7130008040 comp rx=0 tx=0).stop 2023-08-25T03:34:55.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.286+0000 7f7145e61700 1 -- 172.21.15.2:0/3480903911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f714015d400 msgr2=0x7f7140157650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:55.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.286+0000 7f7145e61700 1 --2- 172.21.15.2:0/3480903911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f714015d400 0x7f7140157650 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7138012000 tx=0x7f7138004ad0 comp rx=0 tx=0).stop 2023-08-25T03:34:55.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.286+0000 7f7145e61700 1 -- 172.21.15.2:0/3480903911 shutdown_connections 2023-08-25T03:34:55.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.286+0000 7f7145e61700 1 --2- 172.21.15.2:0/3480903911 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f712c06a110 0x7f712c06c5d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:55.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.286+0000 7f7145e61700 1 --2- 172.21.15.2:0/3480903911 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f714015d400 0x7f7140157650 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:55.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.286+0000 7f7145e61700 1 --2- 172.21.15.2:0/3480903911 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f71400cba10 0x7f71400c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:55.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.286+0000 7f7145e61700 1 --2- 172.21.15.2:0/3480903911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71400d04e0 0x7f7140157030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:55.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.286+0000 7f7145e61700 1 -- 172.21.15.2:0/3480903911 >> 172.21.15.2:0/3480903911 conn(0x7f714009a270 msgr2=0x7f71400cbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:55.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.287+0000 7f7145e61700 1 -- 172.21.15.2:0/3480903911 shutdown_connections 2023-08-25T03:34:55.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.287+0000 7f7145e61700 1 -- 172.21.15.2:0/3480903911 wait complete. 2023-08-25T03:34:55.300 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-25T03:34:55.301 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T03:34:55.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.425+0000 7f8be91ef700 1 Processor -- start 2023-08-25T03:34:55.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.425+0000 7f8be91ef700 1 -- start start 2023-08-25T03:34:55.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.426+0000 7f8be91ef700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8be40cd0a0 0x7f8be4161020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:55.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.426+0000 7f8be91ef700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8be40ce220 0x7f8be406a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:55.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.426+0000 7f8be91ef700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8be4161fa0 con 0x7f8be4071d90 2023-08-25T03:34:55.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.426+0000 7f8be91ef700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8be4072210 con 0x7f8be40ce220 2023-08-25T03:34:55.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.426+0000 7f8be91ef700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be4161700 con 0x7f8be40cd0a0 2023-08-25T03:34:55.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.426+0000 7f8be259c700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f8be4071d90 0x7f8be40cdb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47630/0 (socket says 172.21.15.2:47630) 2023-08-25T03:34:55.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.426+0000 7f8be259c700 1 -- 172.21.15.2:0/4195443855 learned_addr learned my addr 172.21.15.2:0/4195443855 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:55.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.426+0000 7f8be2d9d700 1 --2- 172.21.15.2:0/4195443855 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8be40cd0a0 0x7f8be4161020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:55.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.426+0000 7f8be359e700 1 --2- 172.21.15.2:0/4195443855 >> v2:172.21.15.31:3300/0 conn(0x7f8be40ce220 0x7f8be406a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:55.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.426+0000 7f8be1d9b700 1 -- 172.21.15.2:0/4195443855 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3658784952 0 0) 0x7f8be4161fa0 con 0x7f8be4071d90 2023-08-25T03:34:55.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.426+0000 7f8be1d9b700 1 -- 172.21.15.2:0/4195443855 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bcc0035b0 con 0x7f8be4071d90 2023-08-25T03:34:55.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.427+0000 7f8be359e700 1 -- 172.21.15.2:0/4195443855 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8be40cd0a0 msgr2=0x7f8be4161020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:55.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.427+0000 7f8be359e700 1 --2- 172.21.15.2:0/4195443855 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8be40cd0a0 0x7f8be4161020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:55.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.427+0000 7f8be359e700 1 -- 172.21.15.2:0/4195443855 >> v1:172.21.15.2:6789/0 conn(0x7f8be4071d90 legacy=0x7f8be40cdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:55.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.427+0000 7f8be359e700 1 -- 172.21.15.2:0/4195443855 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be41619e0 con 0x7f8be40ce220 2023-08-25T03:34:55.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.427+0000 7f8be359e700 1 --2- 172.21.15.2:0/4195443855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8be40ce220 0x7f8be406a830 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc009d90 tx=0x7f8bdc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38e97009e7d03e91 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:55.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.427+0000 7f8be1d9b700 1 -- 172.21.15.2:0/4195443855 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc00e070 con 0x7f8be40ce220 2023-08-25T03:34:55.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.427+0000 7f8be1d9b700 1 -- 172.21.15.2:0/4195443855 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bdc00bb20 con 0x7f8be40ce220 2023-08-25T03:34:55.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.427+0000 7f8be1d9b700 1 -- 172.21.15.2:0/4195443855 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc00be90 con 0x7f8be40ce220 2023-08-25T03:34:55.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.428+0000 7f8be91ef700 1 -- 172.21.15.2:0/4195443855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8be40ce220 msgr2=0x7f8be406a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:55.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.428+0000 7f8be91ef700 1 --2- 172.21.15.2:0/4195443855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8be40ce220 0x7f8be406a830 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc009d90 tx=0x7f8bdc00b330 comp rx=0 tx=0).stop 2023-08-25T03:34:55.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.428+0000 7f8be91ef700 1 -- 172.21.15.2:0/4195443855 shutdown_connections 2023-08-25T03:34:55.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.428+0000 7f8be91ef700 1 --2- 172.21.15.2:0/4195443855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8be40ce220 0x7f8be406a830 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:55.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.428+0000 7f8be91ef700 1 --2- 172.21.15.2:0/4195443855 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8be40cd0a0 0x7f8be4161020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:55.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.428+0000 7f8be91ef700 1 -- 172.21.15.2:0/4195443855 >> 172.21.15.2:0/4195443855 conn(0x7f8be40b1cf0 msgr2=0x7f8be40b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:55.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.428+0000 7f8be91ef700 1 -- 172.21.15.2:0/4195443855 shutdown_connections 2023-08-25T03:34:55.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.428+0000 7f8be91ef700 1 -- 172.21.15.2:0/4195443855 wait complete. 2023-08-25T03:34:55.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.429+0000 7f8be91ef700 1 Processor -- start 2023-08-25T03:34:55.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.429+0000 7f8be91ef700 1 -- start start 2023-08-25T03:34:55.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.430+0000 7f8be91ef700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8be4071d90 0x7f8be40c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:55.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.430+0000 7f8be91ef700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8be40cd0a0 0x7f8be415bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:55.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.430+0000 7f8be91ef700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be40ce220 0x7f8be41f2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:55.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.430+0000 7f8be91ef700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be415c4e0 con 0x7f8be40ce220 2023-08-25T03:34:55.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.430+0000 7f8be91ef700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be41f2d40 con 0x7f8be40cd0a0 2023-08-25T03:34:55.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.430+0000 7f8be91ef700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be40c7190 con 0x7f8be4071d90 2023-08-25T03:34:55.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.430+0000 7f8be359e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be40ce220 0x7f8be41f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:55.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.430+0000 7f8be359e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be40ce220 0x7f8be41f2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60972/0 (socket says 172.21.15.2:60972) 2023-08-25T03:34:55.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.430+0000 7f8be359e700 1 -- 172.21.15.2:0/1018696303 learned_addr learned my addr 172.21.15.2:0/1018696303 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:55.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.430+0000 7f8be259c700 1 --2- 172.21.15.2:0/1018696303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8be40cd0a0 0x7f8be415bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:55.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.430+0000 7f8be2d9d700 1 --2- 172.21.15.2:0/1018696303 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8be4071d90 0x7f8be40c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:55.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.430+0000 7f8be359e700 1 -- 172.21.15.2:0/1018696303 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8be4071d90 msgr2=0x7f8be40c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:55.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.430+0000 7f8be359e700 1 --2- 172.21.15.2:0/1018696303 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8be4071d90 0x7f8be40c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:55.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.430+0000 7f8be359e700 1 -- 172.21.15.2:0/1018696303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8be40cd0a0 msgr2=0x7f8be415bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:55.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.430+0000 7f8be359e700 1 --2- 172.21.15.2:0/1018696303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8be40cd0a0 0x7f8be415bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:55.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.430+0000 7f8be359e700 1 -- 172.21.15.2:0/1018696303 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bdc009a70 con 0x7f8be40ce220 2023-08-25T03:34:55.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.431+0000 7f8be359e700 1 --2- 172.21.15.2:0/1018696303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be40ce220 0x7f8be41f2800 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc014040 tx=0x7f8bdc004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:55.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.431+0000 7f8bcb7fe700 1 -- 172.21.15.2:0/1018696303 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc0098f0 con 0x7f8be40ce220 2023-08-25T03:34:55.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.431+0000 7f8bcb7fe700 1 -- 172.21.15.2:0/1018696303 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bdc003e10 con 0x7f8be40ce220 2023-08-25T03:34:55.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.431+0000 7f8bcb7fe700 1 -- 172.21.15.2:0/1018696303 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bdc012650 con 0x7f8be40ce220 2023-08-25T03:34:55.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.431+0000 7f8be91ef700 1 -- 172.21.15.2:0/1018696303 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be41f2fe0 con 0x7f8be40ce220 2023-08-25T03:34:55.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.431+0000 7f8be91ef700 1 -- 172.21.15.2:0/1018696303 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be41f35a0 con 0x7f8be40ce220 2023-08-25T03:34:55.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.432+0000 7f8be91ef700 1 -- 172.21.15.2:0/1018696303 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8be4066f70 con 0x7f8be40ce220 2023-08-25T03:34:55.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.432+0000 7f8bcb7fe700 1 -- 172.21.15.2:0/1018696303 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8bdc0127b0 con 0x7f8be40ce220 2023-08-25T03:34:55.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.433+0000 7f8bcb7fe700 1 --2- 172.21.15.2:0/1018696303 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8bd006a0e0 0x7f8bd006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:55.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.433+0000 7f8bcb7fe700 1 -- 172.21.15.2:0/1018696303 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f8bdc08a080 con 0x7f8be40ce220 2023-08-25T03:34:55.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.433+0000 7f8be2d9d700 1 --2- 172.21.15.2:0/1018696303 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8bd006a0e0 0x7f8bd006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:55.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.433+0000 7f8be2d9d700 1 --2- 172.21.15.2:0/1018696303 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8bd006a0e0 0x7f8bd006c5a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009b40 tx=0x7f8bd8009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:55.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.437+0000 7f8bcb7fe700 1 -- 172.21.15.2:0/1018696303 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8bdc055870 con 0x7f8be40ce220 2023-08-25T03:34:55.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.682+0000 7f8be91ef700 1 -- 172.21.15.2:0/1018696303 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8be40c73b0 con 0x7f8be40ce220 2023-08-25T03:34:55.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.683+0000 7f8bcb7fe700 1 -- 172.21.15.2:0/1018696303 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4857 (secure 0 0 0) 0x7f8be40c73b0 con 0x7f8be40ce220 2023-08-25T03:34:55.684 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:55.684 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2074312983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2074312983},{"type":"v1","addr":"172.21.15.2:6837","nonce":2074312983}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4059180922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4059180922},{"type":"v1","addr":"172.21.15.31:6835","nonce":4059180922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/922384230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":922384230},{"type":"v1","addr":"172.21.15.31:6834","nonce":922384230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T03:34:03.009109+0000","modified":"2023-08-25T03:34:06.613200+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.2:6835/193990892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":193990892},{"type":"v1","addr":"172.21.15.2:6835","nonce":193990892}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T03:34:55.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.685+0000 7f8be91ef700 1 -- 172.21.15.2:0/1018696303 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8bd006a0e0 msgr2=0x7f8bd006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:55.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.685+0000 7f8be91ef700 1 --2- 172.21.15.2:0/1018696303 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8bd006a0e0 0x7f8bd006c5a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009b40 tx=0x7f8bd8009210 comp rx=0 tx=0).stop 2023-08-25T03:34:55.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.685+0000 7f8be91ef700 1 -- 172.21.15.2:0/1018696303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be40ce220 msgr2=0x7f8be41f2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:55.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.685+0000 7f8be91ef700 1 --2- 172.21.15.2:0/1018696303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be40ce220 0x7f8be41f2800 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc014040 tx=0x7f8bdc004ad0 comp rx=0 tx=0).stop 2023-08-25T03:34:55.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.685+0000 7f8be91ef700 1 -- 172.21.15.2:0/1018696303 shutdown_connections 2023-08-25T03:34:55.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.685+0000 7f8be91ef700 1 --2- 172.21.15.2:0/1018696303 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8bd006a0e0 0x7f8bd006c5a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:55.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.685+0000 7f8be91ef700 1 --2- 172.21.15.2:0/1018696303 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8be40ce220 0x7f8be41f2800 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:55.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.685+0000 7f8be91ef700 1 --2- 172.21.15.2:0/1018696303 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8be4071d90 0x7f8be40c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:55.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.685+0000 7f8be91ef700 1 --2- 172.21.15.2:0/1018696303 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8be40cd0a0 0x7f8be415bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:55.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.685+0000 7f8be91ef700 1 -- 172.21.15.2:0/1018696303 >> 172.21.15.2:0/1018696303 conn(0x7f8be40b1cf0 msgr2=0x7f8be40cf940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:55.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.685+0000 7f8be91ef700 1 -- 172.21.15.2:0/1018696303 shutdown_connections 2023-08-25T03:34:55.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.685+0000 7f8be91ef700 1 -- 172.21.15.2:0/1018696303 wait complete. 2023-08-25T03:34:55.688 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 8 2023-08-25T03:34:55.698 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T03:34:55.822 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.822+0000 7f265de6e700 1 Processor -- start 2023-08-25T03:34:55.823 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.822+0000 7f265de6e700 1 -- start start 2023-08-25T03:34:55.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.823+0000 7f265de6e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f26580b1fb0 0x7f2658160f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:55.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.823+0000 7f265de6e700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2658161450 0x7f26580cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:55.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.823+0000 7f265de6e700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f265806d450 con 0x7f26580ce0b0 2023-08-25T03:34:55.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.823+0000 7f265de6e700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f265806d730 con 0x7f2658161450 2023-08-25T03:34:55.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.823+0000 7f265de6e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f265806da10 con 0x7f26580b1fb0 2023-08-25T03:34:55.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.823+0000 7f2656ffd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2658161450 0x7f26580cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:55.825 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.823+0000 7f2657fff700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f26580ce0b0 0x7f265806b8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47646/0 (socket says 172.21.15.2:47646) 2023-08-25T03:34:55.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.823+0000 7f2657fff700 1 -- 172.21.15.2:0/3638904778 learned_addr learned my addr 172.21.15.2:0/3638904778 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:55.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.823+0000 7f26577fe700 1 --2- 172.21.15.2:0/3638904778 >> v2:172.21.15.31:3300/0 conn(0x7f26580b1fb0 0x7f2658160f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:55.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.823+0000 7f2656ffd700 1 -- 172.21.15.2:0/3638904778 >> v2:172.21.15.31:3300/0 conn(0x7f26580b1fb0 msgr2=0x7f2658160f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:55.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.823+0000 7f2656ffd700 1 --2- 172.21.15.2:0/3638904778 >> v2:172.21.15.31:3300/0 conn(0x7f26580b1fb0 0x7f2658160f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:55.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.823+0000 7f2656ffd700 1 -- 172.21.15.2:0/3638904778 >> v1:172.21.15.2:6789/0 conn(0x7f26580ce0b0 legacy=0x7f265806b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:55.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.823+0000 7f2656ffd700 1 -- 172.21.15.2:0/3638904778 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f265806dcf0 con 0x7f2658161450 2023-08-25T03:34:55.826 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.823+0000 7f26567fc700 1 -- 172.21.15.2:0/3638904778 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 460013325 0 0) 0x7f265806d450 con 0x7f26580ce0b0 2023-08-25T03:34:55.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.823+0000 7f2656ffd700 1 --2- 172.21.15.2:0/3638904778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2658161450 0x7f26580cdb70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f265806e0d0 tx=0x7f264c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8cfdcf94a2c5762 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:55.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.823+0000 7f26567fc700 1 -- 172.21.15.2:0/3638904778 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264c011830 con 0x7f2658161450 2023-08-25T03:34:55.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.823+0000 7f26567fc700 1 -- 172.21.15.2:0/3638904778 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264c011990 con 0x7f2658161450 2023-08-25T03:34:55.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.824+0000 7f26567fc700 1 -- 172.21.15.2:0/3638904778 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264c011d20 con 0x7f2658161450 2023-08-25T03:34:55.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.824+0000 7f265de6e700 1 -- 172.21.15.2:0/3638904778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2658161450 msgr2=0x7f26580cdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:55.827 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.824+0000 7f265de6e700 1 --2- 172.21.15.2:0/3638904778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2658161450 0x7f26580cdb70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f265806e0d0 tx=0x7f264c009b20 comp rx=0 tx=0).stop 2023-08-25T03:34:55.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.824+0000 7f265de6e700 1 -- 172.21.15.2:0/3638904778 shutdown_connections 2023-08-25T03:34:55.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.824+0000 7f265de6e700 1 --2- 172.21.15.2:0/3638904778 >> v2:172.21.15.31:3300/0 conn(0x7f26580b1fb0 0x7f2658160f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:55.828 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.824+0000 7f265de6e700 1 --2- 172.21.15.2:0/3638904778 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2658161450 0x7f26580cdb70 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:55.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.824+0000 7f265de6e700 1 -- 172.21.15.2:0/3638904778 >> 172.21.15.2:0/3638904778 conn(0x7f26580cfea0 msgr2=0x7f26580d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:55.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.824+0000 7f265de6e700 1 -- 172.21.15.2:0/3638904778 shutdown_connections 2023-08-25T03:34:55.829 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.824+0000 7f265de6e700 1 -- 172.21.15.2:0/3638904778 wait complete. 2023-08-25T03:34:55.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.825+0000 7f265de6e700 1 Processor -- start 2023-08-25T03:34:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.825+0000 7f265de6e700 1 -- start start 2023-08-25T03:34:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.826+0000 7f265de6e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26580b1fb0 0x7f2658148160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:55.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.826+0000 7f265de6e700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f26580ce0b0 0x7f26581486a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:55.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.826+0000 7f265de6e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f265815a4d0 0x7f265815a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:55.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.826+0000 7f265de6e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2658145f00 con 0x7f26580b1fb0 2023-08-25T03:34:55.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.826+0000 7f265de6e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26580b4640 con 0x7f265815a4d0 2023-08-25T03:34:55.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.826+0000 7f265de6e700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26580b4920 con 0x7f26580ce0b0 2023-08-25T03:34:55.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.826+0000 7f26577fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26580b1fb0 0x7f2658148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:55.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.826+0000 7f26577fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26580b1fb0 0x7f2658148160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60982/0 (socket says 172.21.15.2:60982) 2023-08-25T03:34:55.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.826+0000 7f26577fe700 1 -- 172.21.15.2:0/2470619791 learned_addr learned my addr 172.21.15.2:0/2470619791 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:55.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.826+0000 7f2657fff700 1 --2- 172.21.15.2:0/2470619791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f265815a4d0 0x7f265815a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:55.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.826+0000 7f2656ffd700 1 --2- 172.21.15.2:0/2470619791 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f26580ce0b0 0x7f26581486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:55.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.827+0000 7f26577fe700 1 -- 172.21.15.2:0/2470619791 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f26580ce0b0 msgr2=0x7f26581486a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:55.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.827+0000 7f26577fe700 1 --2- 172.21.15.2:0/2470619791 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f26580ce0b0 0x7f26581486a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:55.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.827+0000 7f26577fe700 1 -- 172.21.15.2:0/2470619791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f265815a4d0 msgr2=0x7f265815a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:55.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.827+0000 7f26577fe700 1 --2- 172.21.15.2:0/2470619791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f265815a4d0 0x7f265815a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:55.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.827+0000 7f26577fe700 1 -- 172.21.15.2:0/2470619791 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264c0094a0 con 0x7f26580b1fb0 2023-08-25T03:34:55.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.827+0000 7f2656ffd700 1 --2- 172.21.15.2:0/2470619791 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f26580ce0b0 0x7f26581486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:55.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.827+0000 7f26577fe700 1 --2- 172.21.15.2:0/2470619791 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26580b1fb0 0x7f2658148160 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f264800e870 tx=0x7f264800eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:55.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.827+0000 7f263ffff700 1 -- 172.21.15.2:0/2470619791 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2648009590 con 0x7f26580b1fb0 2023-08-25T03:34:55.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.827+0000 7f263ffff700 1 -- 172.21.15.2:0/2470619791 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2648013680 con 0x7f26580b1fb0 2023-08-25T03:34:55.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.827+0000 7f265de6e700 1 -- 172.21.15.2:0/2470619791 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26580b4c00 con 0x7f26580b1fb0 2023-08-25T03:34:55.839 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.827+0000 7f263ffff700 1 -- 172.21.15.2:0/2470619791 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26480139f0 con 0x7f26580b1fb0 2023-08-25T03:34:55.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.827+0000 7f265de6e700 1 -- 172.21.15.2:0/2470619791 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26580b51c0 con 0x7f26580b1fb0 2023-08-25T03:34:55.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.828+0000 7f265de6e700 1 -- 172.21.15.2:0/2470619791 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26580b5530 con 0x7f26580b1fb0 2023-08-25T03:34:55.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.828+0000 7f263ffff700 1 -- 172.21.15.2:0/2470619791 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2648013b50 con 0x7f26580b1fb0 2023-08-25T03:34:55.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.829+0000 7f263ffff700 1 --2- 172.21.15.2:0/2470619791 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f264406a0e0 0x7f264406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:55.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.829+0000 7f2656ffd700 1 --2- 172.21.15.2:0/2470619791 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f264406a0e0 0x7f264406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:55.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.829+0000 7f263ffff700 1 -- 172.21.15.2:0/2470619791 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f2648088500 con 0x7f26580b1fb0 2023-08-25T03:34:55.841 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.829+0000 7f2656ffd700 1 --2- 172.21.15.2:0/2470619791 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f264406a0e0 0x7f264406c5a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f265806e0d0 tx=0x7f264c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:55.842 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:55.833+0000 7f263ffff700 1 -- 172.21.15.2:0/2470619791 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2648053bb0 con 0x7f26580b1fb0 2023-08-25T03:34:56.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.080+0000 7f265de6e700 1 -- 172.21.15.2:0/2470619791 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2658066f70 con 0x7f26580b1fb0 2023-08-25T03:34:56.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.081+0000 7f263ffff700 1 -- 172.21.15.2:0/2470619791 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4857 (secure 0 0 0) 0x7f26480571d0 con 0x7f26580b1fb0 2023-08-25T03:34:56.082 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:56.082 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4250,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/2074312983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6836","nonce":2074312983},{"type":"v1","addr":"172.21.15.2:6837","nonce":2074312983}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4256,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/4059180922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":4059180922},{"type":"v1","addr":"172.21.15.31:6835","nonce":4059180922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4282,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/922384230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":922384230},{"type":"v1","addr":"172.21.15.31:6834","nonce":922384230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T03:34:03.009109+0000","modified":"2023-08-25T03:34:06.613200+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4299},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4299":{"gid":4299,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.2:6835/193990892","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":193990892},{"type":"v1","addr":"172.21.15.2:6835","nonce":193990892}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-08-25T03:34:56.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.083+0000 7f265de6e700 1 -- 172.21.15.2:0/2470619791 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f264406a0e0 msgr2=0x7f264406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:56.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.083+0000 7f265de6e700 1 --2- 172.21.15.2:0/2470619791 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f264406a0e0 0x7f264406c5a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f265806e0d0 tx=0x7f264c008030 comp rx=0 tx=0).stop 2023-08-25T03:34:56.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.083+0000 7f265de6e700 1 -- 172.21.15.2:0/2470619791 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26580b1fb0 msgr2=0x7f2658148160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:56.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.083+0000 7f265de6e700 1 --2- 172.21.15.2:0/2470619791 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26580b1fb0 0x7f2658148160 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f264800e870 tx=0x7f264800eb80 comp rx=0 tx=0).stop 2023-08-25T03:34:56.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.083+0000 7f265de6e700 1 -- 172.21.15.2:0/2470619791 shutdown_connections 2023-08-25T03:34:56.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.083+0000 7f265de6e700 1 --2- 172.21.15.2:0/2470619791 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f264406a0e0 0x7f264406c5a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:56.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.083+0000 7f265de6e700 1 --2- 172.21.15.2:0/2470619791 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f26580b1fb0 0x7f2658148160 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:56.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.083+0000 7f265de6e700 1 --2- 172.21.15.2:0/2470619791 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f265815a4d0 0x7f265815a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:56.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.083+0000 7f265de6e700 1 --2- 172.21.15.2:0/2470619791 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f26580ce0b0 0x7f26581486a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:56.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.083+0000 7f265de6e700 1 -- 172.21.15.2:0/2470619791 >> 172.21.15.2:0/2470619791 conn(0x7f26580cfea0 msgr2=0x7f265806a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:56.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.083+0000 7f265de6e700 1 -- 172.21.15.2:0/2470619791 shutdown_connections 2023-08-25T03:34:56.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.083+0000 7f265de6e700 1 -- 172.21.15.2:0/2470619791 wait complete. 2023-08-25T03:34:56.087 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 8 2023-08-25T03:34:56.096 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:34:56.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.220+0000 7f7f4cfdf700 1 Processor -- start 2023-08-25T03:34:56.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.220+0000 7f7f4cfdf700 1 -- start start 2023-08-25T03:34:56.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.221+0000 7f7f4cfdf700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7f480cfcc0 0x7f7f480b2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:56.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.221+0000 7f7f4cfdf700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f480ccf00 con 0x7f7f480cca90 2023-08-25T03:34:56.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.221+0000 7f7f4cfdf700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f480cbd10 con 0x7f7f4815ea20 2023-08-25T03:34:56.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.221+0000 7f7f4cfdf700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f480cbfb0 con 0x7f7f480cfcc0 2023-08-25T03:34:56.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.221+0000 7f7f46d9d700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f7f480cca90 0x7f7f480b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47656/0 (socket says 172.21.15.2:47656) 2023-08-25T03:34:56.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.222+0000 7f7f46d9d700 1 -- 172.21.15.2:0/1712158295 learned_addr learned my addr 172.21.15.2:0/1712158295 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:56.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.222+0000 7f7f45d9b700 1 --2- 172.21.15.2:0/1712158295 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7f480cfcc0 0x7f7f480b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:56.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.222+0000 7f7f4559a700 1 -- 172.21.15.2:0/1712158295 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3221506018 0 0) 0x7f7f480ccf00 con 0x7f7f480cca90 2023-08-25T03:34:56.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.222+0000 7f7f4559a700 1 -- 172.21.15.2:0/1712158295 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f340035b0 con 0x7f7f480cca90 2023-08-25T03:34:56.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.222+0000 7f7f4559a700 1 -- 172.21.15.2:0/1712158295 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3990445702 0 0) 0x7f7f480cbd10 con 0x7f7f4815ea20 2023-08-25T03:34:56.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.222+0000 7f7f4559a700 1 -- 172.21.15.2:0/1712158295 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f480ccf00 con 0x7f7f4815ea20 2023-08-25T03:34:56.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.222+0000 7f7f45d9b700 1 -- 172.21.15.2:0/1712158295 >> v1:172.21.15.31:6789/0 conn(0x7f7f4815ea20 legacy=0x7f7f480cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:56.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.222+0000 7f7f45d9b700 1 -- 172.21.15.2:0/1712158295 >> v1:172.21.15.2:6789/0 conn(0x7f7f480cca90 legacy=0x7f7f480b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:56.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.222+0000 7f7f45d9b700 1 -- 172.21.15.2:0/1712158295 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f480cc290 con 0x7f7f480cfcc0 2023-08-25T03:34:56.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.222+0000 7f7f45d9b700 1 --2- 172.21.15.2:0/1712158295 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7f480cfcc0 0x7f7f480b2020 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009870 tx=0x7f7f30009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4c0ed72d948f0c8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:56.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.222+0000 7f7f4559a700 1 -- 172.21.15.2:0/1712158295 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f30011830 con 0x7f7f480cfcc0 2023-08-25T03:34:56.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.222+0000 7f7f4559a700 1 -- 172.21.15.2:0/1712158295 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f30011990 con 0x7f7f480cfcc0 2023-08-25T03:34:56.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.222+0000 7f7f4559a700 1 -- 172.21.15.2:0/1712158295 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f30011d20 con 0x7f7f480cfcc0 2023-08-25T03:34:56.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.223+0000 7f7f4cfdf700 1 -- 172.21.15.2:0/1712158295 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7f480cfcc0 msgr2=0x7f7f480b2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:56.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.223+0000 7f7f4cfdf700 1 --2- 172.21.15.2:0/1712158295 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7f480cfcc0 0x7f7f480b2020 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009870 tx=0x7f7f30009b80 comp rx=0 tx=0).stop 2023-08-25T03:34:56.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.223+0000 7f7f4cfdf700 1 -- 172.21.15.2:0/1712158295 shutdown_connections 2023-08-25T03:34:56.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.223+0000 7f7f4cfdf700 1 --2- 172.21.15.2:0/1712158295 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7f480cfcc0 0x7f7f480b2020 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:56.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.223+0000 7f7f4cfdf700 1 -- 172.21.15.2:0/1712158295 >> 172.21.15.2:0/1712158295 conn(0x7f7f4809a2b0 msgr2=0x7f7f480d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:56.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.223+0000 7f7f4cfdf700 1 -- 172.21.15.2:0/1712158295 shutdown_connections 2023-08-25T03:34:56.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.223+0000 7f7f4cfdf700 1 -- 172.21.15.2:0/1712158295 wait complete. 2023-08-25T03:34:56.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.224+0000 7f7f4cfdf700 1 Processor -- start 2023-08-25T03:34:56.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.225+0000 7f7f4cfdf700 1 -- start start 2023-08-25T03:34:56.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.225+0000 7f7f4cfdf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f480cca90 0x7f7f4815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:56.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.225+0000 7f7f4cfdf700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7f480cfcc0 0x7f7f481515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:56.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.225+0000 7f7f4cfdf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f4815ea20 0x7f7f48157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:56.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.225+0000 7f7f4cfdf700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f480c9800 con 0x7f7f480cca90 2023-08-25T03:34:56.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.225+0000 7f7f4cfdf700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f480c91d0 con 0x7f7f4815ea20 2023-08-25T03:34:56.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.225+0000 7f7f4cfdf700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f48151f80 con 0x7f7f480cfcc0 2023-08-25T03:34:56.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.225+0000 7f7f4659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f480cca90 0x7f7f4815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:56.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.225+0000 7f7f4659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f480cca90 0x7f7f4815dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:60998/0 (socket says 172.21.15.2:60998) 2023-08-25T03:34:56.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.225+0000 7f7f4659c700 1 -- 172.21.15.2:0/2383941374 learned_addr learned my addr 172.21.15.2:0/2383941374 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:56.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.226+0000 7f7f46d9d700 1 --2- 172.21.15.2:0/2383941374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f4815ea20 0x7f7f48157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:56.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.226+0000 7f7f45d9b700 1 --2- 172.21.15.2:0/2383941374 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7f480cfcc0 0x7f7f481515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:56.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.226+0000 7f7f4659c700 1 -- 172.21.15.2:0/2383941374 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7f480cfcc0 msgr2=0x7f7f481515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:56.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.226+0000 7f7f4659c700 1 --2- 172.21.15.2:0/2383941374 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7f480cfcc0 0x7f7f481515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:56.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.226+0000 7f7f4659c700 1 -- 172.21.15.2:0/2383941374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f4815ea20 msgr2=0x7f7f48157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:56.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.226+0000 7f7f4659c700 1 --2- 172.21.15.2:0/2383941374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f4815ea20 0x7f7f48157a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:56.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.226+0000 7f7f4659c700 1 -- 172.21.15.2:0/2383941374 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f300094a0 con 0x7f7f480cca90 2023-08-25T03:34:56.231 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.226+0000 7f7f46d9d700 1 --2- 172.21.15.2:0/2383941374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f4815ea20 0x7f7f48157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:56.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.226+0000 7f7f45d9b700 1 --2- 172.21.15.2:0/2383941374 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7f480cfcc0 0x7f7f481515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:34:56.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.226+0000 7f7f4659c700 1 --2- 172.21.15.2:0/2383941374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f480cca90 0x7f7f4815dca0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800c500 tx=0x7f7f3800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:56.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.226+0000 7f7f3effd700 1 -- 172.21.15.2:0/2383941374 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f380052d0 con 0x7f7f480cca90 2023-08-25T03:34:56.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.226+0000 7f7f3effd700 1 -- 172.21.15.2:0/2383941374 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f38005450 con 0x7f7f480cca90 2023-08-25T03:34:56.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.226+0000 7f7f4cfdf700 1 -- 172.21.15.2:0/2383941374 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f48152260 con 0x7f7f480cca90 2023-08-25T03:34:56.232 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.226+0000 7f7f3effd700 1 -- 172.21.15.2:0/2383941374 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f380038c0 con 0x7f7f480cca90 2023-08-25T03:34:56.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.226+0000 7f7f4cfdf700 1 -- 172.21.15.2:0/2383941374 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f4815e1e0 con 0x7f7f480cca90 2023-08-25T03:34:56.233 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.227+0000 7f7f4cfdf700 1 -- 172.21.15.2:0/2383941374 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f480ce580 con 0x7f7f480cca90 2023-08-25T03:34:56.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.228+0000 7f7f3effd700 1 -- 172.21.15.2:0/2383941374 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7f38003a20 con 0x7f7f480cca90 2023-08-25T03:34:56.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.228+0000 7f7f3effd700 1 --2- 172.21.15.2:0/2383941374 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f7f2806a0e0 0x7f7f2806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:56.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.228+0000 7f7f3effd700 1 -- 172.21.15.2:0/2383941374 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7f7f38087a50 con 0x7f7f480cca90 2023-08-25T03:34:56.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.228+0000 7f7f45d9b700 1 --2- 172.21.15.2:0/2383941374 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f7f2806a0e0 0x7f7f2806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:56.234 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.228+0000 7f7f45d9b700 1 --2- 172.21.15.2:0/2383941374 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f7f2806a0e0 0x7f7f2806c5a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7f481f3bc0 tx=0x7f7f3001c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:56.235 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.233+0000 7f7f3effd700 1 -- 172.21.15.2:0/2383941374 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7f38053100 con 0x7f7f480cca90 2023-08-25T03:34:56.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.434+0000 7f7f4cfdf700 1 -- 172.21.15.2:0/2383941374 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f4806eb30 con 0x7f7f480cca90 2023-08-25T03:34:56.470 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.435+0000 7f7f3effd700 1 -- 172.21.15.2:0/2383941374 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18230 (secure 0 0 0) 0x7f7f38056720 con 0x7f7f480cca90 2023-08-25T03:34:56.471 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:34:56.472 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":26,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:34:44.588865+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:34:00.869118+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-08-25T03:34:01.987016+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.f","mappings":[{"from":1,"to":5}]},{"pgid":"4.2b","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:34:56.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.437+0000 7f7f4cfdf700 1 -- 172.21.15.2:0/2383941374 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f7f2806a0e0 msgr2=0x7f7f2806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:56.474 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.437+0000 7f7f4cfdf700 1 --2- 172.21.15.2:0/2383941374 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f7f2806a0e0 0x7f7f2806c5a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7f481f3bc0 tx=0x7f7f3001c040 comp rx=0 tx=0).stop 2023-08-25T03:34:56.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.437+0000 7f7f4cfdf700 1 -- 172.21.15.2:0/2383941374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f480cca90 msgr2=0x7f7f4815dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:56.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.437+0000 7f7f4cfdf700 1 --2- 172.21.15.2:0/2383941374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f480cca90 0x7f7f4815dca0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7f3800c500 tx=0x7f7f3800c8c0 comp rx=0 tx=0).stop 2023-08-25T03:34:56.475 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.437+0000 7f7f4cfdf700 1 -- 172.21.15.2:0/2383941374 shutdown_connections 2023-08-25T03:34:56.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.437+0000 7f7f4cfdf700 1 --2- 172.21.15.2:0/2383941374 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f7f2806a0e0 0x7f7f2806c5a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:56.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.437+0000 7f7f4cfdf700 1 --2- 172.21.15.2:0/2383941374 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7f480cca90 0x7f7f4815dca0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:56.476 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.437+0000 7f7f4cfdf700 1 --2- 172.21.15.2:0/2383941374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7f4815ea20 0x7f7f48157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:56.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.437+0000 7f7f4cfdf700 1 --2- 172.21.15.2:0/2383941374 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7f480cfcc0 0x7f7f481515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:56.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.437+0000 7f7f4cfdf700 1 -- 172.21.15.2:0/2383941374 >> 172.21.15.2:0/2383941374 conn(0x7f7f4809a2b0 msgr2=0x7f7f4806e080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:56.477 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.437+0000 7f7f4cfdf700 1 -- 172.21.15.2:0/2383941374 shutdown_connections 2023-08-25T03:34:56.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.437+0000 7f7f4cfdf700 1 -- 172.21.15.2:0/2383941374 wait complete. 2023-08-25T03:34:56.479 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-25T03:34:56.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.614+0000 7fc0d2cbf700 1 Processor -- start 2023-08-25T03:34:56.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.614+0000 7fc0d2cbf700 1 -- start start 2023-08-25T03:34:56.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.614+0000 7fc0d2cbf700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc0cc15e890 0x7fc0cc071430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:56.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.614+0000 7fc0d2cbf700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fc0cc0caea0 0x7fc0cc0cb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:56.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.614+0000 7fc0d2cbf700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0cc0720b0 con 0x7fc0cc0b19a0 2023-08-25T03:34:56.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.614+0000 7fc0d2cbf700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0cc071dd0 con 0x7fc0cc0caea0 2023-08-25T03:34:56.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.614+0000 7fc0d2cbf700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc15ddb0 con 0x7fc0cc15e890 2023-08-25T03:34:56.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.615+0000 7fc0d125c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fc0cc0caea0 0x7fc0cc0cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:56.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.615+0000 7fc0d125c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fc0cc0caea0 0x7fc0cc0cb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32776/0 (socket says 172.21.15.2:32776) 2023-08-25T03:34:56.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.615+0000 7fc0d125c700 1 -- 172.21.15.2:0/273939833 learned_addr learned my addr 172.21.15.2:0/273939833 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:56.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.615+0000 7fc0cbfff700 1 --2- 172.21.15.2:0/273939833 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc0cc15e890 0x7fc0cc071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:56.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.615+0000 7fc0d125c700 1 -- 172.21.15.2:0/273939833 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc0cc15e890 msgr2=0x7fc0cc071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:56.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.615+0000 7fc0d125c700 1 --2- 172.21.15.2:0/273939833 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc0cc15e890 0x7fc0cc071430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:56.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.615+0000 7fc0d125c700 1 -- 172.21.15.2:0/273939833 >> v1:172.21.15.31:6789/0 conn(0x7fc0cc0b19a0 legacy=0x7fc0cc0b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:56.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.615+0000 7fc0d125c700 1 -- 172.21.15.2:0/273939833 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0cc15e090 con 0x7fc0cc0caea0 2023-08-25T03:34:56.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.615+0000 7fc0d125c700 1 --2- 172.21.15.2:0/273939833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0cc0caea0 0x7fc0cc0cb330 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009e50 tx=0x7fc0c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab52abacd3d1ca3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:56.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.615+0000 7fc0cb7fe700 1 -- 172.21.15.2:0/273939833 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c400e070 con 0x7fc0cc0caea0 2023-08-25T03:34:56.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.615+0000 7fc0cb7fe700 1 -- 172.21.15.2:0/273939833 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c4003a20 con 0x7fc0cc0caea0 2023-08-25T03:34:56.620 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.615+0000 7fc0cb7fe700 1 -- 172.21.15.2:0/273939833 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4003dd0 con 0x7fc0cc0caea0 2023-08-25T03:34:56.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.616+0000 7fc0d2cbf700 1 -- 172.21.15.2:0/273939833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0cc0caea0 msgr2=0x7fc0cc0cb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:56.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.616+0000 7fc0d2cbf700 1 --2- 172.21.15.2:0/273939833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0cc0caea0 0x7fc0cc0cb330 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4009e50 tx=0x7fc0c400b330 comp rx=0 tx=0).stop 2023-08-25T03:34:56.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.616+0000 7fc0d2cbf700 1 -- 172.21.15.2:0/273939833 shutdown_connections 2023-08-25T03:34:56.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.616+0000 7fc0d2cbf700 1 --2- 172.21.15.2:0/273939833 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0cc0caea0 0x7fc0cc0cb330 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:56.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.616+0000 7fc0d2cbf700 1 --2- 172.21.15.2:0/273939833 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc0cc15e890 0x7fc0cc071430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:56.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.616+0000 7fc0d2cbf700 1 -- 172.21.15.2:0/273939833 >> 172.21.15.2:0/273939833 conn(0x7fc0cc0cfea0 msgr2=0x7fc0cc0d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:56.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.616+0000 7fc0d2cbf700 1 -- 172.21.15.2:0/273939833 shutdown_connections 2023-08-25T03:34:56.621 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.616+0000 7fc0d2cbf700 1 -- 172.21.15.2:0/273939833 wait complete. 2023-08-25T03:34:56.623 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.617+0000 7fc0d2cbf700 1 Processor -- start 2023-08-25T03:34:56.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.617+0000 7fc0d2cbf700 1 -- start start 2023-08-25T03:34:56.624 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.618+0000 7fc0d2cbf700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc0cc0b19a0 0x7fc0cc0b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:56.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.618+0000 7fc0d2cbf700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0cc0caea0 0x7fc0cc153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:56.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.618+0000 7fc0d2cbf700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0cc15e890 0x7fc0cc155e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:56.625 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.618+0000 7fc0d2cbf700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc154a30 con 0x7fc0cc15e890 2023-08-25T03:34:56.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.618+0000 7fc0d2cbf700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc153b50 con 0x7fc0cc0caea0 2023-08-25T03:34:56.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.618+0000 7fc0d2cbf700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc153e20 con 0x7fc0cc0b19a0 2023-08-25T03:34:56.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.618+0000 7fc0d125c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0cc15e890 0x7fc0cc155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:56.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.618+0000 7fc0d125c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0cc15e890 0x7fc0cc155e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32782/0 (socket says 172.21.15.2:32782) 2023-08-25T03:34:56.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.618+0000 7fc0d125c700 1 -- 172.21.15.2:0/2095022131 learned_addr learned my addr 172.21.15.2:0/2095022131 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:56.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.618+0000 7fc0cbfff700 1 --2- 172.21.15.2:0/2095022131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0cc0caea0 0x7fc0cc153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:56.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.618+0000 7fc0d0a5b700 1 --2- 172.21.15.2:0/2095022131 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc0cc0b19a0 0x7fc0cc0b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:56.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.618+0000 7fc0d125c700 1 -- 172.21.15.2:0/2095022131 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc0cc0b19a0 msgr2=0x7fc0cc0b4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:56.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.618+0000 7fc0d125c700 1 --2- 172.21.15.2:0/2095022131 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc0cc0b19a0 0x7fc0cc0b4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:56.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.618+0000 7fc0d125c700 1 -- 172.21.15.2:0/2095022131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0cc0caea0 msgr2=0x7fc0cc153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:56.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.618+0000 7fc0d125c700 1 --2- 172.21.15.2:0/2095022131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0cc0caea0 0x7fc0cc153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:56.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.618+0000 7fc0d125c700 1 -- 172.21.15.2:0/2095022131 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c4009b30 con 0x7fc0cc15e890 2023-08-25T03:34:56.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.619+0000 7fc0d125c700 1 --2- 172.21.15.2:0/2095022131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0cc15e890 0x7fc0cc155e80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4015040 tx=0x7fc0c4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:56.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.619+0000 7fc0c97fa700 1 -- 172.21.15.2:0/2095022131 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4004440 con 0x7fc0cc15e890 2023-08-25T03:34:56.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.619+0000 7fc0c97fa700 1 -- 172.21.15.2:0/2095022131 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c40045a0 con 0x7fc0cc15e890 2023-08-25T03:34:56.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.619+0000 7fc0d2cbf700 1 -- 172.21.15.2:0/2095022131 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0cc154100 con 0x7fc0cc15e890 2023-08-25T03:34:56.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.619+0000 7fc0c97fa700 1 -- 172.21.15.2:0/2095022131 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4004950 con 0x7fc0cc15e890 2023-08-25T03:34:56.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.619+0000 7fc0d2cbf700 1 -- 172.21.15.2:0/2095022131 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0cc1546c0 con 0x7fc0cc15e890 2023-08-25T03:34:56.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.620+0000 7fc0d2cbf700 1 -- 172.21.15.2:0/2095022131 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0cc066f70 con 0x7fc0cc15e890 2023-08-25T03:34:56.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.620+0000 7fc0c97fa700 1 -- 172.21.15.2:0/2095022131 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc0c4012430 con 0x7fc0cc15e890 2023-08-25T03:34:56.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.620+0000 7fc0c97fa700 1 --2- 172.21.15.2:0/2095022131 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc0b006a0e0 0x7fc0b006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:56.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.621+0000 7fc0c97fa700 1 -- 172.21.15.2:0/2095022131 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6120+0+0 (secure 0 0 0) 0x7fc0c4089930 con 0x7fc0cc15e890 2023-08-25T03:34:56.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.621+0000 7fc0d0a5b700 1 --2- 172.21.15.2:0/2095022131 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc0b006a0e0 0x7fc0b006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:56.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.621+0000 7fc0d0a5b700 1 --2- 172.21.15.2:0/2095022131 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc0b006a0e0 0x7fc0b006c5a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0007fd0 tx=0x7fc0c0007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:56.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.625+0000 7fc0c97fa700 1 -- 172.21.15.2:0/2095022131 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc0c4058b00 con 0x7fc0cc15e890 2023-08-25T03:34:56.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.870+0000 7fc0d2cbf700 1 -- 172.21.15.2:0/2095022131 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc0cc0b2a40 con 0x7fc0cc15e890 2023-08-25T03:34:56.872 INFO:tasks.ceph.mon.a.smithi002.stderr:2023-08-25T03:34:56.871+0000 7efd0737a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-25T03:34:56.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.875+0000 7fc0c97fa700 1 -- 172.21.15.2:0/2095022131 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc0cc0b2a40 con 0x7fc0cc15e890 2023-08-25T03:34:56.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.877+0000 7fc0d2cbf700 1 -- 172.21.15.2:0/2095022131 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc0b006a0e0 msgr2=0x7fc0b006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:56.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.877+0000 7fc0d2cbf700 1 --2- 172.21.15.2:0/2095022131 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc0b006a0e0 0x7fc0b006c5a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0007fd0 tx=0x7fc0c0007ec0 comp rx=0 tx=0).stop 2023-08-25T03:34:56.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.877+0000 7fc0d2cbf700 1 -- 172.21.15.2:0/2095022131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0cc15e890 msgr2=0x7fc0cc155e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:56.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.877+0000 7fc0d2cbf700 1 --2- 172.21.15.2:0/2095022131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0cc15e890 0x7fc0cc155e80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4015040 tx=0x7fc0c4004ad0 comp rx=0 tx=0).stop 2023-08-25T03:34:56.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.877+0000 7fc0d2cbf700 1 -- 172.21.15.2:0/2095022131 shutdown_connections 2023-08-25T03:34:56.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.877+0000 7fc0d2cbf700 1 --2- 172.21.15.2:0/2095022131 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc0b006a0e0 0x7fc0b006c5a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:56.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.877+0000 7fc0d2cbf700 1 --2- 172.21.15.2:0/2095022131 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc0cc15e890 0x7fc0cc155e80 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:56.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.877+0000 7fc0d2cbf700 1 --2- 172.21.15.2:0/2095022131 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc0cc0b19a0 0x7fc0cc0b4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:56.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.877+0000 7fc0d2cbf700 1 --2- 172.21.15.2:0/2095022131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0cc0caea0 0x7fc0cc153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:56.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.877+0000 7fc0d2cbf700 1 -- 172.21.15.2:0/2095022131 >> 172.21.15.2:0/2095022131 conn(0x7fc0cc0cfea0 msgr2=0x7fc0cc0cd9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:56.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.877+0000 7fc0d2cbf700 1 -- 172.21.15.2:0/2095022131 shutdown_connections 2023-08-25T03:34:56.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:56.877+0000 7fc0d2cbf700 1 -- 172.21.15.2:0/2095022131 wait complete. 2023-08-25T03:34:56.881 INFO:teuthology.orchestra.run.smithi002.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-25T03:34:56.892 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-25T03:34:56.913 INFO:tasks.ceph.mds.a.smithi002.stdout:starting mds.a at 2023-08-25T03:34:57.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.019+0000 7f920b629700 1 Processor -- start 2023-08-25T03:34:57.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.019+0000 7f920b629700 1 -- start start 2023-08-25T03:34:57.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.020+0000 7f920b629700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f92040ae6b0 0x7f920406a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:57.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.020+0000 7f920b629700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92040b0f80 con 0x7f9204065fe0 2023-08-25T03:34:57.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.020+0000 7f920b629700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92040b1b00 con 0x7f9204072370 2023-08-25T03:34:57.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.020+0000 7f920b629700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920415e990 con 0x7f92040ae6b0 2023-08-25T03:34:57.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.020+0000 7f9209bc6700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f9204065fe0 0x7f92040b48c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47696/0 (socket says 172.21.15.2:47696) 2023-08-25T03:34:57.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.020+0000 7f9209bc6700 1 -- 172.21.15.2:0/2891222669 learned_addr learned my addr 172.21.15.2:0/2891222669 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:34:57.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.020+0000 7f92093c5700 1 --2- 172.21.15.2:0/2891222669 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f92040ae6b0 0x7f920406a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:57.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.020+0000 7f91fbfff700 1 -- 172.21.15.2:0/2891222669 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 284860789 0 0) 0x7f92040b0f80 con 0x7f9204065fe0 2023-08-25T03:34:57.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.020+0000 7f91fbfff700 1 -- 172.21.15.2:0/2891222669 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91f00035b0 con 0x7f9204065fe0 2023-08-25T03:34:57.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.020+0000 7f91fbfff700 1 -- 172.21.15.2:0/2891222669 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1929490409 0 0) 0x7f92040b1b00 con 0x7f9204072370 2023-08-25T03:34:57.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.020+0000 7f91fbfff700 1 -- 172.21.15.2:0/2891222669 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92040b0f80 con 0x7f9204072370 2023-08-25T03:34:57.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.020+0000 7f91fbfff700 1 -- 172.21.15.2:0/2891222669 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 762109573 0 0) 0x7f91f00035b0 con 0x7f9204065fe0 2023-08-25T03:34:57.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.020+0000 7f91fbfff700 1 -- 172.21.15.2:0/2891222669 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f92040ae6b0 msgr2=0x7f920406a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:57.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.020+0000 7f91fbfff700 1 --2- 172.21.15.2:0/2891222669 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f92040ae6b0 0x7f920406a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:57.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.020+0000 7f91fbfff700 1 -- 172.21.15.2:0/2891222669 >> v1:172.21.15.31:6789/0 conn(0x7f9204072370 legacy=0x7f92040b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:57.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.020+0000 7f91fbfff700 1 -- 172.21.15.2:0/2891222669 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92040b4fd0 con 0x7f9204065fe0 2023-08-25T03:34:57.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.020+0000 7f91fbfff700 1 -- 172.21.15.2:0/2891222669 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f91fc0019f0 con 0x7f9204065fe0 2023-08-25T03:34:57.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.020+0000 7f91fbfff700 1 -- 172.21.15.2:0/2891222669 >> v1:172.21.15.2:6789/0 conn(0x7f9204065fe0 legacy=0x7f92040b48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:57.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.020+0000 7f91fbfff700 1 --2- 172.21.15.2:0/2891222669 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f92040ae6b0 0x7f91f0003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:57.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.021+0000 7f91fbfff700 1 --2- 172.21.15.2:0/2891222669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91f0004120 0x7f91f0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:57.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.021+0000 7f91fbfff700 1 --2- 172.21.15.2:0/2891222669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f0004d90 0x7f91f0005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:57.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.021+0000 7f91fbfff700 1 -- 172.21.15.2:0/2891222669 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f920415e990 con 0x7f91f0004120 2023-08-25T03:34:57.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.021+0000 7f91fbfff700 1 -- 172.21.15.2:0/2891222669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91f000baf0 con 0x7f91f0004d90 2023-08-25T03:34:57.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.021+0000 7f91fbfff700 1 -- 172.21.15.2:0/2891222669 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91f000bcb0 con 0x7f92040ae6b0 2023-08-25T03:34:57.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.021+0000 7f9208bc4700 1 --2- 172.21.15.2:0/2891222669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91f0004120 0x7f91f0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:57.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.021+0000 7f92093c5700 1 --2- 172.21.15.2:0/2891222669 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f92040ae6b0 0x7f91f0003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:57.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.021+0000 7f9209bc6700 1 --2- 172.21.15.2:0/2891222669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f0004d90 0x7f91f0005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:57.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.021+0000 7f9208bc4700 1 -- 172.21.15.2:0/2891222669 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f92040ae6b0 msgr2=0x7f91f0003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:57.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.021+0000 7f9208bc4700 1 --2- 172.21.15.2:0/2891222669 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f92040ae6b0 0x7f91f0003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:57.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.021+0000 7f9208bc4700 1 -- 172.21.15.2:0/2891222669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f0004d90 msgr2=0x7f91f0005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:57.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.021+0000 7f9208bc4700 1 --2- 172.21.15.2:0/2891222669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f0004d90 0x7f91f0005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:57.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.021+0000 7f9208bc4700 1 -- 172.21.15.2:0/2891222669 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91f00035b0 con 0x7f91f0004120 2023-08-25T03:34:57.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.021+0000 7f9208bc4700 1 --2- 172.21.15.2:0/2891222669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91f0004120 0x7f91f0004850 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f92040727d0 tx=0x7f91f0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb87478aeeb99cba server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:57.027 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.021+0000 7f91fbfff700 1 -- 172.21.15.2:0/2891222669 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9200001430 con 0x7f91f0004120 2023-08-25T03:34:57.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.021+0000 7f91fbfff700 1 -- 172.21.15.2:0/2891222669 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92000052d0 con 0x7f91f0004120 2023-08-25T03:34:57.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.021+0000 7f91fbfff700 1 -- 172.21.15.2:0/2891222669 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9200001d40 con 0x7f91f0004120 2023-08-25T03:34:57.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.022+0000 7f920b629700 1 -- 172.21.15.2:0/2891222669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91f0004120 msgr2=0x7f91f0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:57.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.022+0000 7f920b629700 1 --2- 172.21.15.2:0/2891222669 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f91f0004120 0x7f91f0004850 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f92040727d0 tx=0x7f91f0005880 comp rx=0 tx=0).stop 2023-08-25T03:34:57.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.022+0000 7f9209bc6700 1 -- 172.21.15.2:0/2891222669 reap_dead start 2023-08-25T03:34:57.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.022+0000 7f920b629700 1 -- 172.21.15.2:0/2891222669 shutdown_connections 2023-08-25T03:34:57.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.022+0000 7f920b629700 1 -- 172.21.15.2:0/2891222669 >> 172.21.15.2:0/2891222669 conn(0x7f92040cf500 msgr2=0x7f92040cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:57.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.022+0000 7f920b629700 1 -- 172.21.15.2:0/2891222669 shutdown_connections 2023-08-25T03:34:57.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.022+0000 7f920b629700 1 -- 172.21.15.2:0/2891222669 wait complete. 2023-08-25T03:34:57.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.023+0000 7f920b629700 1 Processor -- start 2023-08-25T03:34:57.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.024+0000 7f920b629700 1 -- start start 2023-08-25T03:34:57.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.024+0000 7f920b629700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f0004120 0x7f92040ced90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:57.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.024+0000 7f920b629700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92040aea90 0x7f92040ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:57.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.025+0000 7f920b629700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f92040aef10 0x7f92040ce590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:57.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.025+0000 7f920b629700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92040cd040 con 0x7f92040aea90 2023-08-25T03:34:57.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.025+0000 7f920b629700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92040cdbc0 con 0x7f91f0004120 2023-08-25T03:34:57.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.025+0000 7f920b629700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f920414b490 con 0x7f92040aef10 2023-08-25T03:34:57.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.025+0000 7f9208bc4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92040aea90 0x7f92040ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:57.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.025+0000 7f9208bc4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92040aea90 0x7f92040ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32808/0 (socket says 172.21.15.2:32808) 2023-08-25T03:34:57.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.025+0000 7f9208bc4700 1 -- 172.21.15.2:0/3846735249 learned_addr learned my addr 172.21.15.2:0/3846735249 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:57.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.025+0000 7f9208bc4700 1 -- 172.21.15.2:0/3846735249 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f92040aef10 msgr2=0x7f92040ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:57.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.025+0000 7f9208bc4700 1 --2- 172.21.15.2:0/3846735249 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f92040aef10 0x7f92040ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:57.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.025+0000 7f9208bc4700 1 -- 172.21.15.2:0/3846735249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f0004120 msgr2=0x7f92040ced90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:57.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.025+0000 7f9208bc4700 1 --2- 172.21.15.2:0/3846735249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f0004120 0x7f92040ced90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:57.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.025+0000 7f9208bc4700 1 -- 172.21.15.2:0/3846735249 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92040cd290 con 0x7f92040aea90 2023-08-25T03:34:57.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.025+0000 7f9208bc4700 1 --2- 172.21.15.2:0/3846735249 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92040aea90 0x7f92040ce050 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9200006ac0 tx=0x7f92000012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:57.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.025+0000 7f91f9ffb700 1 -- 172.21.15.2:0/3846735249 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9200001d40 con 0x7f92040aea90 2023-08-25T03:34:57.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.025+0000 7f91f9ffb700 1 -- 172.21.15.2:0/3846735249 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92000013d0 con 0x7f92040aea90 2023-08-25T03:34:57.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.025+0000 7f920b629700 1 -- 172.21.15.2:0/3846735249 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92040cd570 con 0x7f92040aea90 2023-08-25T03:34:57.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.025+0000 7f91f9ffb700 1 -- 172.21.15.2:0/3846735249 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9200007280 con 0x7f92040aea90 2023-08-25T03:34:57.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.025+0000 7f920b629700 1 -- 172.21.15.2:0/3846735249 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92040b0410 con 0x7f92040aea90 2023-08-25T03:34:57.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.026+0000 7f920b629700 1 -- 172.21.15.2:0/3846735249 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92040746d0 con 0x7f92040aea90 2023-08-25T03:34:57.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.027+0000 7f91f9ffb700 1 -- 172.21.15.2:0/3846735249 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f92000073e0 con 0x7f92040aea90 2023-08-25T03:34:57.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.027+0000 7f91f9ffb700 1 --2- 172.21.15.2:0/3846735249 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f91e806a0e0 0x7f91e806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:57.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.027+0000 7f91f9ffb700 1 -- 172.21.15.2:0/3846735249 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6206+0+0 (secure 0 0 0) 0x7f92000894f0 con 0x7f92040aea90 2023-08-25T03:34:57.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.027+0000 7f92093c5700 1 --2- 172.21.15.2:0/3846735249 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f91e806a0e0 0x7f91e806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:57.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.028+0000 7f92093c5700 1 --2- 172.21.15.2:0/3846735249 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f91e806a0e0 0x7f91e806c5a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009a60 tx=0x7f91f4006c60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:57.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.032+0000 7f91f9ffb700 1 -- 172.21.15.2:0/3846735249 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9200054d00 con 0x7f92040aea90 2023-08-25T03:34:57.284 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.284+0000 7f920b629700 1 -- 172.21.15.2:0/3846735249 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f92040cdd00 con 0x7f92040aea90 2023-08-25T03:34:57.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.877+0000 7f91f9ffb700 1 -- 172.21.15.2:0/3846735249 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f92040cdd00 con 0x7f92040aea90 2023-08-25T03:34:57.880 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.880+0000 7f920b629700 1 -- 172.21.15.2:0/3846735249 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f91e806a0e0 msgr2=0x7f91e806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:57.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.880+0000 7f920b629700 1 --2- 172.21.15.2:0/3846735249 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f91e806a0e0 0x7f91e806c5a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f91f4009a60 tx=0x7f91f4006c60 comp rx=0 tx=0).stop 2023-08-25T03:34:57.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.880+0000 7f920b629700 1 -- 172.21.15.2:0/3846735249 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92040aea90 msgr2=0x7f92040ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:57.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.880+0000 7f920b629700 1 --2- 172.21.15.2:0/3846735249 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92040aea90 0x7f92040ce050 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9200006ac0 tx=0x7f92000012d0 comp rx=0 tx=0).stop 2023-08-25T03:34:57.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.880+0000 7f920b629700 1 -- 172.21.15.2:0/3846735249 shutdown_connections 2023-08-25T03:34:57.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.880+0000 7f920b629700 1 --2- 172.21.15.2:0/3846735249 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f92040aef10 0x7f92040ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:57.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.880+0000 7f920b629700 1 --2- 172.21.15.2:0/3846735249 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91f0004120 0x7f92040ced90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.880+0000 7f920b629700 1 --2- 172.21.15.2:0/3846735249 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f91e806a0e0 0x7f91e806c5a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.880+0000 7f920b629700 1 --2- 172.21.15.2:0/3846735249 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92040aea90 0x7f92040ce050 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.880+0000 7f920b629700 1 -- 172.21.15.2:0/3846735249 >> 172.21.15.2:0/3846735249 conn(0x7f92040cf500 msgr2=0x7f9204157190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.880+0000 7f920b629700 1 -- 172.21.15.2:0/3846735249 shutdown_connections 2023-08-25T03:34:57.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:57.880+0000 7f920b629700 1 -- 172.21.15.2:0/3846735249 wait complete. 2023-08-25T03:34:57.896 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-25T03:34:58.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.074+0000 7fc5d707b700 1 Processor -- start 2023-08-25T03:34:58.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.074+0000 7fc5d707b700 1 -- start start 2023-08-25T03:34:58.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.075+0000 7fc5d707b700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc5d00cadf0 0x7fc5d00ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:58.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.075+0000 7fc5d707b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc5d006a1e0 0x7fc5d006a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:58.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.075+0000 7fc5d707b700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d006ab30 con 0x7fc5d00cb1d0 2023-08-25T03:34:58.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.075+0000 7fc5d707b700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d00cddc0 con 0x7fc5d006a1e0 2023-08-25T03:34:58.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.075+0000 7fc5d707b700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d0161a50 con 0x7fc5d00cadf0 2023-08-25T03:34:58.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.075+0000 7fc5d5618700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc5d006a1e0 0x7fc5d006a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:58.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.075+0000 7fc5d4e17700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc5d00cadf0 0x7fc5d00ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:58.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.075+0000 7fc5d5618700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc5d006a1e0 0x7fc5d006a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.2:52342/0 (socket says 172.21.15.2:52342) 2023-08-25T03:34:58.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.075+0000 7fc5cffff700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fc5d00cb1d0 0x7fc5d0160f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.2:55738/0 (socket says 172.21.15.2:55738) 2023-08-25T03:34:58.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.075+0000 7fc5d4e17700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc5d00cadf0 0x7fc5d00ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.2:33060/0 (socket says 172.21.15.2:33060) 2023-08-25T03:34:58.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.075+0000 7fc5d5618700 1 -- 172.21.15.2:0/3970814508 learned_addr learned my addr 172.21.15.2:0/3970814508 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:58.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.076+0000 7fc5d4e17700 1 -- 172.21.15.2:0/3970814508 >> v2:172.21.15.31:3300/0 conn(0x7fc5d006a1e0 msgr2=0x7fc5d006a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:58.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.076+0000 7fc5d4e17700 1 --2- 172.21.15.2:0/3970814508 >> v2:172.21.15.31:3300/0 conn(0x7fc5d006a1e0 0x7fc5d006a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:58.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.076+0000 7fc5d4e17700 1 -- 172.21.15.2:0/3970814508 >> v1:172.21.15.31:6789/0 conn(0x7fc5d00cb1d0 legacy=0x7fc5d0160f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:58.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.076+0000 7fc5d4e17700 1 -- 172.21.15.2:0/3970814508 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d0161d30 con 0x7fc5d00cadf0 2023-08-25T03:34:58.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.076+0000 7fc5d5618700 1 --2- 172.21.15.2:0/3970814508 >> v2:172.21.15.31:3300/0 conn(0x7fc5d006a1e0 0x7fc5d006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:34:58.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.076+0000 7fc5d4e17700 1 --2- 172.21.15.2:0/3970814508 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc5d00cadf0 0x7fc5d00ce860 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009840 tx=0x7fc5c4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77d4134ea930c4ff server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:58.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.076+0000 7fc5cf7fe700 1 -- 172.21.15.2:0/3970814508 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4011830 con 0x7fc5d00cadf0 2023-08-25T03:34:58.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.076+0000 7fc5cf7fe700 1 -- 172.21.15.2:0/3970814508 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c4011990 con 0x7fc5d00cadf0 2023-08-25T03:34:58.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.076+0000 7fc5cf7fe700 1 -- 172.21.15.2:0/3970814508 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4011d20 con 0x7fc5d00cadf0 2023-08-25T03:34:58.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.077+0000 7fc5d707b700 1 -- 172.21.15.2:0/3970814508 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc5d00cadf0 msgr2=0x7fc5d00ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:58.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.077+0000 7fc5d707b700 1 --2- 172.21.15.2:0/3970814508 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc5d00cadf0 0x7fc5d00ce860 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4009840 tx=0x7fc5c4009b50 comp rx=0 tx=0).stop 2023-08-25T03:34:58.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.077+0000 7fc5d707b700 1 -- 172.21.15.2:0/3970814508 shutdown_connections 2023-08-25T03:34:58.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.077+0000 7fc5d707b700 1 --2- 172.21.15.2:0/3970814508 >> v2:172.21.15.31:3300/0 conn(0x7fc5d006a1e0 0x7fc5d006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:58.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.077+0000 7fc5d707b700 1 --2- 172.21.15.2:0/3970814508 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc5d00cadf0 0x7fc5d00ce860 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:58.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.077+0000 7fc5d707b700 1 -- 172.21.15.2:0/3970814508 >> 172.21.15.2:0/3970814508 conn(0x7fc5d00b1990 msgr2=0x7fc5d00b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:58.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.077+0000 7fc5d707b700 1 -- 172.21.15.2:0/3970814508 shutdown_connections 2023-08-25T03:34:58.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.077+0000 7fc5d707b700 1 -- 172.21.15.2:0/3970814508 wait complete. 2023-08-25T03:34:58.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.078+0000 7fc5d707b700 1 Processor -- start 2023-08-25T03:34:58.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.078+0000 7fc5d707b700 1 -- start start 2023-08-25T03:34:58.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.079+0000 7fc5d707b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5d006a1e0 0x7fc5d00b5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:58.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.079+0000 7fc5d707b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5d00cadf0 0x7fc5d01f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:58.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.079+0000 7fc5d707b700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc5d00cb1d0 0x7fc5d00b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:58.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.079+0000 7fc5d707b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d01f32d0 con 0x7fc5d00cadf0 2023-08-25T03:34:58.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.079+0000 7fc5d707b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d01f40d0 con 0x7fc5d006a1e0 2023-08-25T03:34:58.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.079+0000 7fc5d707b700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d01f3570 con 0x7fc5d00cb1d0 2023-08-25T03:34:58.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.079+0000 7fc5cffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5d00cadf0 0x7fc5d01f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:58.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.079+0000 7fc5cffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5d00cadf0 0x7fc5d01f2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32824/0 (socket says 172.21.15.2:32824) 2023-08-25T03:34:58.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.079+0000 7fc5cffff700 1 -- 172.21.15.2:0/3589800365 learned_addr learned my addr 172.21.15.2:0/3589800365 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:58.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.079+0000 7fc5d5618700 1 --2- 172.21.15.2:0/3589800365 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc5d00cb1d0 0x7fc5d00b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:58.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.079+0000 7fc5d4e17700 1 --2- 172.21.15.2:0/3589800365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5d006a1e0 0x7fc5d00b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:58.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.079+0000 7fc5cffff700 1 -- 172.21.15.2:0/3589800365 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc5d00cb1d0 msgr2=0x7fc5d00b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:58.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.079+0000 7fc5cffff700 1 --2- 172.21.15.2:0/3589800365 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc5d00cb1d0 0x7fc5d00b3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:58.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.079+0000 7fc5cffff700 1 -- 172.21.15.2:0/3589800365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5d006a1e0 msgr2=0x7fc5d00b5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:58.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.080+0000 7fc5cffff700 1 --2- 172.21.15.2:0/3589800365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5d006a1e0 0x7fc5d00b5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:58.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.080+0000 7fc5cffff700 1 -- 172.21.15.2:0/3589800365 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c8009b30 con 0x7fc5d00cadf0 2023-08-25T03:34:58.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.080+0000 7fc5d4e17700 1 --2- 172.21.15.2:0/3589800365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5d006a1e0 0x7fc5d00b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:34:58.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.080+0000 7fc5cffff700 1 --2- 172.21.15.2:0/3589800365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5d00cadf0 0x7fc5d01f2d90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00c430 tx=0x7fc5bc00c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:58.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.080+0000 7fc5cd7fa700 1 -- 172.21.15.2:0/3589800365 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc005ab0 con 0x7fc5d00cadf0 2023-08-25T03:34:58.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.080+0000 7fc5cd7fa700 1 -- 172.21.15.2:0/3589800365 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5bc005c10 con 0x7fc5d00cadf0 2023-08-25T03:34:58.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.080+0000 7fc5d707b700 1 -- 172.21.15.2:0/3589800365 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5c40094a0 con 0x7fc5d00cadf0 2023-08-25T03:34:58.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.080+0000 7fc5cd7fa700 1 -- 172.21.15.2:0/3589800365 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc0070d0 con 0x7fc5d00cadf0 2023-08-25T03:34:58.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.080+0000 7fc5d707b700 1 -- 172.21.15.2:0/3589800365 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d01f3b30 con 0x7fc5d00cadf0 2023-08-25T03:34:58.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.081+0000 7fc5d707b700 1 -- 172.21.15.2:0/3589800365 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d0066f70 con 0x7fc5d00cadf0 2023-08-25T03:34:58.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.081+0000 7fc5cd7fa700 1 -- 172.21.15.2:0/3589800365 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc5bc007230 con 0x7fc5d00cadf0 2023-08-25T03:34:58.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.082+0000 7fc5cd7fa700 1 --2- 172.21.15.2:0/3589800365 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc5b406a0e0 0x7fc5b406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:58.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.082+0000 7fc5cd7fa700 1 -- 172.21.15.2:0/3589800365 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6206+0+0 (secure 0 0 0) 0x7fc5bc087770 con 0x7fc5d00cadf0 2023-08-25T03:34:58.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.082+0000 7fc5d4e17700 1 --2- 172.21.15.2:0/3589800365 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc5b406a0e0 0x7fc5b406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:58.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.082+0000 7fc5d4e17700 1 --2- 172.21.15.2:0/3589800365 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc5b406a0e0 0x7fc5b406c5a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4000c00 tx=0x7fc5c4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:58.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.087+0000 7fc5cd7fa700 1 -- 172.21.15.2:0/3589800365 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc5bc052e70 con 0x7fc5d00cadf0 2023-08-25T03:34:58.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.289+0000 7fc5d707b700 1 -- 172.21.15.2:0/3589800365 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc5d00746d0 con 0x7fc5d00cadf0 2023-08-25T03:34:58.881 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.880+0000 7fc5cd7fa700 1 -- 172.21.15.2:0/3589800365 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc5bc056490 con 0x7fc5d00cadf0 2023-08-25T03:34:58.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.884+0000 7fc5d707b700 1 -- 172.21.15.2:0/3589800365 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc5b406a0e0 msgr2=0x7fc5b406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:58.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.884+0000 7fc5d707b700 1 --2- 172.21.15.2:0/3589800365 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc5b406a0e0 0x7fc5b406c5a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4000c00 tx=0x7fc5c4008030 comp rx=0 tx=0).stop 2023-08-25T03:34:58.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.884+0000 7fc5d707b700 1 -- 172.21.15.2:0/3589800365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5d00cadf0 msgr2=0x7fc5d01f2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:58.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.884+0000 7fc5d707b700 1 --2- 172.21.15.2:0/3589800365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5d00cadf0 0x7fc5d01f2d90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00c430 tx=0x7fc5bc00c740 comp rx=0 tx=0).stop 2023-08-25T03:34:58.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.884+0000 7fc5d707b700 1 -- 172.21.15.2:0/3589800365 shutdown_connections 2023-08-25T03:34:58.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.884+0000 7fc5d707b700 1 --2- 172.21.15.2:0/3589800365 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc5d00cb1d0 0x7fc5d00b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:58.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.884+0000 7fc5d707b700 1 --2- 172.21.15.2:0/3589800365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5d006a1e0 0x7fc5d00b5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:58.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.884+0000 7fc5d707b700 1 --2- 172.21.15.2:0/3589800365 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc5b406a0e0 0x7fc5b406c5a0 secure :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4000c00 tx=0x7fc5c4008030 comp rx=0 tx=0).stop 2023-08-25T03:34:58.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.884+0000 7fc5d707b700 1 --2- 172.21.15.2:0/3589800365 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc5d00cadf0 0x7fc5d01f2d90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:58.887 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.884+0000 7fc5d707b700 1 -- 172.21.15.2:0/3589800365 >> 172.21.15.2:0/3589800365 conn(0x7fc5d00b1990 msgr2=0x7fc5d00ace80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:58.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.884+0000 7fc5d707b700 1 -- 172.21.15.2:0/3589800365 shutdown_connections 2023-08-25T03:34:58.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:58.884+0000 7fc5d707b700 1 -- 172.21.15.2:0/3589800365 wait complete. 2023-08-25T03:34:58.889 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_metadata' removed 2023-08-25T03:34:58.906 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-25T03:34:59.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.041+0000 7f6e4bef9700 1 Processor -- start 2023-08-25T03:34:59.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.041+0000 7f6e4bef9700 1 -- start start 2023-08-25T03:34:59.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.042+0000 7f6e4bef9700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6e440cb1d0 0x7f6e44160f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:59.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.042+0000 7f6e4bef9700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6e441614d0 0x7f6e4406a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:59.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.042+0000 7f6e4bef9700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e440cddc0 con 0x7f6e440cadf0 2023-08-25T03:34:59.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.042+0000 7f6e4bef9700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e44161a50 con 0x7f6e440cb1d0 2023-08-25T03:34:59.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.042+0000 7f6e4bef9700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e44161d30 con 0x7f6e441614d0 2023-08-25T03:34:59.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.043+0000 7f6e4a496700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6e441614d0 0x7f6e4406a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:59.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.043+0000 7f6e49494700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6e440cb1d0 0x7f6e44160f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:59.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.043+0000 7f6e4a496700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6e441614d0 0x7f6e4406a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.2:33078/0 (socket says 172.21.15.2:33078) 2023-08-25T03:34:59.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.043+0000 7f6e49c95700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f6e440cadf0 0x7f6e440ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.2:55752/0 (socket says 172.21.15.2:55752) 2023-08-25T03:34:59.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.043+0000 7f6e49494700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6e440cb1d0 0x7f6e44160f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.2:52354/0 (socket says 172.21.15.2:52354) 2023-08-25T03:34:59.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.043+0000 7f6e4a496700 1 -- 172.21.15.2:0/1846051861 learned_addr learned my addr 172.21.15.2:0/1846051861 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:59.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.043+0000 7f6e4a496700 1 -- 172.21.15.2:0/1846051861 >> v2:172.21.15.31:3300/0 conn(0x7f6e440cb1d0 msgr2=0x7f6e44160f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:34:59.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.043+0000 7f6e4a496700 1 --2- 172.21.15.2:0/1846051861 >> v2:172.21.15.31:3300/0 conn(0x7f6e440cb1d0 0x7f6e44160f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:59.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.043+0000 7f6e4a496700 1 -- 172.21.15.2:0/1846051861 >> v1:172.21.15.31:6789/0 conn(0x7f6e440cadf0 legacy=0x7f6e440ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:59.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.043+0000 7f6e4a496700 1 -- 172.21.15.2:0/1846051861 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e440ce080 con 0x7f6e441614d0 2023-08-25T03:34:59.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.043+0000 7f6e49494700 1 --2- 172.21.15.2:0/1846051861 >> v2:172.21.15.31:3300/0 conn(0x7f6e440cb1d0 0x7f6e44160f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:34:59.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.043+0000 7f6e4a496700 1 --2- 172.21.15.2:0/1846051861 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6e441614d0 0x7f6e4406a1e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009e50 tx=0x7f6e3c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a271222ed9479c7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:59.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.043+0000 7f6e48c93700 1 -- 172.21.15.2:0/1846051861 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c00e070 con 0x7f6e441614d0 2023-08-25T03:34:59.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.044+0000 7f6e48c93700 1 -- 172.21.15.2:0/1846051861 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3c00b910 con 0x7f6e441614d0 2023-08-25T03:34:59.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.044+0000 7f6e48c93700 1 -- 172.21.15.2:0/1846051861 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c00bcc0 con 0x7f6e441614d0 2023-08-25T03:34:59.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.044+0000 7f6e4bef9700 1 -- 172.21.15.2:0/1846051861 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6e441614d0 msgr2=0x7f6e4406a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:59.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.044+0000 7f6e4bef9700 1 --2- 172.21.15.2:0/1846051861 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6e441614d0 0x7f6e4406a1e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c009e50 tx=0x7f6e3c00b330 comp rx=0 tx=0).stop 2023-08-25T03:34:59.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.044+0000 7f6e4bef9700 1 -- 172.21.15.2:0/1846051861 shutdown_connections 2023-08-25T03:34:59.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.044+0000 7f6e4bef9700 1 --2- 172.21.15.2:0/1846051861 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6e441614d0 0x7f6e4406a1e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:59.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.044+0000 7f6e4bef9700 1 --2- 172.21.15.2:0/1846051861 >> v2:172.21.15.31:3300/0 conn(0x7f6e440cb1d0 0x7f6e44160f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:59.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.044+0000 7f6e4bef9700 1 -- 172.21.15.2:0/1846051861 >> 172.21.15.2:0/1846051861 conn(0x7f6e440b1990 msgr2=0x7f6e440b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:59.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.044+0000 7f6e4bef9700 1 -- 172.21.15.2:0/1846051861 shutdown_connections 2023-08-25T03:34:59.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.044+0000 7f6e4bef9700 1 -- 172.21.15.2:0/1846051861 wait complete. 2023-08-25T03:34:59.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.045+0000 7f6e4bef9700 1 Processor -- start 2023-08-25T03:34:59.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.045+0000 7f6e4bef9700 1 -- start start 2023-08-25T03:34:59.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.046+0000 7f6e4bef9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e440cadf0 0x7f6e44154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:59.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.046+0000 7f6e4bef9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e440cb1d0 0x7f6e4415d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:59.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.046+0000 7f6e4bef9700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6e441614d0 0x7f6e440c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:59.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.046+0000 7f6e4bef9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e440c9dd0 con 0x7f6e440cadf0 2023-08-25T03:34:59.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.046+0000 7f6e4bef9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4415de60 con 0x7f6e440cb1d0 2023-08-25T03:34:59.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.046+0000 7f6e4bef9700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4415e140 con 0x7f6e441614d0 2023-08-25T03:34:59.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.046+0000 7f6e49c95700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e440cadf0 0x7f6e44154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:59.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.046+0000 7f6e49c95700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e440cadf0 0x7f6e44154f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32836/0 (socket says 172.21.15.2:32836) 2023-08-25T03:34:59.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.046+0000 7f6e49c95700 1 -- 172.21.15.2:0/1376904634 learned_addr learned my addr 172.21.15.2:0/1376904634 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:34:59.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.046+0000 7f6e49494700 1 --2- 172.21.15.2:0/1376904634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e440cb1d0 0x7f6e4415d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:59.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.046+0000 7f6e4a496700 1 --2- 172.21.15.2:0/1376904634 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6e441614d0 0x7f6e440c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:59.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.047+0000 7f6e49c95700 1 -- 172.21.15.2:0/1376904634 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6e441614d0 msgr2=0x7f6e440c37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:59.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.047+0000 7f6e49c95700 1 --2- 172.21.15.2:0/1376904634 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6e441614d0 0x7f6e440c37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:59.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.047+0000 7f6e49c95700 1 -- 172.21.15.2:0/1376904634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e440cb1d0 msgr2=0x7f6e4415d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:59.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.047+0000 7f6e49c95700 1 --2- 172.21.15.2:0/1376904634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e440cb1d0 0x7f6e4415d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:59.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.047+0000 7f6e49c95700 1 -- 172.21.15.2:0/1376904634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e400094a0 con 0x7f6e440cadf0 2023-08-25T03:34:59.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.047+0000 7f6e49494700 1 --2- 172.21.15.2:0/1376904634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e440cb1d0 0x7f6e4415d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:34:59.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.047+0000 7f6e49c95700 1 --2- 172.21.15.2:0/1376904634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e440cadf0 0x7f6e44154f40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6e3400c520 tx=0x7f6e3400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:59.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.047+0000 7f6e3a7fc700 1 -- 172.21.15.2:0/1376904634 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e34005cb0 con 0x7f6e440cadf0 2023-08-25T03:34:59.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.047+0000 7f6e3a7fc700 1 -- 172.21.15.2:0/1376904634 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e34005e10 con 0x7f6e440cadf0 2023-08-25T03:34:59.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.047+0000 7f6e4bef9700 1 -- 172.21.15.2:0/1376904634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e3c009b30 con 0x7f6e440cadf0 2023-08-25T03:34:59.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.047+0000 7f6e3a7fc700 1 -- 172.21.15.2:0/1376904634 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e340072d0 con 0x7f6e440cadf0 2023-08-25T03:34:59.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.047+0000 7f6e4bef9700 1 -- 172.21.15.2:0/1376904634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e440b2f20 con 0x7f6e440cadf0 2023-08-25T03:34:59.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.048+0000 7f6e4bef9700 1 -- 172.21.15.2:0/1376904634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e44066f70 con 0x7f6e440cadf0 2023-08-25T03:34:59.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.048+0000 7f6e3a7fc700 1 -- 172.21.15.2:0/1376904634 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6e34007430 con 0x7f6e440cadf0 2023-08-25T03:34:59.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.049+0000 7f6e3a7fc700 1 --2- 172.21.15.2:0/1376904634 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6e2806a0e0 0x7f6e2806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:34:59.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.049+0000 7f6e3a7fc700 1 -- 172.21.15.2:0/1376904634 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5768+0+0 (secure 0 0 0) 0x7f6e34087710 con 0x7f6e440cadf0 2023-08-25T03:34:59.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.049+0000 7f6e49494700 1 --2- 172.21.15.2:0/1376904634 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6e2806a0e0 0x7f6e2806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:34:59.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.049+0000 7f6e49494700 1 --2- 172.21.15.2:0/1376904634 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6e2806a0e0 0x7f6e2806c5a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6e44155d20 tx=0x7f6e40008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:34:59.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.053+0000 7f6e3a7fc700 1 -- 172.21.15.2:0/1376904634 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6e34053060 con 0x7f6e440cadf0 2023-08-25T03:34:59.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.255+0000 7f6e4bef9700 1 -- 172.21.15.2:0/1376904634 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6e440746d0 con 0x7f6e440cadf0 2023-08-25T03:34:59.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.885+0000 7f6e3a7fc700 1 -- 172.21.15.2:0/1376904634 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6e34001250 con 0x7f6e440cadf0 2023-08-25T03:34:59.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.890+0000 7f6e4bef9700 1 -- 172.21.15.2:0/1376904634 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6e2806a0e0 msgr2=0x7f6e2806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:59.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.890+0000 7f6e4bef9700 1 --2- 172.21.15.2:0/1376904634 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6e2806a0e0 0x7f6e2806c5a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6e44155d20 tx=0x7f6e40008040 comp rx=0 tx=0).stop 2023-08-25T03:34:59.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.890+0000 7f6e4bef9700 1 -- 172.21.15.2:0/1376904634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e440cadf0 msgr2=0x7f6e44154f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:34:59.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.890+0000 7f6e4bef9700 1 --2- 172.21.15.2:0/1376904634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e440cadf0 0x7f6e44154f40 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6e3400c520 tx=0x7f6e3400c830 comp rx=0 tx=0).stop 2023-08-25T03:34:59.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.890+0000 7f6e4bef9700 1 -- 172.21.15.2:0/1376904634 shutdown_connections 2023-08-25T03:34:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.890+0000 7f6e4bef9700 1 --2- 172.21.15.2:0/1376904634 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6e2806a0e0 0x7f6e2806c5a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.890+0000 7f6e4bef9700 1 --2- 172.21.15.2:0/1376904634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e440cadf0 0x7f6e44154f40 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.890+0000 7f6e4bef9700 1 --2- 172.21.15.2:0/1376904634 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6e441614d0 0x7f6e440c37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:59.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.890+0000 7f6e4bef9700 1 --2- 172.21.15.2:0/1376904634 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e440cb1d0 0x7f6e4415d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:34:59.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.890+0000 7f6e4bef9700 1 -- 172.21.15.2:0/1376904634 >> 172.21.15.2:0/1376904634 conn(0x7f6e440b1990 msgr2=0x7f6e4406dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:34:59.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.890+0000 7f6e4bef9700 1 -- 172.21.15.2:0/1376904634 shutdown_connections 2023-08-25T03:34:59.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:34:59.890+0000 7f6e4bef9700 1 -- 172.21.15.2:0/1376904634 wait complete. 2023-08-25T03:34:59.894 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_data' removed 2023-08-25T03:34:59.907 INFO:tasks.ceph.mds.a:Restarting daemon 2023-08-25T03:34:59.907 INFO:tasks.ceph.mds.a:Stopping old one... 2023-08-25T03:34:59.907 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-25T03:34:59.908 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T03:34:59.908 INFO:tasks.ceph.mds.c:Restarting daemon 2023-08-25T03:34:59.908 INFO:tasks.ceph.mds.c:Stopping old one... 2023-08-25T03:34:59.909 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-25T03:34:59.909 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T03:34:59.910 INFO:tasks.ceph.mds.b:Restarting daemon 2023-08-25T03:34:59.910 INFO:tasks.ceph.mds.b:Stopping old one... 2023-08-25T03:34:59.910 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-25T03:34:59.911 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T03:34:59.911 INFO:tasks.ceph.mds.d:Restarting daemon 2023-08-25T03:34:59.912 INFO:tasks.ceph.mds.d:Stopping old one... 2023-08-25T03:34:59.912 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-25T03:34:59.912 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T03:34:59.918 INFO:tasks.ceph.mds.a:Stopped 2023-08-25T03:34:59.918 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-08-25T03:34:59.919 INFO:tasks.ceph.mds.c:Stopped 2023-08-25T03:34:59.919 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-08-25T03:34:59.922 INFO:tasks.ceph.mds.b:Stopped 2023-08-25T03:34:59.922 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-08-25T03:34:59.924 INFO:tasks.ceph.mds.d:Stopped 2023-08-25T03:34:59.924 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-08-25T03:34:59.961 INFO:tasks.ceph.mds.a:Started 2023-08-25T03:34:59.962 DEBUG:teuthology.parallel:result is None 2023-08-25T03:34:59.963 INFO:tasks.ceph.mds.c:Started 2023-08-25T03:34:59.963 DEBUG:teuthology.parallel:result is None 2023-08-25T03:34:59.966 INFO:tasks.ceph.mds.b:Started 2023-08-25T03:34:59.967 DEBUG:teuthology.parallel:result is None 2023-08-25T03:34:59.968 INFO:tasks.ceph.mds.d:Started 2023-08-25T03:34:59.968 DEBUG:teuthology.parallel:result is None 2023-08-25T03:34:59.969 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-08-25T03:35:00.046 INFO:tasks.ceph.mds.a.smithi002.stdout:starting mds.a at 2023-08-25T03:35:00.047 INFO:tasks.ceph.mds.c.smithi002.stdout:starting mds.c at 2023-08-25T03:35:00.047 INFO:tasks.ceph.mds.c.smithi002.stderr:2023-08-25T03:35:00.047+0000 7f07949dc700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-08-25T03:35:00.059 INFO:tasks.ceph.mds.b.smithi031.stdout:starting mds.b at 2023-08-25T03:35:00.059 INFO:tasks.ceph.mds.d.smithi031.stdout:starting mds.d at 2023-08-25T03:35:00.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.103+0000 7f9ffbc5d700 1 Processor -- start 2023-08-25T03:35:00.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.103+0000 7f9ffbc5d700 1 -- start start 2023-08-25T03:35:00.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.104+0000 7f9ffbc5d700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f9ff40b8aa0 0x7f9ff40cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:00.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.104+0000 7f9ffbc5d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9ff40d2b40 0x7f9ff40ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:00.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.104+0000 7f9ffbc5d700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ff415d750 con 0x7f9ff415d370 2023-08-25T03:35:00.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.104+0000 7f9ffbc5d700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff40d1aa0 con 0x7f9ff40b8aa0 2023-08-25T03:35:00.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.104+0000 7f9ffbc5d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff40d1d80 con 0x7f9ff40d2b40 2023-08-25T03:35:00.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.104+0000 7f9ff91f8700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f9ff40b8aa0 0x7f9ff40cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:00.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.104+0000 7f9ff91f8700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f9ff40b8aa0 0x7f9ff40cdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32870/0 (socket says 172.21.15.2:32870) 2023-08-25T03:35:00.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.104+0000 7f9ff99f9700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f9ff415d370 0x7f9ff40d1250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47766/0 (socket says 172.21.15.2:47766) 2023-08-25T03:35:00.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.104+0000 7f9ff91f8700 1 -- 172.21.15.2:0/3178540166 learned_addr learned my addr 172.21.15.2:0/3178540166 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:00.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.104+0000 7f9ffa1fa700 1 --2- 172.21.15.2:0/3178540166 >> v2:172.21.15.31:3300/0 conn(0x7f9ff40d2b40 0x7f9ff40ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:00.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.104+0000 7f9ff91f8700 1 -- 172.21.15.2:0/3178540166 >> v2:172.21.15.31:3300/0 conn(0x7f9ff40d2b40 msgr2=0x7f9ff40ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:00.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.104+0000 7f9ff91f8700 1 --2- 172.21.15.2:0/3178540166 >> v2:172.21.15.31:3300/0 conn(0x7f9ff40d2b40 0x7f9ff40ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:00.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.104+0000 7f9ff91f8700 1 -- 172.21.15.2:0/3178540166 >> v1:172.21.15.2:6789/0 conn(0x7f9ff415d370 legacy=0x7f9ff40d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:00.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.104+0000 7f9ff91f8700 1 -- 172.21.15.2:0/3178540166 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff40d2060 con 0x7f9ff40b8aa0 2023-08-25T03:35:00.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.104+0000 7f9ff89f7700 1 -- 172.21.15.2:0/3178540166 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1249054214 0 0) 0x7f9ff415d750 con 0x7f9ff415d370 2023-08-25T03:35:00.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.104+0000 7f9ff91f8700 1 --2- 172.21.15.2:0/3178540166 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ff40b8aa0 0x7f9ff40cdde0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9ff406bbf0 tx=0x7f9fe4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36139a102981b50a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:00.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.105+0000 7f9ff89f7700 1 -- 172.21.15.2:0/3178540166 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4011830 con 0x7f9ff40b8aa0 2023-08-25T03:35:00.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.105+0000 7f9ff89f7700 1 -- 172.21.15.2:0/3178540166 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe4011990 con 0x7f9ff40b8aa0 2023-08-25T03:35:00.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.105+0000 7f9ff89f7700 1 -- 172.21.15.2:0/3178540166 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4011d20 con 0x7f9ff40b8aa0 2023-08-25T03:35:00.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.105+0000 7f9ffbc5d700 1 -- 172.21.15.2:0/3178540166 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ff40b8aa0 msgr2=0x7f9ff40cdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:00.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.105+0000 7f9ffbc5d700 1 --2- 172.21.15.2:0/3178540166 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ff40b8aa0 0x7f9ff40cdde0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f9ff406bbf0 tx=0x7f9fe4009bd0 comp rx=0 tx=0).stop 2023-08-25T03:35:00.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.105+0000 7f9ffbc5d700 1 -- 172.21.15.2:0/3178540166 shutdown_connections 2023-08-25T03:35:00.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.105+0000 7f9ffbc5d700 1 --2- 172.21.15.2:0/3178540166 >> v2:172.21.15.31:3300/0 conn(0x7f9ff40d2b40 0x7f9ff40ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:00.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.105+0000 7f9ffbc5d700 1 --2- 172.21.15.2:0/3178540166 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ff40b8aa0 0x7f9ff40cdde0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:00.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.105+0000 7f9ffbc5d700 1 -- 172.21.15.2:0/3178540166 >> 172.21.15.2:0/3178540166 conn(0x7f9ff406ab50 msgr2=0x7f9ff406af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:00.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.105+0000 7f9ffbc5d700 1 -- 172.21.15.2:0/3178540166 shutdown_connections 2023-08-25T03:35:00.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.105+0000 7f9ffbc5d700 1 -- 172.21.15.2:0/3178540166 wait complete. 2023-08-25T03:35:00.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.107+0000 7f9ffbc5d700 1 Processor -- start 2023-08-25T03:35:00.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.107+0000 7f9ffbc5d700 1 -- start start 2023-08-25T03:35:00.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.107+0000 7f9ffbc5d700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9ff40b8aa0 0x7f9ff40ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:00.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.107+0000 7f9ffbc5d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ff40d2b40 0x7f9ff4155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:00.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.107+0000 7f9ffbc5d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ff415d370 0x7f9ff40ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:00.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.107+0000 7f9ffbc5d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff41562b0 con 0x7f9ff415d370 2023-08-25T03:35:00.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.107+0000 7f9ffbc5d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff4155dc0 con 0x7f9ff40d2b40 2023-08-25T03:35:00.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.107+0000 7f9ffbc5d700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff415e660 con 0x7f9ff40b8aa0 2023-08-25T03:35:00.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.107+0000 7f9ffa1fa700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ff415d370 0x7f9ff40ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:00.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.107+0000 7f9ffa1fa700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ff415d370 0x7f9ff40ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32878/0 (socket says 172.21.15.2:32878) 2023-08-25T03:35:00.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.107+0000 7f9ffa1fa700 1 -- 172.21.15.2:0/1421225921 learned_addr learned my addr 172.21.15.2:0/1421225921 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:00.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.108+0000 7f9ff99f9700 1 --2- 172.21.15.2:0/1421225921 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9ff40b8aa0 0x7f9ff40ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:00.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.108+0000 7f9ff91f8700 1 --2- 172.21.15.2:0/1421225921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ff40d2b40 0x7f9ff4155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:00.121 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.108+0000 7f9ffa1fa700 1 -- 172.21.15.2:0/1421225921 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9ff40b8aa0 msgr2=0x7f9ff40ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:00.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.108+0000 7f9ffa1fa700 1 --2- 172.21.15.2:0/1421225921 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9ff40b8aa0 0x7f9ff40ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:00.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.108+0000 7f9ffa1fa700 1 -- 172.21.15.2:0/1421225921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ff40d2b40 msgr2=0x7f9ff4155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:00.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.108+0000 7f9ffa1fa700 1 --2- 172.21.15.2:0/1421225921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ff40d2b40 0x7f9ff4155880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:00.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.108+0000 7f9ffa1fa700 1 -- 172.21.15.2:0/1421225921 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe4009550 con 0x7f9ff415d370 2023-08-25T03:35:00.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.108+0000 7f9ffa1fa700 1 --2- 172.21.15.2:0/1421225921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ff415d370 0x7f9ff40ca1e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00efc0 tx=0x7f9fec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:00.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.108+0000 7f9fea7fc700 1 -- 172.21.15.2:0/1421225921 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec009ea0 con 0x7f9ff415d370 2023-08-25T03:35:00.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.108+0000 7f9fea7fc700 1 -- 172.21.15.2:0/1421225921 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fec00f040 con 0x7f9ff415d370 2023-08-25T03:35:00.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.108+0000 7f9ffbc5d700 1 -- 172.21.15.2:0/1421225921 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff4155fd0 con 0x7f9ff415d370 2023-08-25T03:35:00.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.108+0000 7f9fea7fc700 1 -- 172.21.15.2:0/1421225921 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fec00ced0 con 0x7f9ff415d370 2023-08-25T03:35:00.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.108+0000 7f9ffbc5d700 1 -- 172.21.15.2:0/1421225921 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff415ebe0 con 0x7f9ff415d370 2023-08-25T03:35:00.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.109+0000 7f9ffbc5d700 1 -- 172.21.15.2:0/1421225921 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff40cffa0 con 0x7f9ff415d370 2023-08-25T03:35:00.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.110+0000 7f9fea7fc700 1 -- 172.21.15.2:0/1421225921 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9fec014480 con 0x7f9ff415d370 2023-08-25T03:35:00.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.110+0000 7f9fea7fc700 1 --2- 172.21.15.2:0/1421225921 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9fd806a0e0 0x7f9fd806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:00.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.110+0000 7f9fea7fc700 1 -- 172.21.15.2:0/1421225921 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9fec088a00 con 0x7f9ff415d370 2023-08-25T03:35:00.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.110+0000 7f9ff99f9700 1 --2- 172.21.15.2:0/1421225921 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9fd806a0e0 0x7f9fd806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:00.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.111+0000 7f9ff99f9700 1 --2- 172.21.15.2:0/1421225921 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9fd806a0e0 0x7f9fd806c5a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0000f60 tx=0x7f9ff0007e70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:00.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.115+0000 7f9fea7fc700 1 -- 172.21.15.2:0/1421225921 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9fec054480 con 0x7f9ff415d370 2023-08-25T03:35:00.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.331+0000 7f9ffbc5d700 1 -- 172.21.15.2:0/1421225921 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f9ff40cfcc0 con 0x7f9ff415d370 2023-08-25T03:35:00.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.897+0000 7f9fea7fc700 1 -- 172.21.15.2:0/1421225921 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v31) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9fec018020 con 0x7f9ff415d370 2023-08-25T03:35:00.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.899+0000 7f9ffbc5d700 1 -- 172.21.15.2:0/1421225921 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9fd806a0e0 msgr2=0x7f9fd806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:00.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.899+0000 7f9ffbc5d700 1 --2- 172.21.15.2:0/1421225921 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9fd806a0e0 0x7f9fd806c5a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0000f60 tx=0x7f9ff0007e70 comp rx=0 tx=0).stop 2023-08-25T03:35:00.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.899+0000 7f9ffbc5d700 1 -- 172.21.15.2:0/1421225921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ff415d370 msgr2=0x7f9ff40ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:00.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.899+0000 7f9ffbc5d700 1 --2- 172.21.15.2:0/1421225921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ff415d370 0x7f9ff40ca1e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f9fec00efc0 tx=0x7f9fec00c3a0 comp rx=0 tx=0).stop 2023-08-25T03:35:00.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.899+0000 7f9ffbc5d700 1 -- 172.21.15.2:0/1421225921 shutdown_connections 2023-08-25T03:35:00.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.899+0000 7f9ffbc5d700 1 --2- 172.21.15.2:0/1421225921 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9fd806a0e0 0x7f9fd806c5a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:00.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.899+0000 7f9ffbc5d700 1 --2- 172.21.15.2:0/1421225921 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9ff415d370 0x7f9ff40ca1e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:00.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.899+0000 7f9ffbc5d700 1 --2- 172.21.15.2:0/1421225921 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9ff40b8aa0 0x7f9ff40ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:00.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.899+0000 7f9ffbc5d700 1 --2- 172.21.15.2:0/1421225921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ff40d2b40 0x7f9ff4155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:00.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.900+0000 7f9ffbc5d700 1 -- 172.21.15.2:0/1421225921 >> 172.21.15.2:0/1421225921 conn(0x7f9ff406ab50 msgr2=0x7f9ff415b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:00.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.900+0000 7f9ffbc5d700 1 -- 172.21.15.2:0/1421225921 shutdown_connections 2023-08-25T03:35:00.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:00.900+0000 7f9ffbc5d700 1 -- 172.21.15.2:0/1421225921 wait complete. 2023-08-25T03:35:00.903 INFO:teuthology.orchestra.run.smithi002.stderr: removed all blocklist entries 2023-08-25T03:35:00.917 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-08-25T03:35:01.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.049+0000 7f513eea2700 1 Processor -- start 2023-08-25T03:35:01.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.050+0000 7f513eea2700 1 -- start start 2023-08-25T03:35:01.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.050+0000 7f513eea2700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f51380cfcc0 0x7f51380b2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:01.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.050+0000 7f513eea2700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f51380cca90 0x7f51380d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:01.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.050+0000 7f513eea2700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51380cbd10 con 0x7f513815ea20 2023-08-25T03:35:01.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.050+0000 7f513eea2700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51380cceb0 con 0x7f51380cca90 2023-08-25T03:35:01.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.050+0000 7f513eea2700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51380cbfb0 con 0x7f51380cfcc0 2023-08-25T03:35:01.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.051+0000 7f513d43f700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f51380cca90 0x7f51380d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:01.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.051+0000 7f513d43f700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f51380cca90 0x7f51380d1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32888/0 (socket says 172.21.15.2:32888) 2023-08-25T03:35:01.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.051+0000 7f513d43f700 1 -- 172.21.15.2:0/2412758741 learned_addr learned my addr 172.21.15.2:0/2412758741 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:01.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.051+0000 7f5137fff700 1 --2- 172.21.15.2:0/2412758741 >> v2:172.21.15.31:3300/0 conn(0x7f51380cfcc0 0x7f51380b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:01.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.051+0000 7f513d43f700 1 -- 172.21.15.2:0/2412758741 >> v2:172.21.15.31:3300/0 conn(0x7f51380cfcc0 msgr2=0x7f51380b2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:01.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.051+0000 7f513d43f700 1 --2- 172.21.15.2:0/2412758741 >> v2:172.21.15.31:3300/0 conn(0x7f51380cfcc0 0x7f51380b2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:01.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.051+0000 7f513d43f700 1 -- 172.21.15.2:0/2412758741 >> v1:172.21.15.31:6789/0 conn(0x7f513815ea20 legacy=0x7f51380cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:01.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.051+0000 7f513d43f700 1 -- 172.21.15.2:0/2412758741 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51380cc290 con 0x7f51380cca90 2023-08-25T03:35:01.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.051+0000 7f513d43f700 1 --2- 172.21.15.2:0/2412758741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51380cca90 0x7f51380d1a10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5138072480 tx=0x7f513000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5d763bba875fbe server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:01.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.051+0000 7f51377fe700 1 -- 172.21.15.2:0/2412758741 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513000e070 con 0x7f51380cca90 2023-08-25T03:35:01.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.051+0000 7f51377fe700 1 -- 172.21.15.2:0/2412758741 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f513000b910 con 0x7f51380cca90 2023-08-25T03:35:01.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.051+0000 7f51377fe700 1 -- 172.21.15.2:0/2412758741 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513000bcc0 con 0x7f51380cca90 2023-08-25T03:35:01.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.052+0000 7f513eea2700 1 -- 172.21.15.2:0/2412758741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51380cca90 msgr2=0x7f51380d1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:01.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.052+0000 7f513eea2700 1 --2- 172.21.15.2:0/2412758741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51380cca90 0x7f51380d1a10 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5138072480 tx=0x7f513000b330 comp rx=0 tx=0).stop 2023-08-25T03:35:01.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.052+0000 7f513eea2700 1 -- 172.21.15.2:0/2412758741 shutdown_connections 2023-08-25T03:35:01.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.052+0000 7f513eea2700 1 --2- 172.21.15.2:0/2412758741 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51380cca90 0x7f51380d1a10 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:01.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.052+0000 7f513eea2700 1 --2- 172.21.15.2:0/2412758741 >> v2:172.21.15.31:3300/0 conn(0x7f51380cfcc0 0x7f51380b2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:01.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.052+0000 7f513eea2700 1 -- 172.21.15.2:0/2412758741 >> 172.21.15.2:0/2412758741 conn(0x7f513809a2b0 msgr2=0x7f51380d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:01.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.052+0000 7f513eea2700 1 -- 172.21.15.2:0/2412758741 shutdown_connections 2023-08-25T03:35:01.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.052+0000 7f513eea2700 1 -- 172.21.15.2:0/2412758741 wait complete. 2023-08-25T03:35:01.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.053+0000 7f513eea2700 1 Processor -- start 2023-08-25T03:35:01.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.053+0000 7f513eea2700 1 -- start start 2023-08-25T03:35:01.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.054+0000 7f513eea2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51380cca90 0x7f5138157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:01.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.054+0000 7f513eea2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51380cfcc0 0x7f51380c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:01.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.054+0000 7f513eea2700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f513815ea20 0x7f513815abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:01.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.054+0000 7f513eea2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51380c9b10 con 0x7f51380cfcc0 2023-08-25T03:35:01.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.054+0000 7f513eea2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51380adba0 con 0x7f51380cca90 2023-08-25T03:35:01.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.054+0000 7f513eea2700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51380c57c0 con 0x7f513815ea20 2023-08-25T03:35:01.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.054+0000 7f5137fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51380cfcc0 0x7f51380c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:01.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.054+0000 7f5137fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51380cfcc0 0x7f51380c4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32892/0 (socket says 172.21.15.2:32892) 2023-08-25T03:35:01.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.054+0000 7f5137fff700 1 -- 172.21.15.2:0/3664088227 learned_addr learned my addr 172.21.15.2:0/3664088227 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:01.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.054+0000 7f513cc3e700 1 --2- 172.21.15.2:0/3664088227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51380cca90 0x7f5138157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:01.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.054+0000 7f513d43f700 1 --2- 172.21.15.2:0/3664088227 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f513815ea20 0x7f513815abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:01.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.055+0000 7f5137fff700 1 -- 172.21.15.2:0/3664088227 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f513815ea20 msgr2=0x7f513815abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:01.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.055+0000 7f5137fff700 1 --2- 172.21.15.2:0/3664088227 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f513815ea20 0x7f513815abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:01.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.055+0000 7f5137fff700 1 -- 172.21.15.2:0/3664088227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51380cca90 msgr2=0x7f5138157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:01.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.055+0000 7f5137fff700 1 --2- 172.21.15.2:0/3664088227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51380cca90 0x7f5138157990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:01.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.055+0000 7f5137fff700 1 -- 172.21.15.2:0/3664088227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5130009b30 con 0x7f51380cfcc0 2023-08-25T03:35:01.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.055+0000 7f5137fff700 1 --2- 172.21.15.2:0/3664088227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51380cfcc0 0x7f51380c4b00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f512400e9e0 tx=0x7f512400ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:01.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.055+0000 7f51357fa700 1 -- 172.21.15.2:0/3664088227 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5124009710 con 0x7f51380cfcc0 2023-08-25T03:35:01.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.055+0000 7f51357fa700 1 -- 172.21.15.2:0/3664088227 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512400ee80 con 0x7f51380cfcc0 2023-08-25T03:35:01.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.055+0000 7f513eea2700 1 -- 172.21.15.2:0/3664088227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51380c8e90 con 0x7f51380cfcc0 2023-08-25T03:35:01.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.055+0000 7f51357fa700 1 -- 172.21.15.2:0/3664088227 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512400ca50 con 0x7f51380cfcc0 2023-08-25T03:35:01.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.055+0000 7f513eea2700 1 -- 172.21.15.2:0/3664088227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51380c9450 con 0x7f51380cfcc0 2023-08-25T03:35:01.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.056+0000 7f513eea2700 1 -- 172.21.15.2:0/3664088227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51380c9c50 con 0x7f51380cfcc0 2023-08-25T03:35:01.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.056+0000 7f51357fa700 1 -- 172.21.15.2:0/3664088227 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f512400cbb0 con 0x7f51380cfcc0 2023-08-25T03:35:01.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.057+0000 7f51357fa700 1 --2- 172.21.15.2:0/3664088227 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f511c06a0e0 0x7f511c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:01.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.057+0000 7f51357fa700 1 -- 172.21.15.2:0/3664088227 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5124088510 con 0x7f51380cfcc0 2023-08-25T03:35:01.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.057+0000 7f513cc3e700 1 --2- 172.21.15.2:0/3664088227 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f511c06a0e0 0x7f511c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:01.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.057+0000 7f513cc3e700 1 --2- 172.21.15.2:0/3664088227 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f511c06a0e0 0x7f511c06c5a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f512c007fd0 tx=0x7f512c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:01.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.061+0000 7f51357fa700 1 -- 172.21.15.2:0/3664088227 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f51380c9c50 con 0x7f51380cfcc0 2023-08-25T03:35:01.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.347+0000 7f513eea2700 1 -- 172.21.15.2:0/3664088227 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f51380c9c50 con 0x7f51380cfcc0 2023-08-25T03:35:01.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.347+0000 7f51357fa700 1 -- 172.21.15.2:0/3664088227 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f51380c9c50 con 0x7f51380cfcc0 2023-08-25T03:35:01.348 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:35:01.349 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-08-25T03:35:01.349 INFO:teuthology.orchestra.run.smithi002.stdout: "auth_dump": [ 2023-08-25T03:35:01.349 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:01.350 INFO:teuthology.orchestra.run.smithi002.stdout: "entity": "mds.a", 2023-08-25T03:35:01.350 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "AQDxIOhkPRSpGxAAlym3fRx8TUwZVWGws/30Nw==", 2023-08-25T03:35:01.350 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": { 2023-08-25T03:35:01.350 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": "allow", 2023-08-25T03:35:01.350 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": "allow *", 2023-08-25T03:35:01.350 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": "allow *", 2023-08-25T03:35:01.351 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": "allow *" 2023-08-25T03:35:01.351 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:01.351 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-08-25T03:35:01.351 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:01.351 INFO:teuthology.orchestra.run.smithi002.stdout: "entity": "mds.b", 2023-08-25T03:35:01.351 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "AQDxIOhkNAqPKBAA6WWkiOL5E6fAboTSSPpkjA==", 2023-08-25T03:35:01.351 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": { 2023-08-25T03:35:01.352 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": "allow", 2023-08-25T03:35:01.352 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": "allow *", 2023-08-25T03:35:01.352 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": "allow *", 2023-08-25T03:35:01.352 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": "allow *" 2023-08-25T03:35:01.352 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:01.352 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-08-25T03:35:01.353 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:01.353 INFO:teuthology.orchestra.run.smithi002.stdout: "entity": "mds.c", 2023-08-25T03:35:01.353 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "AQDxIOhk2hoLIxAA/n01Ior/9O72xdsI5timoA==", 2023-08-25T03:35:01.353 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": { 2023-08-25T03:35:01.353 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": "allow", 2023-08-25T03:35:01.354 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": "allow *", 2023-08-25T03:35:01.354 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": "allow *", 2023-08-25T03:35:01.354 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": "allow *" 2023-08-25T03:35:01.354 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:01.354 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-08-25T03:35:01.354 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:01.354 INFO:teuthology.orchestra.run.smithi002.stdout: "entity": "mds.d", 2023-08-25T03:35:01.354 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "AQDxIOhk4pE9MBAAelxvHaPZaiXLMIVPxXu5bQ==", 2023-08-25T03:35:01.355 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": { 2023-08-25T03:35:01.355 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": "allow", 2023-08-25T03:35:01.355 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": "allow *", 2023-08-25T03:35:01.355 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": "allow *", 2023-08-25T03:35:01.355 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": "allow *" 2023-08-25T03:35:01.355 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:01.355 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-08-25T03:35:01.356 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:01.356 INFO:teuthology.orchestra.run.smithi002.stdout: "entity": "osd.0", 2023-08-25T03:35:01.357 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "AQDzIOhkYU/wHRAA516XnQ537/DrH2XeVOhs2Q==", 2023-08-25T03:35:01.357 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": { 2023-08-25T03:35:01.358 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": "allow profile osd", 2023-08-25T03:35:01.358 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": "allow profile osd", 2023-08-25T03:35:01.358 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": "allow *" 2023-08-25T03:35:01.358 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:01.358 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-08-25T03:35:01.358 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:01.358 INFO:teuthology.orchestra.run.smithi002.stdout: "entity": "osd.1", 2023-08-25T03:35:01.359 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "AQD2IOhkEdSbHRAAIduQDl/OxgzmFmMV83S38g==", 2023-08-25T03:35:01.359 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": { 2023-08-25T03:35:01.359 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": "allow profile osd", 2023-08-25T03:35:01.359 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": "allow profile osd", 2023-08-25T03:35:01.359 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": "allow *" 2023-08-25T03:35:01.360 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:01.360 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-08-25T03:35:01.360 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:01.360 INFO:teuthology.orchestra.run.smithi002.stdout: "entity": "osd.2", 2023-08-25T03:35:01.360 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "AQD5IOhkm9dlHRAAGL6kwSQgXPQ+RJt+Uqj4xQ==", 2023-08-25T03:35:01.360 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": { 2023-08-25T03:35:01.360 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": "allow profile osd", 2023-08-25T03:35:01.361 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": "allow profile osd", 2023-08-25T03:35:01.361 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": "allow *" 2023-08-25T03:35:01.361 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:01.361 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-08-25T03:35:01.361 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:01.361 INFO:teuthology.orchestra.run.smithi002.stdout: "entity": "osd.3", 2023-08-25T03:35:01.361 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "AQD8IOhk5jg9HRAAIMJhGJ3s7PbQ6XYoDbQ2kA==", 2023-08-25T03:35:01.362 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": { 2023-08-25T03:35:01.362 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": "allow profile osd", 2023-08-25T03:35:01.362 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": "allow profile osd", 2023-08-25T03:35:01.362 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": "allow *" 2023-08-25T03:35:01.362 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:01.362 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-08-25T03:35:01.362 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:01.362 INFO:teuthology.orchestra.run.smithi002.stdout: "entity": "osd.4", 2023-08-25T03:35:01.363 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "AQACIehk/emkIxAAsj0+YCvCJXSBujajtCOWSQ==", 2023-08-25T03:35:01.363 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": { 2023-08-25T03:35:01.363 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": "allow profile osd", 2023-08-25T03:35:01.363 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": "allow profile osd", 2023-08-25T03:35:01.363 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": "allow *" 2023-08-25T03:35:01.363 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:01.363 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-08-25T03:35:01.364 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:01.364 INFO:teuthology.orchestra.run.smithi002.stdout: "entity": "osd.5", 2023-08-25T03:35:01.364 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "AQAFIehkQGGsIxAAZg0CHqCjmN74rhDKmTlirQ==", 2023-08-25T03:35:01.364 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": { 2023-08-25T03:35:01.364 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": "allow profile osd", 2023-08-25T03:35:01.364 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": "allow profile osd", 2023-08-25T03:35:01.365 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": "allow *" 2023-08-25T03:35:01.365 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:01.365 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-08-25T03:35:01.365 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:01.365 INFO:teuthology.orchestra.run.smithi002.stdout: "entity": "osd.6", 2023-08-25T03:35:01.365 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "AQAIIehksCOrIxAAZFtoY9baekBEIgaRWo+5UA==", 2023-08-25T03:35:01.366 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": { 2023-08-25T03:35:01.366 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": "allow profile osd", 2023-08-25T03:35:01.366 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": "allow profile osd", 2023-08-25T03:35:01.366 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": "allow *" 2023-08-25T03:35:01.366 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:01.367 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-08-25T03:35:01.368 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:01.368 INFO:teuthology.orchestra.run.smithi002.stdout: "entity": "osd.7", 2023-08-25T03:35:01.368 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "AQALIehkhhnYIxAAzQQ2qxOVDdUKc9a0C3YH4A==", 2023-08-25T03:35:01.368 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": { 2023-08-25T03:35:01.368 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": "allow profile osd", 2023-08-25T03:35:01.368 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": "allow profile osd", 2023-08-25T03:35:01.369 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": "allow *" 2023-08-25T03:35:01.369 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:01.369 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-08-25T03:35:01.370 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:01.370 INFO:teuthology.orchestra.run.smithi002.stdout: "entity": "client.0", 2023-08-25T03:35:01.370 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "AQDxIOhkOijPNBAAx276DY0meH7ng2YxV3536A==", 2023-08-25T03:35:01.371 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": { 2023-08-25T03:35:01.371 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": "allow", 2023-08-25T03:35:01.371 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": "allow r", 2023-08-25T03:35:01.371 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": "allow rw", 2023-08-25T03:35:01.372 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": "allow rwx" 2023-08-25T03:35:01.372 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:01.372 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-08-25T03:35:01.373 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:01.373 INFO:teuthology.orchestra.run.smithi002.stdout: "entity": "client.1", 2023-08-25T03:35:01.373 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "AQDyIOhkQlM+ABAA6hnKw9yLbHTeZWxIjmru/w==", 2023-08-25T03:35:01.374 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": { 2023-08-25T03:35:01.374 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": "allow", 2023-08-25T03:35:01.374 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": "allow r", 2023-08-25T03:35:01.375 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": "allow rw", 2023-08-25T03:35:01.375 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": "allow rwx" 2023-08-25T03:35:01.375 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:01.375 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-08-25T03:35:01.376 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:01.376 INFO:teuthology.orchestra.run.smithi002.stdout: "entity": "client.2", 2023-08-25T03:35:01.376 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "AQDxIOhkKjdAOBAATR/CiHdKxI+68Sd2hI5Mcw==", 2023-08-25T03:35:01.377 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": { 2023-08-25T03:35:01.377 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": "allow", 2023-08-25T03:35:01.378 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": "allow r", 2023-08-25T03:35:01.378 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": "allow rw", 2023-08-25T03:35:01.379 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": "allow rwx" 2023-08-25T03:35:01.379 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:01.379 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-08-25T03:35:01.379 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:01.380 INFO:teuthology.orchestra.run.smithi002.stdout: "entity": "client.3", 2023-08-25T03:35:01.380 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "AQDyIOhkk4GpAxAATAWzeGdDpAann/fbqCtIzw==", 2023-08-25T03:35:01.380 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": { 2023-08-25T03:35:01.381 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": "allow", 2023-08-25T03:35:01.381 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": "allow r", 2023-08-25T03:35:01.381 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": "allow rw", 2023-08-25T03:35:01.381 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": "allow rwx" 2023-08-25T03:35:01.382 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:01.382 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-08-25T03:35:01.382 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:01.382 INFO:teuthology.orchestra.run.smithi002.stdout: "entity": "client.admin", 2023-08-25T03:35:01.383 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "AQDwIOhkTrxROBAAGbfeYMubvCLTOnUJWBjqoQ==", 2023-08-25T03:35:01.383 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": { 2023-08-25T03:35:01.383 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": "allow *", 2023-08-25T03:35:01.383 INFO:teuthology.orchestra.run.smithi002.stdout: "mgr": "allow *", 2023-08-25T03:35:01.384 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": "allow *", 2023-08-25T03:35:01.384 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": "allow *" 2023-08-25T03:35:01.384 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:01.384 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-08-25T03:35:01.385 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:01.385 INFO:teuthology.orchestra.run.smithi002.stdout: "entity": "client.bootstrap-mds", 2023-08-25T03:35:01.385 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "AQARIehkk58qJhAATXMAaWYa4FR0+5kg25g1SA==", 2023-08-25T03:35:01.385 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": { 2023-08-25T03:35:01.386 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": "allow profile bootstrap-mds" 2023-08-25T03:35:01.386 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:01.386 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-08-25T03:35:01.386 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:01.386 INFO:teuthology.orchestra.run.smithi002.stdout: "entity": "client.bootstrap-mgr", 2023-08-25T03:35:01.386 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "AQARIehkNawqJhAASccKIcsEwSTdc+yiQDKZNQ==", 2023-08-25T03:35:01.387 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": { 2023-08-25T03:35:01.387 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": "allow profile bootstrap-mgr" 2023-08-25T03:35:01.387 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:01.387 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-08-25T03:35:01.387 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:01.387 INFO:teuthology.orchestra.run.smithi002.stdout: "entity": "client.bootstrap-osd", 2023-08-25T03:35:01.387 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "AQARIehkorQqJhAA517C/vtZd1I5GZ0HtnVbVQ==", 2023-08-25T03:35:01.388 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": { 2023-08-25T03:35:01.388 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": "allow profile bootstrap-osd" 2023-08-25T03:35:01.388 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:01.388 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-08-25T03:35:01.389 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:01.389 INFO:teuthology.orchestra.run.smithi002.stdout: "entity": "client.bootstrap-rbd", 2023-08-25T03:35:01.389 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "AQARIehkuLwqJhAAKxErgIpy8z1vnvuZAbk3NQ==", 2023-08-25T03:35:01.389 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": { 2023-08-25T03:35:01.389 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": "allow profile bootstrap-rbd" 2023-08-25T03:35:01.390 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:01.390 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-08-25T03:35:01.390 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:01.390 INFO:teuthology.orchestra.run.smithi002.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-08-25T03:35:01.390 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "AQARIehkpcQqJhAA3VHZV0fseigiEo3B/Hv6AA==", 2023-08-25T03:35:01.390 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": { 2023-08-25T03:35:01.391 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-08-25T03:35:01.391 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:01.391 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-08-25T03:35:01.392 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:01.392 INFO:teuthology.orchestra.run.smithi002.stdout: "entity": "client.bootstrap-rgw", 2023-08-25T03:35:01.392 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "AQARIehk7MwqJhAAoqegBwR2GhSi2rE6ALKU5A==", 2023-08-25T03:35:01.392 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": { 2023-08-25T03:35:01.392 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": "allow profile bootstrap-rgw" 2023-08-25T03:35:01.393 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:01.393 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-08-25T03:35:01.393 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:01.393 INFO:teuthology.orchestra.run.smithi002.stdout: "entity": "mgr.x", 2023-08-25T03:35:01.393 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "AQDxIOhkS50iGBAAkESO8E2kBz5meycjbPmMFw==", 2023-08-25T03:35:01.394 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": { 2023-08-25T03:35:01.394 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": "allow *", 2023-08-25T03:35:01.394 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": "allow profile mgr", 2023-08-25T03:35:01.394 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": "allow *" 2023-08-25T03:35:01.395 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:01.395 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-08-25T03:35:01.395 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:01.395 INFO:teuthology.orchestra.run.smithi002.stdout: "entity": "mgr.y", 2023-08-25T03:35:01.396 INFO:teuthology.orchestra.run.smithi002.stdout: "key": "AQDxIOhkaC3sEhAAEGY1vTnEmbinFzeG81KHiQ==", 2023-08-25T03:35:01.396 INFO:teuthology.orchestra.run.smithi002.stdout: "caps": { 2023-08-25T03:35:01.397 INFO:teuthology.orchestra.run.smithi002.stdout: "mds": "allow *", 2023-08-25T03:35:01.397 INFO:teuthology.orchestra.run.smithi002.stdout: "mon": "allow profile mgr", 2023-08-25T03:35:01.397 INFO:teuthology.orchestra.run.smithi002.stdout: "osd": "allow *" 2023-08-25T03:35:01.397 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:01.398 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:01.398 INFO:teuthology.orchestra.run.smithi002.stdout: ] 2023-08-25T03:35:01.399 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-08-25T03:35:01.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.349+0000 7f513eea2700 1 -- 172.21.15.2:0/3664088227 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f511c06a0e0 msgr2=0x7f511c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:01.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.349+0000 7f513eea2700 1 --2- 172.21.15.2:0/3664088227 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f511c06a0e0 0x7f511c06c5a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f512c007fd0 tx=0x7f512c009040 comp rx=0 tx=0).stop 2023-08-25T03:35:01.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.349+0000 7f513eea2700 1 -- 172.21.15.2:0/3664088227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51380cfcc0 msgr2=0x7f51380c4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:01.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.349+0000 7f513eea2700 1 --2- 172.21.15.2:0/3664088227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51380cfcc0 0x7f51380c4b00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f512400e9e0 tx=0x7f512400ecf0 comp rx=0 tx=0).stop 2023-08-25T03:35:01.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.349+0000 7f513eea2700 1 -- 172.21.15.2:0/3664088227 shutdown_connections 2023-08-25T03:35:01.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.349+0000 7f513eea2700 1 --2- 172.21.15.2:0/3664088227 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f513815ea20 0x7f513815abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:01.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.349+0000 7f513eea2700 1 --2- 172.21.15.2:0/3664088227 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51380cca90 0x7f5138157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:01.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.349+0000 7f513eea2700 1 --2- 172.21.15.2:0/3664088227 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f511c06a0e0 0x7f511c06c5a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:01.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.349+0000 7f513eea2700 1 --2- 172.21.15.2:0/3664088227 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51380cfcc0 0x7f51380c4b00 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:01.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.349+0000 7f513eea2700 1 -- 172.21.15.2:0/3664088227 >> 172.21.15.2:0/3664088227 conn(0x7f513809a2b0 msgr2=0x7f513815c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:01.404 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.349+0000 7f513eea2700 1 -- 172.21.15.2:0/3664088227 shutdown_connections 2023-08-25T03:35:01.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.349+0000 7f513eea2700 1 -- 172.21.15.2:0/3664088227 wait complete. 2023-08-25T03:35:01.405 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-08-25T03:35:01.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.531+0000 7f485106b700 1 Processor -- start 2023-08-25T03:35:01.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.531+0000 7f485106b700 1 -- start start 2023-08-25T03:35:01.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.532+0000 7f485106b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f484c0cdad0 0x7f484c06a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:01.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.532+0000 7f485106b700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f484c06ab60 con 0x7f484c0ccec0 2023-08-25T03:35:01.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.532+0000 7f485106b700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f484c157ef0 con 0x7f484c06a720 2023-08-25T03:35:01.539 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.532+0000 7f485106b700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f484c0cdf60 con 0x7f484c0cdad0 2023-08-25T03:35:01.539 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.532+0000 7f484ad9d700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f484c0ccec0 0x7f484c0cf740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47796/0 (socket says 172.21.15.2:47796) 2023-08-25T03:35:01.539 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.532+0000 7f484ad9d700 1 -- 172.21.15.2:0/3044728010 learned_addr learned my addr 172.21.15.2:0/3044728010 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:01.539 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.532+0000 7f4849d9b700 1 -- 172.21.15.2:0/3044728010 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 269216817 0 0) 0x7f484c06ab60 con 0x7f484c0ccec0 2023-08-25T03:35:01.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.532+0000 7f4849d9b700 1 -- 172.21.15.2:0/3044728010 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48340035b0 con 0x7f484c0ccec0 2023-08-25T03:35:01.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.532+0000 7f484a59c700 1 --2- 172.21.15.2:0/3044728010 >> v2:172.21.15.31:3300/0 conn(0x7f484c0cdad0 0x7f484c06a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:01.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.533+0000 7f4849d9b700 1 -- 172.21.15.2:0/3044728010 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3245296435 0 0) 0x7f48340035b0 con 0x7f484c0ccec0 2023-08-25T03:35:01.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.533+0000 7f4849d9b700 1 -- 172.21.15.2:0/3044728010 >> v2:172.21.15.31:3300/0 conn(0x7f484c0cdad0 msgr2=0x7f484c06a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:01.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.533+0000 7f4849d9b700 1 --2- 172.21.15.2:0/3044728010 >> v2:172.21.15.31:3300/0 conn(0x7f484c0cdad0 0x7f484c06a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:01.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.533+0000 7f4849d9b700 1 -- 172.21.15.2:0/3044728010 >> v1:172.21.15.31:6789/0 conn(0x7f484c06a720 legacy=0x7f484c15d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:01.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.533+0000 7f4849d9b700 1 -- 172.21.15.2:0/3044728010 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484c0ce1d0 con 0x7f484c0ccec0 2023-08-25T03:35:01.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.533+0000 7f4849d9b700 1 -- 172.21.15.2:0/3044728010 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f4840001670 con 0x7f484c0ccec0 2023-08-25T03:35:01.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.533+0000 7f4849d9b700 1 -- 172.21.15.2:0/3044728010 >> v1:172.21.15.2:6789/0 conn(0x7f484c0ccec0 legacy=0x7f484c0cf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:01.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.533+0000 7f4849d9b700 1 --2- 172.21.15.2:0/3044728010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4834003a00 0x7f4834003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:01.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.533+0000 7f4849d9b700 1 --2- 172.21.15.2:0/3044728010 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4834004850 0x7f4834004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:01.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.533+0000 7f4849d9b700 1 --2- 172.21.15.2:0/3044728010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834005210 0x7f483400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:01.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.533+0000 7f4849d9b700 1 -- 172.21.15.2:0/3044728010 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484c0cdf60 con 0x7f4834003a00 2023-08-25T03:35:01.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.533+0000 7f4849d9b700 1 -- 172.21.15.2:0/3044728010 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483400c0e0 con 0x7f4834005210 2023-08-25T03:35:01.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.533+0000 7f4849d9b700 1 -- 172.21.15.2:0/3044728010 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f483400c2a0 con 0x7f4834004850 2023-08-25T03:35:01.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.533+0000 7f484ad9d700 1 --2- 172.21.15.2:0/3044728010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4834003a00 0x7f4834003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:01.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.533+0000 7f484a59c700 1 --2- 172.21.15.2:0/3044728010 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4834004850 0x7f4834004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:01.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.533+0000 7f484b59e700 1 --2- 172.21.15.2:0/3044728010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834005210 0x7f483400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:01.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.533+0000 7f484ad9d700 1 -- 172.21.15.2:0/3044728010 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4834004850 msgr2=0x7f4834004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:01.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.533+0000 7f484ad9d700 1 --2- 172.21.15.2:0/3044728010 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4834004850 0x7f4834004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:01.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.533+0000 7f484ad9d700 1 -- 172.21.15.2:0/3044728010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834005210 msgr2=0x7f483400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:01.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.533+0000 7f484ad9d700 1 --2- 172.21.15.2:0/3044728010 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4834005210 0x7f483400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:01.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.533+0000 7f484b59e700 1 -- 172.21.15.2:0/3044728010 reap_dead start 2023-08-25T03:35:01.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.533+0000 7f484ad9d700 1 -- 172.21.15.2:0/3044728010 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48340035b0 con 0x7f4834003a00 2023-08-25T03:35:01.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.533+0000 7f484ad9d700 1 --2- 172.21.15.2:0/3044728010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4834003a00 0x7f4834003e40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f484000c240 tx=0x7f4834005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8b98039927814f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:01.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.534+0000 7f4849d9b700 1 -- 172.21.15.2:0/3044728010 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4840007c10 con 0x7f4834003a00 2023-08-25T03:35:01.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.534+0000 7f4849d9b700 1 -- 172.21.15.2:0/3044728010 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f484000cde0 con 0x7f4834003a00 2023-08-25T03:35:01.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.534+0000 7f4849d9b700 1 -- 172.21.15.2:0/3044728010 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4840005ba0 con 0x7f4834003a00 2023-08-25T03:35:01.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.534+0000 7f485106b700 1 -- 172.21.15.2:0/3044728010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4834003a00 msgr2=0x7f4834003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:01.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.534+0000 7f485106b700 1 --2- 172.21.15.2:0/3044728010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4834003a00 0x7f4834003e40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f484000c240 tx=0x7f4834005880 comp rx=0 tx=0).stop 2023-08-25T03:35:01.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.534+0000 7f485106b700 1 -- 172.21.15.2:0/3044728010 shutdown_connections 2023-08-25T03:35:01.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.534+0000 7f485106b700 1 --2- 172.21.15.2:0/3044728010 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4834003a00 0x7f4834003e40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:01.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.534+0000 7f485106b700 1 -- 172.21.15.2:0/3044728010 >> 172.21.15.2:0/3044728010 conn(0x7f484c0b1bf0 msgr2=0x7f484c0b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:01.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.534+0000 7f485106b700 1 -- 172.21.15.2:0/3044728010 shutdown_connections 2023-08-25T03:35:01.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.534+0000 7f485106b700 1 -- 172.21.15.2:0/3044728010 wait complete. 2023-08-25T03:35:01.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.536+0000 7f485106b700 1 Processor -- start 2023-08-25T03:35:01.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.536+0000 7f485106b700 1 -- start start 2023-08-25T03:35:01.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.536+0000 7f485106b700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4834003a00 0x7f484c0d0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:01.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.536+0000 7f485106b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f484c0c3760 0x7f484c152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:01.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.536+0000 7f485106b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f484c153470 0x7f484c0c5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:01.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.536+0000 7f485106b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484c1f37e0 con 0x7f484c0c3760 2023-08-25T03:35:01.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.536+0000 7f485106b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484c153920 con 0x7f484c153470 2023-08-25T03:35:01.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.536+0000 7f485106b700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f484c1f2b60 con 0x7f4834003a00 2023-08-25T03:35:01.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.536+0000 7f484a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f484c0c3760 0x7f484c152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:01.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.537+0000 7f484a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f484c0c3760 0x7f484c152f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32912/0 (socket says 172.21.15.2:32912) 2023-08-25T03:35:01.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.537+0000 7f484a59c700 1 -- 172.21.15.2:0/1461649977 learned_addr learned my addr 172.21.15.2:0/1461649977 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:01.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.537+0000 7f484ad9d700 1 --2- 172.21.15.2:0/1461649977 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4834003a00 0x7f484c0d0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:01.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.537+0000 7f484b59e700 1 --2- 172.21.15.2:0/1461649977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f484c153470 0x7f484c0c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:01.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.537+0000 7f484a59c700 1 -- 172.21.15.2:0/1461649977 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4834003a00 msgr2=0x7f484c0d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:01.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.537+0000 7f484a59c700 1 --2- 172.21.15.2:0/1461649977 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4834003a00 0x7f484c0d0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:01.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.537+0000 7f484a59c700 1 -- 172.21.15.2:0/1461649977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f484c153470 msgr2=0x7f484c0c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:01.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.537+0000 7f484a59c700 1 --2- 172.21.15.2:0/1461649977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f484c153470 0x7f484c0c5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:01.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.537+0000 7f484a59c700 1 -- 172.21.15.2:0/1461649977 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f484c1f2e40 con 0x7f484c0c3760 2023-08-25T03:35:01.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.537+0000 7f484a59c700 1 --2- 172.21.15.2:0/1461649977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f484c0c3760 0x7f484c152f30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f483c011aa0 tx=0x7f483c011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:01.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.537+0000 7f48337fe700 1 -- 172.21.15.2:0/1461649977 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c00f940 con 0x7f484c0c3760 2023-08-25T03:35:01.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.537+0000 7f48337fe700 1 -- 172.21.15.2:0/1461649977 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483c00faa0 con 0x7f484c0c3760 2023-08-25T03:35:01.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.537+0000 7f485106b700 1 -- 172.21.15.2:0/1461649977 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f484c1f3120 con 0x7f484c0c3760 2023-08-25T03:35:01.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.537+0000 7f48337fe700 1 -- 172.21.15.2:0/1461649977 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483c00fe50 con 0x7f484c0c3760 2023-08-25T03:35:01.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.537+0000 7f485106b700 1 -- 172.21.15.2:0/1461649977 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f484c0d06c0 con 0x7f484c0c3760 2023-08-25T03:35:01.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.538+0000 7f48317fa700 1 -- 172.21.15.2:0/1461649977 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48240052f0 con 0x7f484c0c3760 2023-08-25T03:35:01.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.539+0000 7f48337fe700 1 -- 172.21.15.2:0/1461649977 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f483c01d400 con 0x7f484c0c3760 2023-08-25T03:35:01.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.539+0000 7f48337fe700 1 --2- 172.21.15.2:0/1461649977 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f483806a0e0 0x7f483806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:01.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.539+0000 7f48337fe700 1 -- 172.21.15.2:0/1461649977 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f483c088810 con 0x7f484c0c3760 2023-08-25T03:35:01.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.539+0000 7f484ad9d700 1 --2- 172.21.15.2:0/1461649977 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f483806a0e0 0x7f483806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:01.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.540+0000 7f484ad9d700 1 --2- 172.21.15.2:0/1461649977 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f483806a0e0 0x7f483806c5a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4840015040 tx=0x7f4840007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:01.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.544+0000 7f48337fe700 1 -- 172.21.15.2:0/1461649977 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f483c054250 con 0x7f484c0c3760 2023-08-25T03:35:01.830 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.829+0000 7f48317fa700 1 -- 172.21.15.2:0/1461649977 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f4824005b20 con 0x7f484c0c3760 2023-08-25T03:35:01.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.833+0000 7f48337fe700 1 -- 172.21.15.2:0/1461649977 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f483c057870 con 0x7f484c0c3760 2023-08-25T03:35:01.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.835+0000 7f48317fa700 1 -- 172.21.15.2:0/1461649977 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f483806a0e0 msgr2=0x7f483806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:01.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.835+0000 7f48317fa700 1 --2- 172.21.15.2:0/1461649977 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f483806a0e0 0x7f483806c5a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4840015040 tx=0x7f4840007000 comp rx=0 tx=0).stop 2023-08-25T03:35:01.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.835+0000 7f48317fa700 1 -- 172.21.15.2:0/1461649977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f484c0c3760 msgr2=0x7f484c152f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:01.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.835+0000 7f48317fa700 1 --2- 172.21.15.2:0/1461649977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f484c0c3760 0x7f484c152f30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f483c011aa0 tx=0x7f483c011db0 comp rx=0 tx=0).stop 2023-08-25T03:35:01.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.835+0000 7f48317fa700 1 -- 172.21.15.2:0/1461649977 shutdown_connections 2023-08-25T03:35:01.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.835+0000 7f48317fa700 1 --2- 172.21.15.2:0/1461649977 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f484c153470 0x7f484c0c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:01.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.835+0000 7f48317fa700 1 --2- 172.21.15.2:0/1461649977 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4834003a00 0x7f484c0d0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:01.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.835+0000 7f48317fa700 1 --2- 172.21.15.2:0/1461649977 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f483806a0e0 0x7f483806c5a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:01.837 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.835+0000 7f48317fa700 1 --2- 172.21.15.2:0/1461649977 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f484c0c3760 0x7f484c152f30 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:01.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.835+0000 7f48317fa700 1 -- 172.21.15.2:0/1461649977 >> 172.21.15.2:0/1461649977 conn(0x7f484c0b1bf0 msgr2=0x7f484c0b2cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:01.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.835+0000 7f48317fa700 1 -- 172.21.15.2:0/1461649977 shutdown_connections 2023-08-25T03:35:01.838 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.835+0000 7f48317fa700 1 -- 172.21.15.2:0/1461649977 wait complete. 2023-08-25T03:35:01.849 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-08-25T03:35:01.979 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.978+0000 7f59a33f8700 1 Processor -- start 2023-08-25T03:35:01.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.978+0000 7f59a33f8700 1 -- start start 2023-08-25T03:35:01.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.979+0000 7f59a33f8700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f599c0ccec0 0x7f599c0cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:01.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.979+0000 7f59a33f8700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f599c06a8f0 0x7f599c15d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:01.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.979+0000 7f59a33f8700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f599c157ef0 con 0x7f599c0cf740 2023-08-25T03:35:01.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.979+0000 7f59a33f8700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f599c0ce1d0 con 0x7f599c06a8f0 2023-08-25T03:35:01.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.979+0000 7f59a33f8700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f599c161e40 con 0x7f599c0ccec0 2023-08-25T03:35:01.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.979+0000 7f59a0993700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f599c0cf740 0x7f599c06a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47818/0 (socket says 172.21.15.2:47818) 2023-08-25T03:35:01.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.979+0000 7f59a0993700 1 -- 172.21.15.2:0/3309071504 learned_addr learned my addr 172.21.15.2:0/3309071504 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:01.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.979+0000 7f59a1194700 1 --2- 172.21.15.2:0/3309071504 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f599c0ccec0 0x7f599c0cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:01.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.979+0000 7f59a1995700 1 --2- 172.21.15.2:0/3309071504 >> v2:172.21.15.31:3300/0 conn(0x7f599c06a8f0 0x7f599c15d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:01.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.979+0000 7f5993fff700 1 -- 172.21.15.2:0/3309071504 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3589126409 0 0) 0x7f599c157ef0 con 0x7f599c0cf740 2023-08-25T03:35:01.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.980+0000 7f5993fff700 1 -- 172.21.15.2:0/3309071504 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59880035b0 con 0x7f599c0cf740 2023-08-25T03:35:01.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.980+0000 7f59a1194700 1 -- 172.21.15.2:0/3309071504 >> v2:172.21.15.31:3300/0 conn(0x7f599c06a8f0 msgr2=0x7f599c15d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:01.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.980+0000 7f59a1194700 1 --2- 172.21.15.2:0/3309071504 >> v2:172.21.15.31:3300/0 conn(0x7f599c06a8f0 0x7f599c15d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:01.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.980+0000 7f59a1194700 1 -- 172.21.15.2:0/3309071504 >> v1:172.21.15.2:6789/0 conn(0x7f599c0cf740 legacy=0x7f599c06a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:01.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.980+0000 7f59a1194700 1 -- 172.21.15.2:0/3309071504 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f599c161080 con 0x7f599c0ccec0 2023-08-25T03:35:01.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.980+0000 7f59a1995700 1 --2- 172.21.15.2:0/3309071504 >> v2:172.21.15.31:3300/0 conn(0x7f599c06a8f0 0x7f599c15d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:35:01.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.980+0000 7f59a1194700 1 --2- 172.21.15.2:0/3309071504 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f599c0ccec0 0x7f599c0cdad0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f598c009840 tx=0x7f598c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d606de7461d55734 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:01.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.980+0000 7f5993fff700 1 -- 172.21.15.2:0/3309071504 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c011830 con 0x7f599c0ccec0 2023-08-25T03:35:01.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.980+0000 7f5993fff700 1 -- 172.21.15.2:0/3309071504 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598c011990 con 0x7f599c0ccec0 2023-08-25T03:35:01.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.980+0000 7f5993fff700 1 -- 172.21.15.2:0/3309071504 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c011d20 con 0x7f599c0ccec0 2023-08-25T03:35:01.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.981+0000 7f59a33f8700 1 -- 172.21.15.2:0/3309071504 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f599c0ccec0 msgr2=0x7f599c0cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:01.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.981+0000 7f59a33f8700 1 --2- 172.21.15.2:0/3309071504 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f599c0ccec0 0x7f599c0cdad0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f598c009840 tx=0x7f598c009b50 comp rx=0 tx=0).stop 2023-08-25T03:35:01.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.981+0000 7f59a33f8700 1 -- 172.21.15.2:0/3309071504 shutdown_connections 2023-08-25T03:35:01.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.981+0000 7f59a33f8700 1 --2- 172.21.15.2:0/3309071504 >> v2:172.21.15.31:3300/0 conn(0x7f599c06a8f0 0x7f599c15d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:01.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.981+0000 7f59a33f8700 1 --2- 172.21.15.2:0/3309071504 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f599c0ccec0 0x7f599c0cdad0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:01.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.981+0000 7f59a33f8700 1 -- 172.21.15.2:0/3309071504 >> 172.21.15.2:0/3309071504 conn(0x7f599c0b1bf0 msgr2=0x7f599c0b2000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:01.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.981+0000 7f59a33f8700 1 -- 172.21.15.2:0/3309071504 shutdown_connections 2023-08-25T03:35:01.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.981+0000 7f59a33f8700 1 -- 172.21.15.2:0/3309071504 wait complete. 2023-08-25T03:35:01.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.982+0000 7f59a33f8700 1 Processor -- start 2023-08-25T03:35:01.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.982+0000 7f59a33f8700 1 -- start start 2023-08-25T03:35:01.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.983+0000 7f59a33f8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f599c06a8f0 0x7f599c145480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:01.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.983+0000 7f59a33f8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f599c0ccec0 0x7f599c0c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:01.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.983+0000 7f59a33f8700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f599c0cf740 0x7f599c157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:01.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.983+0000 7f59a33f8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c0c3800 con 0x7f599c06a8f0 2023-08-25T03:35:01.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.983+0000 7f59a33f8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c0c3ae0 con 0x7f599c0ccec0 2023-08-25T03:35:01.995 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.983+0000 7f59a33f8700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f599c0c3dc0 con 0x7f599c0cf740 2023-08-25T03:35:01.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.983+0000 7f59a1194700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f599c06a8f0 0x7f599c145480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:01.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.983+0000 7f59a1194700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f599c06a8f0 0x7f599c145480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32926/0 (socket says 172.21.15.2:32926) 2023-08-25T03:35:01.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.983+0000 7f59a1194700 1 -- 172.21.15.2:0/934397069 learned_addr learned my addr 172.21.15.2:0/934397069 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:01.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.983+0000 7f59a0993700 1 --2- 172.21.15.2:0/934397069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f599c0ccec0 0x7f599c0c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:01.996 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.983+0000 7f59a1995700 1 --2- 172.21.15.2:0/934397069 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f599c0cf740 0x7f599c157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:01.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.983+0000 7f59a1194700 1 -- 172.21.15.2:0/934397069 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f599c0cf740 msgr2=0x7f599c157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:01.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.983+0000 7f59a1194700 1 --2- 172.21.15.2:0/934397069 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f599c0cf740 0x7f599c157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:01.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.983+0000 7f59a1194700 1 -- 172.21.15.2:0/934397069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f599c0ccec0 msgr2=0x7f599c0c48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:01.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.983+0000 7f59a1194700 1 --2- 172.21.15.2:0/934397069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f599c0ccec0 0x7f599c0c48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:01.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.983+0000 7f59a1194700 1 -- 172.21.15.2:0/934397069 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5994009a70 con 0x7f599c06a8f0 2023-08-25T03:35:01.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.983+0000 7f59a1194700 1 --2- 172.21.15.2:0/934397069 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f599c06a8f0 0x7f599c145480 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f598c0090b0 tx=0x7f598c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:01.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.983+0000 7f5991ffb700 1 -- 172.21.15.2:0/934397069 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c0036a0 con 0x7f599c06a8f0 2023-08-25T03:35:01.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.983+0000 7f5991ffb700 1 -- 172.21.15.2:0/934397069 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f598c01f070 con 0x7f599c06a8f0 2023-08-25T03:35:01.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.983+0000 7f59a33f8700 1 -- 172.21.15.2:0/934397069 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f598c0094a0 con 0x7f599c06a8f0 2023-08-25T03:35:01.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.984+0000 7f5991ffb700 1 -- 172.21.15.2:0/934397069 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c003a40 con 0x7f599c06a8f0 2023-08-25T03:35:01.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.984+0000 7f59a33f8700 1 -- 172.21.15.2:0/934397069 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f599c0c45c0 con 0x7f599c06a8f0 2023-08-25T03:35:01.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.985+0000 7f59a33f8700 1 -- 172.21.15.2:0/934397069 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f599c066f70 con 0x7f599c06a8f0 2023-08-25T03:35:01.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.985+0000 7f5991ffb700 1 -- 172.21.15.2:0/934397069 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f598c003ba0 con 0x7f599c06a8f0 2023-08-25T03:35:01.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.985+0000 7f5991ffb700 1 --2- 172.21.15.2:0/934397069 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f598006a0e0 0x7f598006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:01.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.986+0000 7f5991ffb700 1 -- 172.21.15.2:0/934397069 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f598c021070 con 0x7f599c06a8f0 2023-08-25T03:35:02.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.986+0000 7f59a0993700 1 --2- 172.21.15.2:0/934397069 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f598006a0e0 0x7f598006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:02.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.986+0000 7f59a0993700 1 --2- 172.21.15.2:0/934397069 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f598006a0e0 0x7f598006c5a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f599c0c6d80 tx=0x7f5998009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:02.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:01.990+0000 7f5991ffb700 1 -- 172.21.15.2:0/934397069 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f598c055550 con 0x7f599c06a8f0 2023-08-25T03:35:02.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.278+0000 7f59a33f8700 1 -- 172.21.15.2:0/934397069 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f599c0746d0 con 0x7f599c06a8f0 2023-08-25T03:35:02.282 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.281+0000 7f5991ffb700 1 -- 172.21.15.2:0/934397069 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f598c0150a0 con 0x7f599c06a8f0 2023-08-25T03:35:02.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.284+0000 7f59a33f8700 1 -- 172.21.15.2:0/934397069 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f598006a0e0 msgr2=0x7f598006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:02.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.284+0000 7f59a33f8700 1 --2- 172.21.15.2:0/934397069 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f598006a0e0 0x7f598006c5a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f599c0c6d80 tx=0x7f5998009040 comp rx=0 tx=0).stop 2023-08-25T03:35:02.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.284+0000 7f59a33f8700 1 -- 172.21.15.2:0/934397069 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f599c06a8f0 msgr2=0x7f599c145480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:02.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.284+0000 7f59a33f8700 1 --2- 172.21.15.2:0/934397069 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f599c06a8f0 0x7f599c145480 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f598c0090b0 tx=0x7f598c004750 comp rx=0 tx=0).stop 2023-08-25T03:35:02.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.284+0000 7f59a33f8700 1 -- 172.21.15.2:0/934397069 shutdown_connections 2023-08-25T03:35:02.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.284+0000 7f59a33f8700 1 --2- 172.21.15.2:0/934397069 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f598006a0e0 0x7f598006c5a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:02.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.284+0000 7f59a33f8700 1 --2- 172.21.15.2:0/934397069 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f599c06a8f0 0x7f599c145480 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:02.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.284+0000 7f59a33f8700 1 --2- 172.21.15.2:0/934397069 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f599c0cf740 0x7f599c157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:02.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.284+0000 7f59a33f8700 1 --2- 172.21.15.2:0/934397069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f599c0ccec0 0x7f599c0c48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:02.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.284+0000 7f59a33f8700 1 -- 172.21.15.2:0/934397069 >> 172.21.15.2:0/934397069 conn(0x7f599c0b1bf0 msgr2=0x7f599c0b2b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:02.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.284+0000 7f59a33f8700 1 -- 172.21.15.2:0/934397069 shutdown_connections 2023-08-25T03:35:02.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.284+0000 7f59a33f8700 1 -- 172.21.15.2:0/934397069 wait complete. 2023-08-25T03:35:02.297 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-08-25T03:35:02.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.423+0000 7f9e06ed7700 1 Processor -- start 2023-08-25T03:35:02.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.423+0000 7f9e06ed7700 1 -- start start 2023-08-25T03:35:02.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.424+0000 7f9e06ed7700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f9e000cfcc0 0x7f9e000b2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:02.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.424+0000 7f9e06ed7700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e000cca90 0x7f9e000d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:02.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.424+0000 7f9e06ed7700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e000cbd10 con 0x7f9e0015ea20 2023-08-25T03:35:02.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.424+0000 7f9e06ed7700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e000cceb0 con 0x7f9e000cfcc0 2023-08-25T03:35:02.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.424+0000 7f9e06ed7700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e000cbfb0 con 0x7f9e000cca90 2023-08-25T03:35:02.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.424+0000 7f9dfffff700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f9e000cfcc0 0x7f9e000b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:02.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.424+0000 7f9e04c73700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f9e0015ea20 0x7f9e000cf5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47832/0 (socket says 172.21.15.2:47832) 2023-08-25T03:35:02.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.424+0000 7f9e04c73700 1 -- 172.21.15.2:0/1314030457 learned_addr learned my addr 172.21.15.2:0/1314030457 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:02.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.424+0000 7f9e05474700 1 --2- 172.21.15.2:0/1314030457 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e000cca90 0x7f9e000d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:02.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.424+0000 7f9dfffff700 1 -- 172.21.15.2:0/1314030457 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e000cca90 msgr2=0x7f9e000d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:02.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.424+0000 7f9dfffff700 1 --2- 172.21.15.2:0/1314030457 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e000cca90 0x7f9e000d1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:02.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.424+0000 7f9dfffff700 1 -- 172.21.15.2:0/1314030457 >> v1:172.21.15.2:6789/0 conn(0x7f9e0015ea20 legacy=0x7f9e000cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:02.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.424+0000 7f9dfffff700 1 -- 172.21.15.2:0/1314030457 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e000cc290 con 0x7f9e000cfcc0 2023-08-25T03:35:02.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.424+0000 7f9dff7fe700 1 -- 172.21.15.2:0/1314030457 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2183745072 0 0) 0x7f9e000cbd10 con 0x7f9e0015ea20 2023-08-25T03:35:02.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.424+0000 7f9dfffff700 1 --2- 172.21.15.2:0/1314030457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e000cfcc0 0x7f9e000b2020 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009840 tx=0x7f9dec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736dcdbd8e072fec server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:02.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.425+0000 7f9dff7fe700 1 -- 172.21.15.2:0/1314030457 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec011830 con 0x7f9e000cfcc0 2023-08-25T03:35:02.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.425+0000 7f9dff7fe700 1 -- 172.21.15.2:0/1314030457 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dec011990 con 0x7f9e000cfcc0 2023-08-25T03:35:02.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.425+0000 7f9dff7fe700 1 -- 172.21.15.2:0/1314030457 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec011d20 con 0x7f9e000cfcc0 2023-08-25T03:35:02.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.425+0000 7f9e06ed7700 1 -- 172.21.15.2:0/1314030457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e000cfcc0 msgr2=0x7f9e000b2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:02.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.425+0000 7f9e06ed7700 1 --2- 172.21.15.2:0/1314030457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e000cfcc0 0x7f9e000b2020 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009840 tx=0x7f9dec009b50 comp rx=0 tx=0).stop 2023-08-25T03:35:02.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.425+0000 7f9e06ed7700 1 -- 172.21.15.2:0/1314030457 shutdown_connections 2023-08-25T03:35:02.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.425+0000 7f9e06ed7700 1 --2- 172.21.15.2:0/1314030457 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e000cca90 0x7f9e000d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:02.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.425+0000 7f9e06ed7700 1 --2- 172.21.15.2:0/1314030457 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e000cfcc0 0x7f9e000b2020 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:02.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.425+0000 7f9e06ed7700 1 -- 172.21.15.2:0/1314030457 >> 172.21.15.2:0/1314030457 conn(0x7f9e0009a2b0 msgr2=0x7f9e000d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:02.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.425+0000 7f9e06ed7700 1 -- 172.21.15.2:0/1314030457 shutdown_connections 2023-08-25T03:35:02.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.425+0000 7f9e06ed7700 1 -- 172.21.15.2:0/1314030457 wait complete. 2023-08-25T03:35:02.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.426+0000 7f9e06ed7700 1 Processor -- start 2023-08-25T03:35:02.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.426+0000 7f9e06ed7700 1 -- start start 2023-08-25T03:35:02.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.427+0000 7f9e06ed7700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e000cca90 0x7f9e001f49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:02.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.427+0000 7f9e06ed7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e000cfcc0 0x7f9e00156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:02.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.427+0000 7f9e06ed7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e0015ea20 0x7f9e000ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:02.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.427+0000 7f9e06ed7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e001f4ee0 con 0x7f9e0015ea20 2023-08-25T03:35:02.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.427+0000 7f9e06ed7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e001f3f50 con 0x7f9e000cfcc0 2023-08-25T03:35:02.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.427+0000 7f9e06ed7700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e001f4510 con 0x7f9e000cca90 2023-08-25T03:35:02.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.427+0000 7f9e05474700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e0015ea20 0x7f9e000ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:02.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.427+0000 7f9e05474700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e0015ea20 0x7f9e000ca330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32944/0 (socket says 172.21.15.2:32944) 2023-08-25T03:35:02.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.427+0000 7f9e05474700 1 -- 172.21.15.2:0/2795894491 learned_addr learned my addr 172.21.15.2:0/2795894491 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:02.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.428+0000 7f9dfffff700 1 --2- 172.21.15.2:0/2795894491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e000cfcc0 0x7f9e00156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:02.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.428+0000 7f9e04c73700 1 --2- 172.21.15.2:0/2795894491 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e000cca90 0x7f9e001f49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:02.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.428+0000 7f9e05474700 1 -- 172.21.15.2:0/2795894491 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e000cca90 msgr2=0x7f9e001f49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:02.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.428+0000 7f9e05474700 1 --2- 172.21.15.2:0/2795894491 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e000cca90 0x7f9e001f49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:02.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.428+0000 7f9e05474700 1 -- 172.21.15.2:0/2795894491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e000cfcc0 msgr2=0x7f9e00156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:02.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.428+0000 7f9e05474700 1 --2- 172.21.15.2:0/2795894491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e000cfcc0 0x7f9e00156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:02.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.428+0000 7f9e05474700 1 -- 172.21.15.2:0/2795894491 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dec0094a0 con 0x7f9e0015ea20 2023-08-25T03:35:02.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.428+0000 7f9e04c73700 1 --2- 172.21.15.2:0/2795894491 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e000cca90 0x7f9e001f49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:35:02.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.428+0000 7f9e05474700 1 --2- 172.21.15.2:0/2795894491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e0015ea20 0x7f9e000ca330 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9df800cda0 tx=0x7f9df800a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:02.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.428+0000 7f9dfd7fa700 1 -- 172.21.15.2:0/2795894491 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df8003a20 con 0x7f9e0015ea20 2023-08-25T03:35:02.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.428+0000 7f9dfd7fa700 1 -- 172.21.15.2:0/2795894491 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df8003b80 con 0x7f9e0015ea20 2023-08-25T03:35:02.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.428+0000 7f9e06ed7700 1 -- 172.21.15.2:0/2795894491 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e000cada0 con 0x7f9e0015ea20 2023-08-25T03:35:02.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.428+0000 7f9dfd7fa700 1 -- 172.21.15.2:0/2795894491 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df8004020 con 0x7f9e0015ea20 2023-08-25T03:35:02.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.428+0000 7f9e06ed7700 1 -- 172.21.15.2:0/2795894491 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e001f41a0 con 0x7f9e0015ea20 2023-08-25T03:35:02.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.429+0000 7f9e06ed7700 1 -- 172.21.15.2:0/2795894491 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e000746d0 con 0x7f9e0015ea20 2023-08-25T03:35:02.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.429+0000 7f9dfd7fa700 1 -- 172.21.15.2:0/2795894491 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9df8004180 con 0x7f9e0015ea20 2023-08-25T03:35:02.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.430+0000 7f9dfd7fa700 1 --2- 172.21.15.2:0/2795894491 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9de406a0e0 0x7f9de406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:02.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.430+0000 7f9dfd7fa700 1 -- 172.21.15.2:0/2795894491 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f9df8012070 con 0x7f9e0015ea20 2023-08-25T03:35:02.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.430+0000 7f9e04c73700 1 --2- 172.21.15.2:0/2795894491 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9de406a0e0 0x7f9de406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:02.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.430+0000 7f9e04c73700 1 --2- 172.21.15.2:0/2795894491 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9de406a0e0 0x7f9de406c5a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9df4007fd0 tx=0x7f9df4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:02.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.434+0000 7f9dfd7fa700 1 -- 172.21.15.2:0/2795894491 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9df80544c0 con 0x7f9e0015ea20 2023-08-25T03:35:02.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.718+0000 7f9e06ed7700 1 -- 172.21.15.2:0/2795894491 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f9e001f4650 con 0x7f9e0015ea20 2023-08-25T03:35:02.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.721+0000 7f9dfd7fa700 1 -- 172.21.15.2:0/2795894491 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9df8057ae0 con 0x7f9e0015ea20 2023-08-25T03:35:02.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.723+0000 7f9e06ed7700 1 -- 172.21.15.2:0/2795894491 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9de406a0e0 msgr2=0x7f9de406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:02.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.723+0000 7f9e06ed7700 1 --2- 172.21.15.2:0/2795894491 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9de406a0e0 0x7f9de406c5a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9df4007fd0 tx=0x7f9df4009040 comp rx=0 tx=0).stop 2023-08-25T03:35:02.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.723+0000 7f9e06ed7700 1 -- 172.21.15.2:0/2795894491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e0015ea20 msgr2=0x7f9e000ca330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:02.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.723+0000 7f9e06ed7700 1 --2- 172.21.15.2:0/2795894491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e0015ea20 0x7f9e000ca330 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9df800cda0 tx=0x7f9df800a570 comp rx=0 tx=0).stop 2023-08-25T03:35:02.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.723+0000 7f9e06ed7700 1 -- 172.21.15.2:0/2795894491 shutdown_connections 2023-08-25T03:35:02.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.723+0000 7f9e06ed7700 1 --2- 172.21.15.2:0/2795894491 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9de406a0e0 0x7f9de406c5a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:02.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.723+0000 7f9e06ed7700 1 --2- 172.21.15.2:0/2795894491 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9e0015ea20 0x7f9e000ca330 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:02.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.723+0000 7f9e06ed7700 1 --2- 172.21.15.2:0/2795894491 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e000cca90 0x7f9e001f49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:02.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.723+0000 7f9e06ed7700 1 --2- 172.21.15.2:0/2795894491 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e000cfcc0 0x7f9e00156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:02.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.723+0000 7f9e06ed7700 1 -- 172.21.15.2:0/2795894491 >> 172.21.15.2:0/2795894491 conn(0x7f9e0009a2b0 msgr2=0x7f9e001e6110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:02.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.724+0000 7f9e06ed7700 1 -- 172.21.15.2:0/2795894491 shutdown_connections 2023-08-25T03:35:02.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.724+0000 7f9e06ed7700 1 -- 172.21.15.2:0/2795894491 wait complete. 2023-08-25T03:35:02.737 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-08-25T03:35:02.863 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.863+0000 7f0b5a01f700 1 Processor -- start 2023-08-25T03:35:02.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.863+0000 7f0b5a01f700 1 -- start start 2023-08-25T03:35:02.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.864+0000 7f0b5a01f700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f0b5415e890 0x7f0b54071430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:02.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.864+0000 7f0b5a01f700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f0b540caea0 0x7f0b540cb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:02.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.864+0000 7f0b5a01f700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b540720b0 con 0x7f0b540b19a0 2023-08-25T03:35:02.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.864+0000 7f0b5a01f700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b54071dd0 con 0x7f0b5415e890 2023-08-25T03:35:02.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.864+0000 7f0b5a01f700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b5415ddb0 con 0x7f0b540caea0 2023-08-25T03:35:02.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.864+0000 7f0b52ffd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f0b5415e890 0x7f0b54071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:02.866 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.864+0000 7f0b52ffd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f0b5415e890 0x7f0b54071430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32950/0 (socket says 172.21.15.2:32950) 2023-08-25T03:35:02.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.864+0000 7f0b52ffd700 1 -- 172.21.15.2:0/1644942102 learned_addr learned my addr 172.21.15.2:0/1644942102 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:02.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.864+0000 7f0b53fff700 1 --2- 172.21.15.2:0/1644942102 >> v2:172.21.15.31:3300/0 conn(0x7f0b540caea0 0x7f0b540cb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:02.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.864+0000 7f0b52ffd700 1 -- 172.21.15.2:0/1644942102 >> v2:172.21.15.31:3300/0 conn(0x7f0b540caea0 msgr2=0x7f0b540cb330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:02.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.864+0000 7f0b52ffd700 1 --2- 172.21.15.2:0/1644942102 >> v2:172.21.15.31:3300/0 conn(0x7f0b540caea0 0x7f0b540cb330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:02.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.864+0000 7f0b52ffd700 1 -- 172.21.15.2:0/1644942102 >> v1:172.21.15.31:6789/0 conn(0x7f0b540b19a0 legacy=0x7f0b540b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:02.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.864+0000 7f0b52ffd700 1 -- 172.21.15.2:0/1644942102 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5415e090 con 0x7f0b5415e890 2023-08-25T03:35:02.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.864+0000 7f0b52ffd700 1 --2- 172.21.15.2:0/1644942102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b5415e890 0x7f0b54071430 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0b540cc8b0 tx=0x7f0b4800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2128359b290e751d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:02.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.864+0000 7f0b527fc700 1 -- 172.21.15.2:0/1644942102 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4800dcf0 con 0x7f0b5415e890 2023-08-25T03:35:02.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.864+0000 7f0b527fc700 1 -- 172.21.15.2:0/1644942102 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4800de50 con 0x7f0b5415e890 2023-08-25T03:35:02.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.864+0000 7f0b527fc700 1 -- 172.21.15.2:0/1644942102 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b48004250 con 0x7f0b5415e890 2023-08-25T03:35:02.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.865+0000 7f0b5a01f700 1 -- 172.21.15.2:0/1644942102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b5415e890 msgr2=0x7f0b54071430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:02.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.865+0000 7f0b5a01f700 1 --2- 172.21.15.2:0/1644942102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b5415e890 0x7f0b54071430 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0b540cc8b0 tx=0x7f0b4800be70 comp rx=0 tx=0).stop 2023-08-25T03:35:02.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.865+0000 7f0b5a01f700 1 -- 172.21.15.2:0/1644942102 shutdown_connections 2023-08-25T03:35:02.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.865+0000 7f0b5a01f700 1 --2- 172.21.15.2:0/1644942102 >> v2:172.21.15.31:3300/0 conn(0x7f0b540caea0 0x7f0b540cb330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:02.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.865+0000 7f0b5a01f700 1 --2- 172.21.15.2:0/1644942102 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b5415e890 0x7f0b54071430 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:02.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.865+0000 7f0b5a01f700 1 -- 172.21.15.2:0/1644942102 >> 172.21.15.2:0/1644942102 conn(0x7f0b540cfea0 msgr2=0x7f0b540d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:02.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.865+0000 7f0b5a01f700 1 -- 172.21.15.2:0/1644942102 shutdown_connections 2023-08-25T03:35:02.869 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.865+0000 7f0b5a01f700 1 -- 172.21.15.2:0/1644942102 wait complete. 2023-08-25T03:35:02.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.866+0000 7f0b5a01f700 1 Processor -- start 2023-08-25T03:35:02.871 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.866+0000 7f0b5a01f700 1 -- start start 2023-08-25T03:35:02.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.867+0000 7f0b5a01f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b540b19a0 0x7f0b5415b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:02.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.867+0000 7f0b5a01f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b540caea0 0x7f0b54146d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:02.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.867+0000 7f0b5a01f700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0b5415e890 0x7f0b540c2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:02.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.867+0000 7f0b5a01f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540c5280 con 0x7f0b540b19a0 2023-08-25T03:35:02.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.867+0000 7f0b5a01f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b54147280 con 0x7f0b540caea0 2023-08-25T03:35:02.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.867+0000 7f0b5a01f700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b54147520 con 0x7f0b5415e890 2023-08-25T03:35:02.872 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.867+0000 7f0b537fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b540b19a0 0x7f0b5415b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:02.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.867+0000 7f0b537fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b540b19a0 0x7f0b5415b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32954/0 (socket says 172.21.15.2:32954) 2023-08-25T03:35:02.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.867+0000 7f0b537fe700 1 -- 172.21.15.2:0/1518419641 learned_addr learned my addr 172.21.15.2:0/1518419641 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:02.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.867+0000 7f0b52ffd700 1 --2- 172.21.15.2:0/1518419641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b540caea0 0x7f0b54146d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:02.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.867+0000 7f0b53fff700 1 --2- 172.21.15.2:0/1518419641 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0b5415e890 0x7f0b540c2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:02.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.868+0000 7f0b537fe700 1 -- 172.21.15.2:0/1518419641 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0b5415e890 msgr2=0x7f0b540c2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:02.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.868+0000 7f0b537fe700 1 --2- 172.21.15.2:0/1518419641 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0b5415e890 0x7f0b540c2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:02.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.868+0000 7f0b537fe700 1 -- 172.21.15.2:0/1518419641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b540caea0 msgr2=0x7f0b54146d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:02.873 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.868+0000 7f0b537fe700 1 --2- 172.21.15.2:0/1518419641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b540caea0 0x7f0b54146d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:02.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.868+0000 7f0b537fe700 1 -- 172.21.15.2:0/1518419641 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4800b7f0 con 0x7f0b540b19a0 2023-08-25T03:35:02.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.868+0000 7f0b53fff700 1 --2- 172.21.15.2:0/1518419641 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0b5415e890 0x7f0b540c2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:02.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.868+0000 7f0b537fe700 1 --2- 172.21.15.2:0/1518419641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b540b19a0 0x7f0b5415b070 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400b4a0 tx=0x7f0b4400b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:02.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.868+0000 7f0b3bfff700 1 -- 172.21.15.2:0/1518419641 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44011830 con 0x7f0b540b19a0 2023-08-25T03:35:02.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.868+0000 7f0b3bfff700 1 -- 172.21.15.2:0/1518419641 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b44011990 con 0x7f0b540b19a0 2023-08-25T03:35:02.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.868+0000 7f0b5a01f700 1 -- 172.21.15.2:0/1518419641 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b54147800 con 0x7f0b540b19a0 2023-08-25T03:35:02.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.868+0000 7f0b3bfff700 1 -- 172.21.15.2:0/1518419641 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44011d40 con 0x7f0b540b19a0 2023-08-25T03:35:02.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.868+0000 7f0b5a01f700 1 -- 172.21.15.2:0/1518419641 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b540c4ee0 con 0x7f0b540b19a0 2023-08-25T03:35:02.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.869+0000 7f0b5a01f700 1 -- 172.21.15.2:0/1518419641 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b54066f70 con 0x7f0b540b19a0 2023-08-25T03:35:02.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.869+0000 7f0b3bfff700 1 -- 172.21.15.2:0/1518419641 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0b4401a400 con 0x7f0b540b19a0 2023-08-25T03:35:02.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.870+0000 7f0b3bfff700 1 --2- 172.21.15.2:0/1518419641 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f0b4006a0e0 0x7f0b4006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:02.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.870+0000 7f0b3bfff700 1 -- 172.21.15.2:0/1518419641 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f0b44087f10 con 0x7f0b540b19a0 2023-08-25T03:35:02.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.870+0000 7f0b52ffd700 1 --2- 172.21.15.2:0/1518419641 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f0b4006a0e0 0x7f0b4006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:02.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.870+0000 7f0b52ffd700 1 --2- 172.21.15.2:0/1518419641 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f0b4006a0e0 0x7f0b4006c5a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0b48000c00 tx=0x7f0b4801d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:02.876 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:02.874+0000 7f0b3bfff700 1 -- 172.21.15.2:0/1518419641 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0b44053950 con 0x7f0b540b19a0 2023-08-25T03:35:03.039 DEBUG:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T03:35:03.041 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T03:35:03.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.164+0000 7f0b5a01f700 1 -- 172.21.15.2:0/1518419641 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f0b540746d0 con 0x7f0b540b19a0 2023-08-25T03:35:03.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.167+0000 7f0b3bfff700 1 -- 172.21.15.2:0/1518419641 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f0b44056f70 con 0x7f0b540b19a0 2023-08-25T03:35:03.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.170+0000 7f0b5a01f700 1 -- 172.21.15.2:0/1518419641 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f0b4006a0e0 msgr2=0x7f0b4006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:03.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.170+0000 7f0b5a01f700 1 --2- 172.21.15.2:0/1518419641 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f0b4006a0e0 0x7f0b4006c5a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0b48000c00 tx=0x7f0b4801d040 comp rx=0 tx=0).stop 2023-08-25T03:35:03.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.170+0000 7f0b5a01f700 1 -- 172.21.15.2:0/1518419641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b540b19a0 msgr2=0x7f0b5415b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:03.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.170+0000 7f0b5a01f700 1 --2- 172.21.15.2:0/1518419641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b540b19a0 0x7f0b5415b070 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400b4a0 tx=0x7f0b4400b7b0 comp rx=0 tx=0).stop 2023-08-25T03:35:03.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.170+0000 7f0b5a01f700 1 -- 172.21.15.2:0/1518419641 shutdown_connections 2023-08-25T03:35:03.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.170+0000 7f0b5a01f700 1 --2- 172.21.15.2:0/1518419641 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f0b4006a0e0 0x7f0b4006c5a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:03.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.170+0000 7f0b5a01f700 1 --2- 172.21.15.2:0/1518419641 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0b540b19a0 0x7f0b5415b070 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:03.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.170+0000 7f0b5a01f700 1 --2- 172.21.15.2:0/1518419641 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0b5415e890 0x7f0b540c2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:03.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.170+0000 7f0b5a01f700 1 --2- 172.21.15.2:0/1518419641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0b540caea0 0x7f0b54146d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:03.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.170+0000 7f0b5a01f700 1 -- 172.21.15.2:0/1518419641 >> 172.21.15.2:0/1518419641 conn(0x7f0b540cfea0 msgr2=0x7f0b54158020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:03.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.170+0000 7f0b5a01f700 1 -- 172.21.15.2:0/1518419641 shutdown_connections 2023-08-25T03:35:03.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.170+0000 7f0b5a01f700 1 -- 172.21.15.2:0/1518419641 wait complete. 2023-08-25T03:35:03.183 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-08-25T03:35:03.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.308+0000 7fc65222e700 1 Processor -- start 2023-08-25T03:35:03.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.308+0000 7fc65222e700 1 -- start start 2023-08-25T03:35:03.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.309+0000 7fc65222e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc64c15e6e0 0x7fc64c0cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:03.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.309+0000 7fc65222e700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fc64c15eac0 0x7fc64c0cca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:03.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.309+0000 7fc65222e700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc64c0cf5d0 con 0x7fc64c0d16d0 2023-08-25T03:35:03.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.309+0000 7fc65222e700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc64c0b3f40 con 0x7fc64c15eac0 2023-08-25T03:35:03.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.309+0000 7fc65222e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc64c0cf870 con 0x7fc64c15e6e0 2023-08-25T03:35:03.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.309+0000 7fc64bfff700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fc64c15eac0 0x7fc64c0cca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:03.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.309+0000 7fc64bfff700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fc64c15eac0 0x7fc64c0cca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32966/0 (socket says 172.21.15.2:32966) 2023-08-25T03:35:03.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.309+0000 7fc64bfff700 1 -- 172.21.15.2:0/3261740844 learned_addr learned my addr 172.21.15.2:0/3261740844 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:03.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.309+0000 7fc64affd700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fc64c0d16d0 0x7fc64c0b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47860/0 (socket says 172.21.15.2:47860) 2023-08-25T03:35:03.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.310+0000 7fc64b7fe700 1 --2- 172.21.15.2:0/3261740844 >> v2:172.21.15.31:3300/0 conn(0x7fc64c15e6e0 0x7fc64c0cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:03.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.310+0000 7fc64bfff700 1 -- 172.21.15.2:0/3261740844 >> v2:172.21.15.31:3300/0 conn(0x7fc64c15e6e0 msgr2=0x7fc64c0cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:03.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.310+0000 7fc64bfff700 1 --2- 172.21.15.2:0/3261740844 >> v2:172.21.15.31:3300/0 conn(0x7fc64c15e6e0 0x7fc64c0cd900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:03.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.310+0000 7fc64bfff700 1 -- 172.21.15.2:0/3261740844 >> v1:172.21.15.2:6789/0 conn(0x7fc64c0d16d0 legacy=0x7fc64c0b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:03.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.310+0000 7fc64bfff700 1 -- 172.21.15.2:0/3261740844 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64c0cfb50 con 0x7fc64c15eac0 2023-08-25T03:35:03.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.310+0000 7fc64bfff700 1 --2- 172.21.15.2:0/3261740844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc64c15eac0 0x7fc64c0cca50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc644009e50 tx=0x7fc64400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835f5196404fa9a8 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:03.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.310+0000 7fc64a7fc700 1 -- 172.21.15.2:0/3261740844 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64400e070 con 0x7fc64c15eac0 2023-08-25T03:35:03.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.310+0000 7fc64a7fc700 1 -- 172.21.15.2:0/3261740844 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64400b980 con 0x7fc64c15eac0 2023-08-25T03:35:03.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.310+0000 7fc64a7fc700 1 -- 172.21.15.2:0/3261740844 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64400bd30 con 0x7fc64c15eac0 2023-08-25T03:35:03.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.310+0000 7fc65222e700 1 -- 172.21.15.2:0/3261740844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc64c15eac0 msgr2=0x7fc64c0cca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:03.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.310+0000 7fc65222e700 1 --2- 172.21.15.2:0/3261740844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc64c15eac0 0x7fc64c0cca50 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc644009e50 tx=0x7fc64400b330 comp rx=0 tx=0).stop 2023-08-25T03:35:03.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.311+0000 7fc65222e700 1 -- 172.21.15.2:0/3261740844 shutdown_connections 2023-08-25T03:35:03.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.311+0000 7fc65222e700 1 --2- 172.21.15.2:0/3261740844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc64c15eac0 0x7fc64c0cca50 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:03.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.311+0000 7fc65222e700 1 --2- 172.21.15.2:0/3261740844 >> v2:172.21.15.31:3300/0 conn(0x7fc64c15e6e0 0x7fc64c0cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:03.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.311+0000 7fc65222e700 1 -- 172.21.15.2:0/3261740844 >> 172.21.15.2:0/3261740844 conn(0x7fc64c09a2b0 msgr2=0x7fc64c0d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:03.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.311+0000 7fc65222e700 1 -- 172.21.15.2:0/3261740844 shutdown_connections 2023-08-25T03:35:03.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.311+0000 7fc65222e700 1 -- 172.21.15.2:0/3261740844 wait complete. 2023-08-25T03:35:03.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.312+0000 7fc65222e700 1 Processor -- start 2023-08-25T03:35:03.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.312+0000 7fc65222e700 1 -- start start 2023-08-25T03:35:03.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.313+0000 7fc65222e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc64c15e6e0 0x7fc64c14d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:03.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.313+0000 7fc65222e700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc64c15eac0 0x7fc64c09da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:03.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.313+0000 7fc65222e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc64c09dfa0 0x7fc64c0c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:03.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.313+0000 7fc65222e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c09bc50 con 0x7fc64c09dfa0 2023-08-25T03:35:03.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.313+0000 7fc65222e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c0caa50 con 0x7fc64c15e6e0 2023-08-25T03:35:03.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.313+0000 7fc65222e700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c09e450 con 0x7fc64c15eac0 2023-08-25T03:35:03.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.313+0000 7fc64bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc64c09dfa0 0x7fc64c0c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:03.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.313+0000 7fc64bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc64c09dfa0 0x7fc64c0c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32972/0 (socket says 172.21.15.2:32972) 2023-08-25T03:35:03.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.313+0000 7fc64bfff700 1 -- 172.21.15.2:0/172880275 learned_addr learned my addr 172.21.15.2:0/172880275 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:03.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.313+0000 7fc64b7fe700 1 --2- 172.21.15.2:0/172880275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc64c15e6e0 0x7fc64c14d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:03.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.313+0000 7fc64affd700 1 --2- 172.21.15.2:0/172880275 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc64c15eac0 0x7fc64c09da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:03.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.313+0000 7fc64bfff700 1 -- 172.21.15.2:0/172880275 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc64c15eac0 msgr2=0x7fc64c09da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:03.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.313+0000 7fc64bfff700 1 --2- 172.21.15.2:0/172880275 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc64c15eac0 0x7fc64c09da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:03.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.313+0000 7fc64bfff700 1 -- 172.21.15.2:0/172880275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc64c15e6e0 msgr2=0x7fc64c14d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:03.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.313+0000 7fc64bfff700 1 --2- 172.21.15.2:0/172880275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc64c15e6e0 0x7fc64c14d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:03.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.313+0000 7fc64bfff700 1 -- 172.21.15.2:0/172880275 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc644009b30 con 0x7fc64c09dfa0 2023-08-25T03:35:03.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.313+0000 7fc64affd700 1 --2- 172.21.15.2:0/172880275 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc64c15eac0 0x7fc64c09da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:03.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.313+0000 7fc64b7fe700 1 --2- 172.21.15.2:0/172880275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc64c15e6e0 0x7fc64c14d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:03.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.313+0000 7fc64bfff700 1 --2- 172.21.15.2:0/172880275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc64c09dfa0 0x7fc64c0c9d20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc644012000 tx=0x7fc644004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:03.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.314+0000 7fc643fff700 1 -- 172.21.15.2:0/172880275 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6440092e0 con 0x7fc64c09dfa0 2023-08-25T03:35:03.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.314+0000 7fc643fff700 1 -- 172.21.15.2:0/172880275 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6440099b0 con 0x7fc64c09dfa0 2023-08-25T03:35:03.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.314+0000 7fc643fff700 1 -- 172.21.15.2:0/172880275 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64401c680 con 0x7fc64c09dfa0 2023-08-25T03:35:03.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.314+0000 7fc65222e700 1 -- 172.21.15.2:0/172880275 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc64c09b2b0 con 0x7fc64c09dfa0 2023-08-25T03:35:03.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.314+0000 7fc65222e700 1 -- 172.21.15.2:0/172880275 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc64c09b870 con 0x7fc64c09dfa0 2023-08-25T03:35:03.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.315+0000 7fc65222e700 1 -- 172.21.15.2:0/172880275 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc64c09bd90 con 0x7fc64c09dfa0 2023-08-25T03:35:03.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.315+0000 7fc643fff700 1 -- 172.21.15.2:0/172880275 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc64401c7e0 con 0x7fc64c09dfa0 2023-08-25T03:35:03.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.315+0000 7fc643fff700 1 --2- 172.21.15.2:0/172880275 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc62c06a0e0 0x7fc62c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:03.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.316+0000 7fc643fff700 1 -- 172.21.15.2:0/172880275 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fc644089ee0 con 0x7fc64c09dfa0 2023-08-25T03:35:03.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.316+0000 7fc64b7fe700 1 --2- 172.21.15.2:0/172880275 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc62c06a0e0 0x7fc62c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:03.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.316+0000 7fc64b7fe700 1 --2- 172.21.15.2:0/172880275 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc62c06a0e0 0x7fc62c06c5a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc63c006fd0 tx=0x7fc63c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:03.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.320+0000 7fc643fff700 1 -- 172.21.15.2:0/172880275 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc6440559b0 con 0x7fc64c09dfa0 2023-08-25T03:35:03.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.605+0000 7fc65222e700 1 -- 172.21.15.2:0/172880275 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fc64c0746d0 con 0x7fc64c09dfa0 2023-08-25T03:35:03.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.608+0000 7fc643fff700 1 -- 172.21.15.2:0/172880275 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc644058fd0 con 0x7fc64c09dfa0 2023-08-25T03:35:03.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.610+0000 7fc65222e700 1 -- 172.21.15.2:0/172880275 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc62c06a0e0 msgr2=0x7fc62c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:03.611 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.610+0000 7fc65222e700 1 --2- 172.21.15.2:0/172880275 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc62c06a0e0 0x7fc62c06c5a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc63c006fd0 tx=0x7fc63c008040 comp rx=0 tx=0).stop 2023-08-25T03:35:03.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.610+0000 7fc65222e700 1 -- 172.21.15.2:0/172880275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc64c09dfa0 msgr2=0x7fc64c0c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:03.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.610+0000 7fc65222e700 1 --2- 172.21.15.2:0/172880275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc64c09dfa0 0x7fc64c0c9d20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc644012000 tx=0x7fc644004ad0 comp rx=0 tx=0).stop 2023-08-25T03:35:03.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.610+0000 7fc65222e700 1 -- 172.21.15.2:0/172880275 shutdown_connections 2023-08-25T03:35:03.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.610+0000 7fc65222e700 1 --2- 172.21.15.2:0/172880275 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc62c06a0e0 0x7fc62c06c5a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:03.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.610+0000 7fc65222e700 1 --2- 172.21.15.2:0/172880275 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc64c09dfa0 0x7fc64c0c9d20 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:03.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.610+0000 7fc65222e700 1 --2- 172.21.15.2:0/172880275 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc64c15e6e0 0x7fc64c14d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:03.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.610+0000 7fc65222e700 1 --2- 172.21.15.2:0/172880275 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc64c15eac0 0x7fc64c09da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:03.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.611+0000 7fc65222e700 1 -- 172.21.15.2:0/172880275 >> 172.21.15.2:0/172880275 conn(0x7fc64c09a2b0 msgr2=0x7fc64c15cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:03.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.611+0000 7fc65222e700 1 -- 172.21.15.2:0/172880275 shutdown_connections 2023-08-25T03:35:03.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.611+0000 7fc65222e700 1 -- 172.21.15.2:0/172880275 wait complete. 2023-08-25T03:35:03.623 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-08-25T03:35:03.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.749+0000 7fd483dc4700 1 Processor -- start 2023-08-25T03:35:03.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.749+0000 7fd483dc4700 1 -- start start 2023-08-25T03:35:03.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.750+0000 7fd483dc4700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd47c15f770 0x7fd47c0cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:03.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.750+0000 7fd483dc4700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd47c0b20b0 0x7fd47c15b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:03.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.750+0000 7fd483dc4700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd47c071b50 con 0x7fd47c0716a0 2023-08-25T03:35:03.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.750+0000 7fd483dc4700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd47c15edd0 con 0x7fd47c15f770 2023-08-25T03:35:03.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.750+0000 7fd483dc4700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c06de40 con 0x7fd47c0b20b0 2023-08-25T03:35:03.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.750+0000 7fd48135f700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fd47c0716a0 0x7fd47c0b19a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47874/0 (socket says 172.21.15.2:47874) 2023-08-25T03:35:03.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.750+0000 7fd48135f700 1 -- 172.21.15.2:0/682088755 learned_addr learned my addr 172.21.15.2:0/682088755 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:03.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.750+0000 7fd482361700 1 --2- 172.21.15.2:0/682088755 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd47c0b20b0 0x7fd47c15b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:03.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.750+0000 7fd481b60700 1 --2- 172.21.15.2:0/682088755 >> v2:172.21.15.31:3300/0 conn(0x7fd47c15f770 0x7fd47c0cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:03.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.750+0000 7fd480b5e700 1 -- 172.21.15.2:0/682088755 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941446078 0 0) 0x7fd47c071b50 con 0x7fd47c0716a0 2023-08-25T03:35:03.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.750+0000 7fd480b5e700 1 -- 172.21.15.2:0/682088755 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4680035b0 con 0x7fd47c0716a0 2023-08-25T03:35:03.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd480b5e700 1 -- 172.21.15.2:0/682088755 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 747793966 0 0) 0x7fd4680035b0 con 0x7fd47c0716a0 2023-08-25T03:35:03.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd480b5e700 1 -- 172.21.15.2:0/682088755 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd47c0b20b0 msgr2=0x7fd47c15b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:03.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd480b5e700 1 --2- 172.21.15.2:0/682088755 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd47c0b20b0 0x7fd47c15b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:03.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd480b5e700 1 -- 172.21.15.2:0/682088755 >> v2:172.21.15.31:3300/0 conn(0x7fd47c15f770 msgr2=0x7fd47c0cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:03.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd480b5e700 1 --2- 172.21.15.2:0/682088755 >> v2:172.21.15.31:3300/0 conn(0x7fd47c15f770 0x7fd47c0cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:03.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd480b5e700 1 -- 172.21.15.2:0/682088755 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47c06d080 con 0x7fd47c0716a0 2023-08-25T03:35:03.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd480b5e700 1 -- 172.21.15.2:0/682088755 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7fd478001670 con 0x7fd47c0716a0 2023-08-25T03:35:03.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd480b5e700 1 -- 172.21.15.2:0/682088755 >> v1:172.21.15.2:6789/0 conn(0x7fd47c0716a0 legacy=0x7fd47c0b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:03.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd480b5e700 1 --2- 172.21.15.2:0/682088755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd468003a00 0x7fd468003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:03.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd480b5e700 1 --2- 172.21.15.2:0/682088755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd468004850 0x7fd468004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:03.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd480b5e700 1 --2- 172.21.15.2:0/682088755 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd47c0b20b0 0x7fd468005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:03.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd480b5e700 1 -- 172.21.15.2:0/682088755 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c06de40 con 0x7fd468003a00 2023-08-25T03:35:03.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd480b5e700 1 -- 172.21.15.2:0/682088755 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46800ba10 con 0x7fd468004850 2023-08-25T03:35:03.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd480b5e700 1 -- 172.21.15.2:0/682088755 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46800bb50 con 0x7fd47c0b20b0 2023-08-25T03:35:03.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd481b60700 1 --2- 172.21.15.2:0/682088755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd468003a00 0x7fd468003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:03.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd482361700 1 --2- 172.21.15.2:0/682088755 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd47c0b20b0 0x7fd468005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:03.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd48135f700 1 --2- 172.21.15.2:0/682088755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd468004850 0x7fd468004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:03.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd481b60700 1 -- 172.21.15.2:0/682088755 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd47c0b20b0 msgr2=0x7fd468005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:03.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd481b60700 1 --2- 172.21.15.2:0/682088755 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd47c0b20b0 0x7fd468005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:03.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd481b60700 1 -- 172.21.15.2:0/682088755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd468004850 msgr2=0x7fd468004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:03.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd481b60700 1 --2- 172.21.15.2:0/682088755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd468004850 0x7fd468004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:03.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd481b60700 1 -- 172.21.15.2:0/682088755 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4680035b0 con 0x7fd468003a00 2023-08-25T03:35:03.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd48135f700 1 --2- 172.21.15.2:0/682088755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd468004850 0x7fd468004cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:03.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd482361700 1 --2- 172.21.15.2:0/682088755 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd47c0b20b0 0x7fd468005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:03.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd481b60700 1 --2- 172.21.15.2:0/682088755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd468003a00 0x7fd468003e20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009b90 tx=0x7fd468005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7302b9e8a3ed74ec server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:03.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd480b5e700 1 -- 172.21.15.2:0/682088755 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c00e6c0 con 0x7fd468003a00 2023-08-25T03:35:03.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.751+0000 7fd480b5e700 1 -- 172.21.15.2:0/682088755 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46c00e820 con 0x7fd468003a00 2023-08-25T03:35:03.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.752+0000 7fd480b5e700 1 -- 172.21.15.2:0/682088755 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46c00eb90 con 0x7fd468003a00 2023-08-25T03:35:03.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.752+0000 7fd483dc4700 1 -- 172.21.15.2:0/682088755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd468003a00 msgr2=0x7fd468003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:03.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.752+0000 7fd483dc4700 1 --2- 172.21.15.2:0/682088755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd468003a00 0x7fd468003e20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd46c009b90 tx=0x7fd468005880 comp rx=0 tx=0).stop 2023-08-25T03:35:03.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.752+0000 7fd482361700 1 -- 172.21.15.2:0/682088755 reap_dead start 2023-08-25T03:35:03.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.752+0000 7fd483dc4700 1 -- 172.21.15.2:0/682088755 shutdown_connections 2023-08-25T03:35:03.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.752+0000 7fd483dc4700 1 -- 172.21.15.2:0/682088755 >> 172.21.15.2:0/682088755 conn(0x7fd47c0cfea0 msgr2=0x7fd47c0d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:03.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.752+0000 7fd483dc4700 1 -- 172.21.15.2:0/682088755 shutdown_connections 2023-08-25T03:35:03.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.752+0000 7fd483dc4700 1 -- 172.21.15.2:0/682088755 wait complete. 2023-08-25T03:35:03.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.753+0000 7fd483dc4700 1 Processor -- start 2023-08-25T03:35:03.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.753+0000 7fd483dc4700 1 -- start start 2023-08-25T03:35:03.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.754+0000 7fd483dc4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd468003a00 0x7fd47c0c9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:03.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.754+0000 7fd483dc4700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd47c0c6ee0 0x7fd47c0c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:03.768 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.754+0000 7fd483dc4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd47c0c8f80 0x7fd47c0c9400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:03.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.754+0000 7fd483dc4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c15de30 con 0x7fd47c0c8f80 2023-08-25T03:35:03.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.754+0000 7fd483dc4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c0c9940 con 0x7fd468003a00 2023-08-25T03:35:03.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.754+0000 7fd483dc4700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd47c15d220 con 0x7fd47c0c6ee0 2023-08-25T03:35:03.769 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.754+0000 7fd482361700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd47c0c8f80 0x7fd47c0c9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:03.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.754+0000 7fd482361700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd47c0c8f80 0x7fd47c0c9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:32992/0 (socket says 172.21.15.2:32992) 2023-08-25T03:35:03.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.754+0000 7fd482361700 1 -- 172.21.15.2:0/473666425 learned_addr learned my addr 172.21.15.2:0/473666425 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:03.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.754+0000 7fd481b60700 1 --2- 172.21.15.2:0/473666425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd468003a00 0x7fd47c0c9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:03.770 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.754+0000 7fd48135f700 1 --2- 172.21.15.2:0/473666425 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd47c0c6ee0 0x7fd47c0c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:03.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.754+0000 7fd482361700 1 -- 172.21.15.2:0/473666425 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd47c0c6ee0 msgr2=0x7fd47c0c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:03.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.754+0000 7fd482361700 1 --2- 172.21.15.2:0/473666425 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd47c0c6ee0 0x7fd47c0c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:03.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.754+0000 7fd482361700 1 -- 172.21.15.2:0/473666425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd468003a00 msgr2=0x7fd47c0c9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:03.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.754+0000 7fd482361700 1 --2- 172.21.15.2:0/473666425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd468003a00 0x7fd47c0c9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:03.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.754+0000 7fd482361700 1 -- 172.21.15.2:0/473666425 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd47c15d500 con 0x7fd47c0c8f80 2023-08-25T03:35:03.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.755+0000 7fd482361700 1 --2- 172.21.15.2:0/473666425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd47c0c8f80 0x7fd47c0c9400 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd47c070d00 tx=0x7fd4740104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:03.771 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.755+0000 7fd4727fc700 1 -- 172.21.15.2:0/473666425 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47401b070 con 0x7fd47c0c8f80 2023-08-25T03:35:03.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.755+0000 7fd4727fc700 1 -- 172.21.15.2:0/473666425 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd47400f040 con 0x7fd47c0c8f80 2023-08-25T03:35:03.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.755+0000 7fd483dc4700 1 -- 172.21.15.2:0/473666425 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd47c15d7e0 con 0x7fd47c0c8f80 2023-08-25T03:35:03.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.755+0000 7fd4727fc700 1 -- 172.21.15.2:0/473666425 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47400e050 con 0x7fd47c0c8f80 2023-08-25T03:35:03.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.755+0000 7fd483dc4700 1 -- 172.21.15.2:0/473666425 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd47c0c7360 con 0x7fd47c0c8f80 2023-08-25T03:35:03.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.756+0000 7fd483dc4700 1 -- 172.21.15.2:0/473666425 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd47c066f70 con 0x7fd47c0c8f80 2023-08-25T03:35:03.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.756+0000 7fd4727fc700 1 -- 172.21.15.2:0/473666425 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd474003a20 con 0x7fd47c0c8f80 2023-08-25T03:35:03.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.757+0000 7fd4727fc700 1 --2- 172.21.15.2:0/473666425 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd46006a110 0x7fd46006c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:03.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.757+0000 7fd481b60700 1 --2- 172.21.15.2:0/473666425 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd46006a110 0x7fd46006c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:03.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.757+0000 7fd4727fc700 1 -- 172.21.15.2:0/473666425 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd4740884f0 con 0x7fd47c0c8f80 2023-08-25T03:35:03.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.757+0000 7fd481b60700 1 --2- 172.21.15.2:0/473666425 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd46006a110 0x7fd46006c5d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd46c0090b0 tx=0x7fd46c01f040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:03.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:03.761+0000 7fd4727fc700 1 -- 172.21.15.2:0/473666425 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd474053eb0 con 0x7fd47c0c8f80 2023-08-25T03:35:04.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.047+0000 7fd483dc4700 1 -- 172.21.15.2:0/473666425 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fd47c15e1c0 con 0x7fd47c0c8f80 2023-08-25T03:35:04.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.053+0000 7fd4727fc700 1 -- 172.21.15.2:0/473666425 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd4740574d0 con 0x7fd47c0c8f80 2023-08-25T03:35:04.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.055+0000 7fd483dc4700 1 -- 172.21.15.2:0/473666425 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd46006a110 msgr2=0x7fd46006c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:04.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.055+0000 7fd483dc4700 1 --2- 172.21.15.2:0/473666425 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd46006a110 0x7fd46006c5d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd46c0090b0 tx=0x7fd46c01f040 comp rx=0 tx=0).stop 2023-08-25T03:35:04.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.055+0000 7fd483dc4700 1 -- 172.21.15.2:0/473666425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd47c0c8f80 msgr2=0x7fd47c0c9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:04.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.055+0000 7fd483dc4700 1 --2- 172.21.15.2:0/473666425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd47c0c8f80 0x7fd47c0c9400 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fd47c070d00 tx=0x7fd4740104a0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.055+0000 7fd483dc4700 1 -- 172.21.15.2:0/473666425 shutdown_connections 2023-08-25T03:35:04.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.056+0000 7fd483dc4700 1 --2- 172.21.15.2:0/473666425 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd46006a110 0x7fd46006c5d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.056+0000 7fd483dc4700 1 --2- 172.21.15.2:0/473666425 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd47c0c8f80 0x7fd47c0c9400 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.056+0000 7fd483dc4700 1 --2- 172.21.15.2:0/473666425 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd468003a00 0x7fd47c0c9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.056+0000 7fd483dc4700 1 --2- 172.21.15.2:0/473666425 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd47c0c6ee0 0x7fd47c0c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.056+0000 7fd483dc4700 1 -- 172.21.15.2:0/473666425 >> 172.21.15.2:0/473666425 conn(0x7fd47c0cfea0 msgr2=0x7fd47c0afb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:04.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.056+0000 7fd483dc4700 1 -- 172.21.15.2:0/473666425 shutdown_connections 2023-08-25T03:35:04.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.056+0000 7fd483dc4700 1 -- 172.21.15.2:0/473666425 wait complete. 2023-08-25T03:35:04.070 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:35:04.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.194+0000 7fd907dbe700 1 Processor -- start 2023-08-25T03:35:04.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.195+0000 7fd907dbe700 1 -- start start 2023-08-25T03:35:04.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.195+0000 7fd907dbe700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd9000b1fb0 0x7fd900160f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:04.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.195+0000 7fd907dbe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd90006b8c0 0x7fd90006bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:04.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.195+0000 7fd907dbe700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd90006c230 con 0x7fd900161450 2023-08-25T03:35:04.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.195+0000 7fd907dbe700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd90006d450 con 0x7fd90006b8c0 2023-08-25T03:35:04.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.195+0000 7fd907dbe700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd90006d730 con 0x7fd9000b1fb0 2023-08-25T03:35:04.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.196+0000 7fd90635b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd90006b8c0 0x7fd90006bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:04.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.196+0000 7fd905b5a700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd9000b1fb0 0x7fd900160f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:04.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.196+0000 7fd90635b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd90006b8c0 0x7fd90006bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.2:52520/0 (socket says 172.21.15.2:52520) 2023-08-25T03:35:04.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.196+0000 7fd90635b700 1 -- 172.21.15.2:0/2051165398 learned_addr learned my addr 172.21.15.2:0/2051165398 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:04.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.196+0000 7fd90635b700 1 -- 172.21.15.2:0/2051165398 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd9000b1fb0 msgr2=0x7fd900160f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:04.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.196+0000 7fd90635b700 1 --2- 172.21.15.2:0/2051165398 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd9000b1fb0 0x7fd900160f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.196+0000 7fd90635b700 1 -- 172.21.15.2:0/2051165398 >> v1:172.21.15.31:6789/0 conn(0x7fd900161450 legacy=0x7fd9000cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:04.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.196+0000 7fd90635b700 1 -- 172.21.15.2:0/2051165398 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90006da10 con 0x7fd90006b8c0 2023-08-25T03:35:04.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.196+0000 7fd904b58700 1 -- 172.21.15.2:0/2051165398 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 701110955 0 0) 0x7fd90006c230 con 0x7fd900161450 2023-08-25T03:35:04.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.196+0000 7fd905b5a700 1 --2- 172.21.15.2:0/2051165398 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd9000b1fb0 0x7fd900160f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:35:04.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.196+0000 7fd90635b700 1 --2- 172.21.15.2:0/2051165398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd90006b8c0 0x7fd90006bcf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8009e50 tx=0x7fd8f800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e5339b4fa970023 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:04.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.197+0000 7fd904b58700 1 -- 172.21.15.2:0/2051165398 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f800e070 con 0x7fd90006b8c0 2023-08-25T03:35:04.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.197+0000 7fd904b58700 1 -- 172.21.15.2:0/2051165398 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f800b910 con 0x7fd90006b8c0 2023-08-25T03:35:04.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.197+0000 7fd904b58700 1 -- 172.21.15.2:0/2051165398 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f800bcc0 con 0x7fd90006b8c0 2023-08-25T03:35:04.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.197+0000 7fd907dbe700 1 -- 172.21.15.2:0/2051165398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd90006b8c0 msgr2=0x7fd90006bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:04.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.197+0000 7fd907dbe700 1 --2- 172.21.15.2:0/2051165398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd90006b8c0 0x7fd90006bcf0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd8f8009e50 tx=0x7fd8f800b330 comp rx=0 tx=0).stop 2023-08-25T03:35:04.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.197+0000 7fd907dbe700 1 -- 172.21.15.2:0/2051165398 shutdown_connections 2023-08-25T03:35:04.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.197+0000 7fd907dbe700 1 --2- 172.21.15.2:0/2051165398 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd90006b8c0 0x7fd90006bcf0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.197+0000 7fd907dbe700 1 --2- 172.21.15.2:0/2051165398 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd9000b1fb0 0x7fd900160f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.204 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.197+0000 7fd907dbe700 1 -- 172.21.15.2:0/2051165398 >> 172.21.15.2:0/2051165398 conn(0x7fd9000cfea0 msgr2=0x7fd9000d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:04.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.197+0000 7fd907dbe700 1 -- 172.21.15.2:0/2051165398 shutdown_connections 2023-08-25T03:35:04.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.197+0000 7fd907dbe700 1 -- 172.21.15.2:0/2051165398 wait complete. 2023-08-25T03:35:04.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.198+0000 7fd907dbe700 1 Processor -- start 2023-08-25T03:35:04.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.198+0000 7fd907dbe700 1 -- start start 2023-08-25T03:35:04.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.199+0000 7fd907dbe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd90006b8c0 0x7fd9000c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:04.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.199+0000 7fd907dbe700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd9000b1fb0 0x7fd9000b41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:04.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.199+0000 7fd907dbe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd900156510 0x7fd9000b46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:04.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.199+0000 7fd907dbe700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd900151d50 con 0x7fd90006b8c0 2023-08-25T03:35:04.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.199+0000 7fd907dbe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd900152030 con 0x7fd900156510 2023-08-25T03:35:04.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.199+0000 7fd907dbe700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd900152310 con 0x7fd9000b1fb0 2023-08-25T03:35:04.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.199+0000 7fd905b5a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd90006b8c0 0x7fd9000c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:04.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.199+0000 7fd905b5a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd90006b8c0 0x7fd9000c9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33000/0 (socket says 172.21.15.2:33000) 2023-08-25T03:35:04.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.199+0000 7fd905b5a700 1 -- 172.21.15.2:0/1830985478 learned_addr learned my addr 172.21.15.2:0/1830985478 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:04.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.200+0000 7fd90635b700 1 --2- 172.21.15.2:0/1830985478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd900156510 0x7fd9000b46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:04.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.200+0000 7fd905359700 1 --2- 172.21.15.2:0/1830985478 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd9000b1fb0 0x7fd9000b41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:04.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.200+0000 7fd905b5a700 1 -- 172.21.15.2:0/1830985478 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd9000b1fb0 msgr2=0x7fd9000b41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:04.207 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.200+0000 7fd905b5a700 1 --2- 172.21.15.2:0/1830985478 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd9000b1fb0 0x7fd9000b41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.200+0000 7fd905b5a700 1 -- 172.21.15.2:0/1830985478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd900156510 msgr2=0x7fd9000b46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:04.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.200+0000 7fd905b5a700 1 --2- 172.21.15.2:0/1830985478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd900156510 0x7fd9000b46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.200+0000 7fd905b5a700 1 -- 172.21.15.2:0/1830985478 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8f00094a0 con 0x7fd90006b8c0 2023-08-25T03:35:04.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.200+0000 7fd90635b700 1 --2- 172.21.15.2:0/1830985478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd900156510 0x7fd9000b46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:04.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.200+0000 7fd905b5a700 1 --2- 172.21.15.2:0/1830985478 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd90006b8c0 0x7fd9000c9b60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000ebe0 tx=0x7fd8f0009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:04.208 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.200+0000 7fd8f67fc700 1 -- 172.21.15.2:0/1830985478 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f000c910 con 0x7fd90006b8c0 2023-08-25T03:35:04.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.200+0000 7fd8f67fc700 1 -- 172.21.15.2:0/1830985478 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f000ca70 con 0x7fd90006b8c0 2023-08-25T03:35:04.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.200+0000 7fd907dbe700 1 -- 172.21.15.2:0/1830985478 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f8009b30 con 0x7fd90006b8c0 2023-08-25T03:35:04.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.200+0000 7fd8f67fc700 1 -- 172.21.15.2:0/1830985478 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f000cde0 con 0x7fd90006b8c0 2023-08-25T03:35:04.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.200+0000 7fd907dbe700 1 -- 172.21.15.2:0/1830985478 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9000c7780 con 0x7fd90006b8c0 2023-08-25T03:35:04.209 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.201+0000 7fd907dbe700 1 -- 172.21.15.2:0/1830985478 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9000c7af0 con 0x7fd90006b8c0 2023-08-25T03:35:04.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.201+0000 7fd8f67fc700 1 -- 172.21.15.2:0/1830985478 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd8f0013470 con 0x7fd90006b8c0 2023-08-25T03:35:04.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.202+0000 7fd8f67fc700 1 --2- 172.21.15.2:0/1830985478 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd8e406a0e0 0x7fd8e406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:04.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.202+0000 7fd8f67fc700 1 -- 172.21.15.2:0/1830985478 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd8f0088400 con 0x7fd90006b8c0 2023-08-25T03:35:04.210 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.202+0000 7fd905359700 1 --2- 172.21.15.2:0/1830985478 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd8e406a0e0 0x7fd8e406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:04.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.202+0000 7fd905359700 1 --2- 172.21.15.2:0/1830985478 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd8e406a0e0 0x7fd8e406c5a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd90015d3c0 tx=0x7fd8fc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:04.211 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.207+0000 7fd8f67fc700 1 -- 172.21.15.2:0/1830985478 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd8f0053e40 con 0x7fd90006b8c0 2023-08-25T03:35:04.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.408+0000 7fd907dbe700 1 -- 172.21.15.2:0/1830985478 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd900066f70 con 0x7fd90006b8c0 2023-08-25T03:35:04.413 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.409+0000 7fd8f67fc700 1 -- 172.21.15.2:0/1830985478 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+14495 (secure 0 0 0) 0x7fd8f0057460 con 0x7fd90006b8c0 2023-08-25T03:35:04.413 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:35:04.414 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":33,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:35:00.906351+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:35:04.421 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.411+0000 7fd907dbe700 1 -- 172.21.15.2:0/1830985478 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd8e406a0e0 msgr2=0x7fd8e406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:04.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.411+0000 7fd907dbe700 1 --2- 172.21.15.2:0/1830985478 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd8e406a0e0 0x7fd8e406c5a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd90015d3c0 tx=0x7fd8fc009040 comp rx=0 tx=0).stop 2023-08-25T03:35:04.423 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.411+0000 7fd907dbe700 1 -- 172.21.15.2:0/1830985478 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd90006b8c0 msgr2=0x7fd9000c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:04.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.411+0000 7fd907dbe700 1 --2- 172.21.15.2:0/1830985478 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd90006b8c0 0x7fd9000c9b60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000ebe0 tx=0x7fd8f0009ec0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.412+0000 7fd907dbe700 1 -- 172.21.15.2:0/1830985478 shutdown_connections 2023-08-25T03:35:04.425 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.412+0000 7fd907dbe700 1 --2- 172.21.15.2:0/1830985478 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd8e406a0e0 0x7fd8e406c5a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.412+0000 7fd907dbe700 1 --2- 172.21.15.2:0/1830985478 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd90006b8c0 0x7fd9000c9b60 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.412+0000 7fd907dbe700 1 --2- 172.21.15.2:0/1830985478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd900156510 0x7fd9000b46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.412+0000 7fd907dbe700 1 --2- 172.21.15.2:0/1830985478 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd9000b1fb0 0x7fd9000b41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.412+0000 7fd907dbe700 1 -- 172.21.15.2:0/1830985478 >> 172.21.15.2:0/1830985478 conn(0x7fd9000cfea0 msgr2=0x7fd90006bca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:04.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.412+0000 7fd907dbe700 1 -- 172.21.15.2:0/1830985478 shutdown_connections 2023-08-25T03:35:04.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.412+0000 7fd907dbe700 1 -- 172.21.15.2:0/1830985478 wait complete. 2023-08-25T03:35:04.431 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T03:33:49.560643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T03:33:56.265184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T03:35:04.432 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T03:35:04.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.559+0000 7fed48aa8700 1 Processor -- start 2023-08-25T03:35:04.560 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.559+0000 7fed48aa8700 1 -- start start 2023-08-25T03:35:04.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.560+0000 7fed48aa8700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fed440cacf0 0x7fed440cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:04.562 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.560+0000 7fed48aa8700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed4406eaf0 con 0x7fed440cedf0 2023-08-25T03:35:04.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.560+0000 7fed48aa8700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed4415e2c0 con 0x7fed44159f10 2023-08-25T03:35:04.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.560+0000 7fed48aa8700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed4406e850 con 0x7fed440cacf0 2023-08-25T03:35:04.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.560+0000 7fed42d9d700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fed440cacf0 0x7fed440cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:04.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.560+0000 7fed41d9b700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fed440cedf0 0x7fed4406ef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47906/0 (socket says 172.21.15.2:47906) 2023-08-25T03:35:04.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.560+0000 7fed41d9b700 1 -- 172.21.15.2:0/989489884 learned_addr learned my addr 172.21.15.2:0/989489884 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:04.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.561+0000 7fed42d9d700 1 -- 172.21.15.2:0/989489884 >> v1:172.21.15.31:6789/0 conn(0x7fed44159f10 legacy=0x7fed4415dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:04.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.561+0000 7fed42d9d700 1 -- 172.21.15.2:0/989489884 >> v1:172.21.15.2:6789/0 conn(0x7fed440cedf0 legacy=0x7fed4406ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:04.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.561+0000 7fed4159a700 1 -- 172.21.15.2:0/989489884 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3617157013 0 0) 0x7fed4406eaf0 con 0x7fed440cedf0 2023-08-25T03:35:04.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.561+0000 7fed42d9d700 1 -- 172.21.15.2:0/989489884 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed4415e5a0 con 0x7fed440cacf0 2023-08-25T03:35:04.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.561+0000 7fed42d9d700 1 --2- 172.21.15.2:0/989489884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed440cacf0 0x7fed440cb1a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009d90 tx=0x7fed3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d095ddcffbd6f492 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:04.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.561+0000 7fed4159a700 1 -- 172.21.15.2:0/989489884 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c00e070 con 0x7fed440cacf0 2023-08-25T03:35:04.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.561+0000 7fed4159a700 1 -- 172.21.15.2:0/989489884 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3c00bbd0 con 0x7fed440cacf0 2023-08-25T03:35:04.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.561+0000 7fed4159a700 1 -- 172.21.15.2:0/989489884 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c012430 con 0x7fed440cacf0 2023-08-25T03:35:04.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.561+0000 7fed48aa8700 1 -- 172.21.15.2:0/989489884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed440cacf0 msgr2=0x7fed440cb1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:04.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.561+0000 7fed48aa8700 1 --2- 172.21.15.2:0/989489884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed440cacf0 0x7fed440cb1a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009d90 tx=0x7fed3c00b330 comp rx=0 tx=0).stop 2023-08-25T03:35:04.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.562+0000 7fed48aa8700 1 -- 172.21.15.2:0/989489884 shutdown_connections 2023-08-25T03:35:04.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.562+0000 7fed48aa8700 1 --2- 172.21.15.2:0/989489884 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed440cacf0 0x7fed440cb1a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.562+0000 7fed48aa8700 1 -- 172.21.15.2:0/989489884 >> 172.21.15.2:0/989489884 conn(0x7fed440cfd90 msgr2=0x7fed440d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:04.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.562+0000 7fed48aa8700 1 -- 172.21.15.2:0/989489884 shutdown_connections 2023-08-25T03:35:04.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.562+0000 7fed48aa8700 1 -- 172.21.15.2:0/989489884 wait complete. 2023-08-25T03:35:04.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.563+0000 7fed48aa8700 1 Processor -- start 2023-08-25T03:35:04.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.563+0000 7fed48aa8700 1 -- start start 2023-08-25T03:35:04.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.564+0000 7fed48aa8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed440cacf0 0x7fed4415b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:04.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.564+0000 7fed48aa8700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fed440cedf0 0x7fed4414ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:04.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.564+0000 7fed48aa8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed44159f10 0x7fed440b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:04.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.564+0000 7fed48aa8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed440c8960 con 0x7fed44159f10 2023-08-25T03:35:04.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.564+0000 7fed48aa8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4415bf60 con 0x7fed440cacf0 2023-08-25T03:35:04.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.564+0000 7fed48aa8700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed4415c240 con 0x7fed440cedf0 2023-08-25T03:35:04.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.564+0000 7fed42d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed44159f10 0x7fed440b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:04.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.565+0000 7fed42d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed44159f10 0x7fed440b4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33018/0 (socket says 172.21.15.2:33018) 2023-08-25T03:35:04.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.565+0000 7fed42d9d700 1 -- 172.21.15.2:0/3882861224 learned_addr learned my addr 172.21.15.2:0/3882861224 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:04.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.565+0000 7fed4259c700 1 --2- 172.21.15.2:0/3882861224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed440cacf0 0x7fed4415b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:04.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.565+0000 7fed41d9b700 1 --2- 172.21.15.2:0/3882861224 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fed440cedf0 0x7fed4414ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:04.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.565+0000 7fed42d9d700 1 -- 172.21.15.2:0/3882861224 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fed440cedf0 msgr2=0x7fed4414ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:04.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.565+0000 7fed42d9d700 1 --2- 172.21.15.2:0/3882861224 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fed440cedf0 0x7fed4414ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.565+0000 7fed42d9d700 1 -- 172.21.15.2:0/3882861224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed440cacf0 msgr2=0x7fed4415b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:04.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.565+0000 7fed42d9d700 1 --2- 172.21.15.2:0/3882861224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed440cacf0 0x7fed4415b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.565+0000 7fed42d9d700 1 -- 172.21.15.2:0/3882861224 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed3c009a70 con 0x7fed44159f10 2023-08-25T03:35:04.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.565+0000 7fed4259c700 1 --2- 172.21.15.2:0/3882861224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed440cacf0 0x7fed4415b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:35:04.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.565+0000 7fed42d9d700 1 --2- 172.21.15.2:0/3882861224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed44159f10 0x7fed440b4c00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fed44072b20 tx=0x7fed3c004b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:04.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.565+0000 7fed32ffd700 1 -- 172.21.15.2:0/3882861224 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c01b8d0 con 0x7fed44159f10 2023-08-25T03:35:04.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.565+0000 7fed32ffd700 1 -- 172.21.15.2:0/3882861224 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3c020070 con 0x7fed44159f10 2023-08-25T03:35:04.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.565+0000 7fed32ffd700 1 -- 172.21.15.2:0/3882861224 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3c01bc90 con 0x7fed44159f10 2023-08-25T03:35:04.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.565+0000 7fed48aa8700 1 -- 172.21.15.2:0/3882861224 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed4415c520 con 0x7fed44159f10 2023-08-25T03:35:04.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.565+0000 7fed48aa8700 1 -- 172.21.15.2:0/3882861224 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed4415cae0 con 0x7fed44159f10 2023-08-25T03:35:04.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.566+0000 7fed48aa8700 1 -- 172.21.15.2:0/3882861224 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed4415ccd0 con 0x7fed44159f10 2023-08-25T03:35:04.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.567+0000 7fed32ffd700 1 -- 172.21.15.2:0/3882861224 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fed3c01c400 con 0x7fed44159f10 2023-08-25T03:35:04.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.567+0000 7fed32ffd700 1 --2- 172.21.15.2:0/3882861224 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fed2406a0e0 0x7fed2406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:04.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.567+0000 7fed32ffd700 1 -- 172.21.15.2:0/3882861224 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fed3c022070 con 0x7fed44159f10 2023-08-25T03:35:04.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.567+0000 7fed4259c700 1 --2- 172.21.15.2:0/3882861224 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fed2406a0e0 0x7fed2406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:04.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.568+0000 7fed4259c700 1 --2- 172.21.15.2:0/3882861224 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fed2406a0e0 0x7fed2406c5a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fed38000f60 tx=0x7fed38009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:04.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.572+0000 7fed32ffd700 1 -- 172.21.15.2:0/3882861224 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fed3c059920 con 0x7fed44159f10 2023-08-25T03:35:04.772 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.771+0000 7fed48aa8700 1 -- 172.21.15.2:0/3882861224 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fed440746d0 con 0x7fed44159f10 2023-08-25T03:35:04.773 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.772+0000 7fed32ffd700 1 -- 172.21.15.2:0/3882861224 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v33) v1 ==== 93+0+10 (secure 0 0 0) 0x7fed3c0594b0 con 0x7fed44159f10 2023-08-25T03:35:04.773 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-08-25T03:35:04.774 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.774+0000 7fed48aa8700 1 -- 172.21.15.2:0/3882861224 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fed2406a0e0 msgr2=0x7fed2406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:04.775 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.774+0000 7fed48aa8700 1 --2- 172.21.15.2:0/3882861224 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fed2406a0e0 0x7fed2406c5a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fed38000f60 tx=0x7fed38009040 comp rx=0 tx=0).stop 2023-08-25T03:35:04.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.774+0000 7fed48aa8700 1 -- 172.21.15.2:0/3882861224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed44159f10 msgr2=0x7fed440b4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:04.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.774+0000 7fed48aa8700 1 --2- 172.21.15.2:0/3882861224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed44159f10 0x7fed440b4c00 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fed44072b20 tx=0x7fed3c004b20 comp rx=0 tx=0).stop 2023-08-25T03:35:04.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.774+0000 7fed48aa8700 1 -- 172.21.15.2:0/3882861224 shutdown_connections 2023-08-25T03:35:04.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.774+0000 7fed48aa8700 1 --2- 172.21.15.2:0/3882861224 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fed2406a0e0 0x7fed2406c5a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.774+0000 7fed48aa8700 1 --2- 172.21.15.2:0/3882861224 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fed44159f10 0x7fed440b4c00 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.774+0000 7fed48aa8700 1 --2- 172.21.15.2:0/3882861224 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fed440cacf0 0x7fed4415b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.774+0000 7fed48aa8700 1 --2- 172.21.15.2:0/3882861224 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fed440cedf0 0x7fed4414ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.774+0000 7fed48aa8700 1 -- 172.21.15.2:0/3882861224 >> 172.21.15.2:0/3882861224 conn(0x7fed440cfd90 msgr2=0x7fed440cb810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:04.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.774+0000 7fed48aa8700 1 -- 172.21.15.2:0/3882861224 shutdown_connections 2023-08-25T03:35:04.777 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.774+0000 7fed48aa8700 1 -- 172.21.15.2:0/3882861224 wait complete. 2023-08-25T03:35:04.787 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T03:35:04.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.915+0000 7f5a8fad4700 1 Processor -- start 2023-08-25T03:35:04.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.915+0000 7f5a8fad4700 1 -- start start 2023-08-25T03:35:04.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.916+0000 7f5a8fad4700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5a88159d10 0x7f5a88072370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:04.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.916+0000 7f5a8fad4700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5a880b4590 0x7f5a880b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:04.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.916+0000 7f5a8fad4700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a880728b0 con 0x7f5a880ce050 2023-08-25T03:35:04.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.916+0000 7f5a8fad4700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a8806e2e0 con 0x7f5a88159d10 2023-08-25T03:35:04.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.916+0000 7f5a8fad4700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a8806dac0 con 0x7f5a880b4590 2023-08-25T03:35:04.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.916+0000 7f5a8d870700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5a88159d10 0x7f5a88072370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:04.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.916+0000 7f5a8d06f700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f5a880ce050 0x7f5a880ce4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47920/0 (socket says 172.21.15.2:47920) 2023-08-25T03:35:04.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.916+0000 7f5a8d06f700 1 -- 172.21.15.2:0/1524222826 learned_addr learned my addr 172.21.15.2:0/1524222826 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:04.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.916+0000 7f5a8d870700 1 -- 172.21.15.2:0/1524222826 >> v2:172.21.15.31:3300/0 conn(0x7f5a880b4590 msgr2=0x7f5a880b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:04.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.916+0000 7f5a8e071700 1 --2- 172.21.15.2:0/1524222826 >> v2:172.21.15.31:3300/0 conn(0x7f5a880b4590 0x7f5a880b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:04.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.916+0000 7f5a8d870700 1 --2- 172.21.15.2:0/1524222826 >> v2:172.21.15.31:3300/0 conn(0x7f5a880b4590 0x7f5a880b49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.916+0000 7f5a8d870700 1 -- 172.21.15.2:0/1524222826 >> v1:172.21.15.2:6789/0 conn(0x7f5a880ce050 legacy=0x7f5a880ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:04.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.916+0000 7f5a8d870700 1 -- 172.21.15.2:0/1524222826 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8806a1e0 con 0x7f5a88159d10 2023-08-25T03:35:04.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.916+0000 7f5a8c86e700 1 -- 172.21.15.2:0/1524222826 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1996400306 0 0) 0x7f5a84001110 con 0x7f5a880ce050 2023-08-25T03:35:04.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.917+0000 7f5a8d870700 1 --2- 172.21.15.2:0/1524222826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a88159d10 0x7f5a88072370 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5a880cbf50 tx=0x7f5a78009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6082f7996a908877 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:04.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.917+0000 7f5a8c86e700 1 -- 172.21.15.2:0/1524222826 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a78005e30 con 0x7f5a88159d10 2023-08-25T03:35:04.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.917+0000 7f5a8c86e700 1 -- 172.21.15.2:0/1524222826 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a78006030 con 0x7f5a88159d10 2023-08-25T03:35:04.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.917+0000 7f5a8c86e700 1 -- 172.21.15.2:0/1524222826 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a780063c0 con 0x7f5a88159d10 2023-08-25T03:35:04.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.917+0000 7f5a8fad4700 1 -- 172.21.15.2:0/1524222826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a88159d10 msgr2=0x7f5a88072370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:04.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.917+0000 7f5a8fad4700 1 --2- 172.21.15.2:0/1524222826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a88159d10 0x7f5a88072370 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5a880cbf50 tx=0x7f5a78009b00 comp rx=0 tx=0).stop 2023-08-25T03:35:04.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.917+0000 7f5a8fad4700 1 -- 172.21.15.2:0/1524222826 shutdown_connections 2023-08-25T03:35:04.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.917+0000 7f5a8fad4700 1 --2- 172.21.15.2:0/1524222826 >> v2:172.21.15.31:3300/0 conn(0x7f5a880b4590 0x7f5a880b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.917+0000 7f5a8fad4700 1 --2- 172.21.15.2:0/1524222826 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a88159d10 0x7f5a88072370 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.917+0000 7f5a8fad4700 1 -- 172.21.15.2:0/1524222826 >> 172.21.15.2:0/1524222826 conn(0x7f5a880cf500 msgr2=0x7f5a880cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:04.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.917+0000 7f5a8fad4700 1 -- 172.21.15.2:0/1524222826 shutdown_connections 2023-08-25T03:35:04.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.917+0000 7f5a8fad4700 1 -- 172.21.15.2:0/1524222826 wait complete. 2023-08-25T03:35:04.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.919+0000 7f5a8fad4700 1 Processor -- start 2023-08-25T03:35:04.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.919+0000 7f5a8fad4700 1 -- start start 2023-08-25T03:35:04.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.919+0000 7f5a8fad4700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5a880b4590 0x7f5a881567b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:04.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.919+0000 7f5a8fad4700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a880ce050 0x7f5a880ad140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:04.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.919+0000 7f5a8fad4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a88159d10 0x7f5a880ad680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:04.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.919+0000 7f5a8fad4700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a880c6540 con 0x7f5a88159d10 2023-08-25T03:35:04.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.919+0000 7f5a8fad4700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a881e54d0 con 0x7f5a880ce050 2023-08-25T03:35:04.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.919+0000 7f5a8fad4700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a88156ee0 con 0x7f5a880b4590 2023-08-25T03:35:04.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.920+0000 7f5a8e071700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a88159d10 0x7f5a880ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:04.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.920+0000 7f5a8e071700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a88159d10 0x7f5a880ad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33030/0 (socket says 172.21.15.2:33030) 2023-08-25T03:35:04.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.920+0000 7f5a8e071700 1 -- 172.21.15.2:0/54854091 learned_addr learned my addr 172.21.15.2:0/54854091 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:04.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.920+0000 7f5a8d870700 1 --2- 172.21.15.2:0/54854091 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5a880b4590 0x7f5a881567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:04.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.920+0000 7f5a8d06f700 1 --2- 172.21.15.2:0/54854091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a880ce050 0x7f5a880ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:04.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.920+0000 7f5a8e071700 1 -- 172.21.15.2:0/54854091 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5a880b4590 msgr2=0x7f5a881567b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:04.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.920+0000 7f5a8e071700 1 --2- 172.21.15.2:0/54854091 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5a880b4590 0x7f5a881567b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.920+0000 7f5a8e071700 1 -- 172.21.15.2:0/54854091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a880ce050 msgr2=0x7f5a880ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:04.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.920+0000 7f5a8e071700 1 --2- 172.21.15.2:0/54854091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a880ce050 0x7f5a880ad140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:04.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.920+0000 7f5a8e071700 1 -- 172.21.15.2:0/54854091 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a78009430 con 0x7f5a88159d10 2023-08-25T03:35:04.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.920+0000 7f5a8d870700 1 --2- 172.21.15.2:0/54854091 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5a880b4590 0x7f5a881567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:35:04.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.920+0000 7f5a8e071700 1 --2- 172.21.15.2:0/54854091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a88159d10 0x7f5a880ad680 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000cc30 tx=0x7f5a8000a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:04.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.920+0000 7f5a7e7fc700 1 -- 172.21.15.2:0/54854091 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8000ab70 con 0x7f5a88159d10 2023-08-25T03:35:04.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.920+0000 7f5a7e7fc700 1 -- 172.21.15.2:0/54854091 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8000acd0 con 0x7f5a88159d10 2023-08-25T03:35:04.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.920+0000 7f5a8fad4700 1 -- 172.21.15.2:0/54854091 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a881571c0 con 0x7f5a88159d10 2023-08-25T03:35:04.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.920+0000 7f5a7e7fc700 1 -- 172.21.15.2:0/54854091 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a80003c40 con 0x7f5a88159d10 2023-08-25T03:35:04.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.920+0000 7f5a8fad4700 1 -- 172.21.15.2:0/54854091 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a88157780 con 0x7f5a88159d10 2023-08-25T03:35:04.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.921+0000 7f5a8fad4700 1 -- 172.21.15.2:0/54854091 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a881579a0 con 0x7f5a88159d10 2023-08-25T03:35:04.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.922+0000 7f5a7e7fc700 1 -- 172.21.15.2:0/54854091 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5a8000f3f0 con 0x7f5a88159d10 2023-08-25T03:35:04.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.922+0000 7f5a7e7fc700 1 --2- 172.21.15.2:0/54854091 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5a6c06a0e0 0x7f5a6c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:04.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.922+0000 7f5a7e7fc700 1 -- 172.21.15.2:0/54854091 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f5a80088340 con 0x7f5a88159d10 2023-08-25T03:35:04.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.922+0000 7f5a8d870700 1 --2- 172.21.15.2:0/54854091 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5a6c06a0e0 0x7f5a6c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:04.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.922+0000 7f5a8d870700 1 --2- 172.21.15.2:0/54854091 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5a6c06a0e0 0x7f5a6c06c5a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5a78013040 tx=0x7f5a78008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:04.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:04.926+0000 7f5a7e7fc700 1 -- 172.21.15.2:0/54854091 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5a80053e10 con 0x7f5a88159d10 2023-08-25T03:35:05.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.126+0000 7f5a8fad4700 1 -- 172.21.15.2:0/54854091 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5a8806ac70 con 0x7f5a88159d10 2023-08-25T03:35:05.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.126+0000 7f5a7e7fc700 1 -- 172.21.15.2:0/54854091 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v33) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5a8001d020 con 0x7f5a88159d10 2023-08-25T03:35:05.127 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 8 2023-08-25T03:35:05.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.128+0000 7f5a8fad4700 1 -- 172.21.15.2:0/54854091 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5a6c06a0e0 msgr2=0x7f5a6c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:05.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.128+0000 7f5a8fad4700 1 --2- 172.21.15.2:0/54854091 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5a6c06a0e0 0x7f5a6c06c5a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5a78013040 tx=0x7f5a78008070 comp rx=0 tx=0).stop 2023-08-25T03:35:05.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.128+0000 7f5a8fad4700 1 -- 172.21.15.2:0/54854091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a88159d10 msgr2=0x7f5a880ad680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:05.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.128+0000 7f5a8fad4700 1 --2- 172.21.15.2:0/54854091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a88159d10 0x7f5a880ad680 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5a8000cc30 tx=0x7f5a8000a1f0 comp rx=0 tx=0).stop 2023-08-25T03:35:05.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.128+0000 7f5a8fad4700 1 -- 172.21.15.2:0/54854091 shutdown_connections 2023-08-25T03:35:05.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.128+0000 7f5a8fad4700 1 --2- 172.21.15.2:0/54854091 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5a6c06a0e0 0x7f5a6c06c5a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:05.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.128+0000 7f5a8fad4700 1 --2- 172.21.15.2:0/54854091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5a88159d10 0x7f5a880ad680 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:05.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.128+0000 7f5a8fad4700 1 --2- 172.21.15.2:0/54854091 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5a880b4590 0x7f5a881567b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:05.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.128+0000 7f5a8fad4700 1 --2- 172.21.15.2:0/54854091 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a880ce050 0x7f5a880ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:05.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.128+0000 7f5a8fad4700 1 -- 172.21.15.2:0/54854091 >> 172.21.15.2:0/54854091 conn(0x7f5a880cf500 msgr2=0x7f5a8815e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:05.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.128+0000 7f5a8fad4700 1 -- 172.21.15.2:0/54854091 shutdown_connections 2023-08-25T03:35:05.131 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.128+0000 7f5a8fad4700 1 -- 172.21.15.2:0/54854091 wait complete. 2023-08-25T03:35:05.143 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-08-25T03:35:05.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.267+0000 7f97cdb53700 1 Processor -- start 2023-08-25T03:35:05.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.267+0000 7f97cdb53700 1 -- start start 2023-08-25T03:35:05.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.267+0000 7f97cdb53700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f97c80cdb70 0x7f97c806b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:05.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.267+0000 7f97cdb53700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f97c80ce000 0x7f97c806be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:05.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.267+0000 7f97cdb53700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c8161620 con 0x7f97c80b1fb0 2023-08-25T03:35:05.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.267+0000 7f97cdb53700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c806d450 con 0x7f97c80ce000 2023-08-25T03:35:05.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.267+0000 7f97cdb53700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c806d730 con 0x7f97c80cdb70 2023-08-25T03:35:05.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.268+0000 7f97c7fff700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f97c80ce000 0x7f97c806be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:05.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.268+0000 7f97c7fff700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f97c80ce000 0x7f97c806be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33038/0 (socket says 172.21.15.2:33038) 2023-08-25T03:35:05.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.268+0000 7f97c7fff700 1 -- 172.21.15.2:0/3577361174 learned_addr learned my addr 172.21.15.2:0/3577361174 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:05.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.268+0000 7f97c77fe700 1 --1- 172.21.15.2:0/3577361174 >> v1:172.21.15.2:6789/0 conn(0x7f97c80b1fb0 0x7f97c8160f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47932/0 (socket says 172.21.15.2:47932) 2023-08-25T03:35:05.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.268+0000 7f97c6ffd700 1 --2- 172.21.15.2:0/3577361174 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f97c80cdb70 0x7f97c806b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:05.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.268+0000 7f97c7fff700 1 -- 172.21.15.2:0/3577361174 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f97c80cdb70 msgr2=0x7f97c806b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:05.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.268+0000 7f97c7fff700 1 --2- 172.21.15.2:0/3577361174 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f97c80cdb70 0x7f97c806b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:05.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.268+0000 7f97c7fff700 1 -- 172.21.15.2:0/3577361174 >> v1:172.21.15.2:6789/0 conn(0x7f97c80b1fb0 legacy=0x7f97c8160f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:05.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.268+0000 7f97c7fff700 1 -- 172.21.15.2:0/3577361174 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c806da10 con 0x7f97c80ce000 2023-08-25T03:35:05.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.268+0000 7f97c67fc700 1 -- 172.21.15.2:0/3577361174 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 809975398 0 0) 0x7f97c8161620 con 0x7f97c80b1fb0 2023-08-25T03:35:05.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.268+0000 7f97c7fff700 1 --2- 172.21.15.2:0/3577361174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97c80ce000 0x7f97c806be00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f97c0009e50 tx=0x7f97c000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b8ffd94a69a8fc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:05.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.268+0000 7f97c67fc700 1 -- 172.21.15.2:0/3577361174 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c000e070 con 0x7f97c80ce000 2023-08-25T03:35:05.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.268+0000 7f97c67fc700 1 -- 172.21.15.2:0/3577361174 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c000b9c0 con 0x7f97c80ce000 2023-08-25T03:35:05.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.268+0000 7f97c67fc700 1 -- 172.21.15.2:0/3577361174 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c000bd70 con 0x7f97c80ce000 2023-08-25T03:35:05.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.269+0000 7f97cdb53700 1 -- 172.21.15.2:0/3577361174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97c80ce000 msgr2=0x7f97c806be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:05.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.269+0000 7f97cdb53700 1 --2- 172.21.15.2:0/3577361174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97c80ce000 0x7f97c806be00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f97c0009e50 tx=0x7f97c000b330 comp rx=0 tx=0).stop 2023-08-25T03:35:05.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.269+0000 7f97cdb53700 1 -- 172.21.15.2:0/3577361174 shutdown_connections 2023-08-25T03:35:05.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.269+0000 7f97cdb53700 1 --2- 172.21.15.2:0/3577361174 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97c80ce000 0x7f97c806be00 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:05.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.269+0000 7f97cdb53700 1 --2- 172.21.15.2:0/3577361174 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f97c80cdb70 0x7f97c806b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:05.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.269+0000 7f97cdb53700 1 -- 172.21.15.2:0/3577361174 >> 172.21.15.2:0/3577361174 conn(0x7f97c80cfea0 msgr2=0x7f97c80d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:05.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.269+0000 7f97cdb53700 1 -- 172.21.15.2:0/3577361174 shutdown_connections 2023-08-25T03:35:05.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.269+0000 7f97cdb53700 1 -- 172.21.15.2:0/3577361174 wait complete. 2023-08-25T03:35:05.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.270+0000 7f97cdb53700 1 Processor -- start 2023-08-25T03:35:05.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.271+0000 7f97cdb53700 1 -- start start 2023-08-25T03:35:05.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.272+0000 7f97cdb53700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f97c80b1fb0 0x7f97c80b4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:05.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.272+0000 7f97cdb53700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97c80cdb70 0x7f97c8157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:05.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.272+0000 7f97cdb53700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97c80ce000 0x7f97c81588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:05.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.272+0000 7f97cdb53700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c8157a80 con 0x7f97c80ce000 2023-08-25T03:35:05.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.272+0000 7f97cdb53700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c80c4680 con 0x7f97c80cdb70 2023-08-25T03:35:05.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.272+0000 7f97cdb53700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c80c3a70 con 0x7f97c80b1fb0 2023-08-25T03:35:05.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.272+0000 7f97c7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97c80ce000 0x7f97c81588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:05.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.272+0000 7f97c7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97c80ce000 0x7f97c81588f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33044/0 (socket says 172.21.15.2:33044) 2023-08-25T03:35:05.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.272+0000 7f97c7fff700 1 -- 172.21.15.2:0/4239009781 learned_addr learned my addr 172.21.15.2:0/4239009781 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:05.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.272+0000 7f97c6ffd700 1 --2- 172.21.15.2:0/4239009781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97c80cdb70 0x7f97c8157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:05.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.272+0000 7f97c77fe700 1 --2- 172.21.15.2:0/4239009781 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f97c80b1fb0 0x7f97c80b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:05.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.272+0000 7f97c7fff700 1 -- 172.21.15.2:0/4239009781 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f97c80b1fb0 msgr2=0x7f97c80b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:05.277 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.272+0000 7f97c7fff700 1 --2- 172.21.15.2:0/4239009781 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f97c80b1fb0 0x7f97c80b4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:05.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.272+0000 7f97c7fff700 1 -- 172.21.15.2:0/4239009781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97c80cdb70 msgr2=0x7f97c8157540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:05.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.272+0000 7f97c7fff700 1 --2- 172.21.15.2:0/4239009781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97c80cdb70 0x7f97c8157540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:05.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.272+0000 7f97c7fff700 1 -- 172.21.15.2:0/4239009781 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c0009b30 con 0x7f97c80ce000 2023-08-25T03:35:05.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.272+0000 7f97c6ffd700 1 --2- 172.21.15.2:0/4239009781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97c80cdb70 0x7f97c8157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:05.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.272+0000 7f97c7fff700 1 --2- 172.21.15.2:0/4239009781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97c80ce000 0x7f97c81588f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f97c000b730 tx=0x7f97c0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:05.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.272+0000 7f97affff700 1 -- 172.21.15.2:0/4239009781 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c0003ea0 con 0x7f97c80ce000 2023-08-25T03:35:05.278 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.272+0000 7f97affff700 1 -- 172.21.15.2:0/4239009781 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c00099b0 con 0x7f97c80ce000 2023-08-25T03:35:05.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.272+0000 7f97cdb53700 1 -- 172.21.15.2:0/4239009781 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c80c3d50 con 0x7f97c80ce000 2023-08-25T03:35:05.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.273+0000 7f97affff700 1 -- 172.21.15.2:0/4239009781 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c001b670 con 0x7f97c80ce000 2023-08-25T03:35:05.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.273+0000 7f97cdb53700 1 -- 172.21.15.2:0/4239009781 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c80c4310 con 0x7f97c80ce000 2023-08-25T03:35:05.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.274+0000 7f97cdb53700 1 -- 172.21.15.2:0/4239009781 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c8066f70 con 0x7f97c80ce000 2023-08-25T03:35:05.279 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.274+0000 7f97affff700 1 -- 172.21.15.2:0/4239009781 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f97c001b7d0 con 0x7f97c80ce000 2023-08-25T03:35:05.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.274+0000 7f97affff700 1 --2- 172.21.15.2:0/4239009781 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f97b406a0e0 0x7f97b406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:05.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.274+0000 7f97affff700 1 -- 172.21.15.2:0/4239009781 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f97c00898f0 con 0x7f97c80ce000 2023-08-25T03:35:05.280 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.274+0000 7f97c77fe700 1 --2- 172.21.15.2:0/4239009781 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f97b406a0e0 0x7f97b406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:05.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.275+0000 7f97c77fe700 1 --2- 172.21.15.2:0/4239009781 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f97b406a0e0 0x7f97b406c5a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f97b8000f60 tx=0x7f97b8007c90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:05.281 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.279+0000 7f97affff700 1 -- 172.21.15.2:0/4239009781 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f97c00553c0 con 0x7f97c80ce000 2023-08-25T03:35:05.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.481+0000 7f97cdb53700 1 -- 172.21.15.2:0/4239009781 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f97c80746d0 con 0x7f97c80ce000 2023-08-25T03:35:05.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.481+0000 7f97affff700 1 -- 172.21.15.2:0/4239009781 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v33) v1 ==== 84+0+126 (secure 0 0 0) 0x7f97c001f090 con 0x7f97c80ce000 2023-08-25T03:35:05.483 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:35:05.483 INFO:teuthology.orchestra.run.smithi002.stdout:[ 2023-08-25T03:35:05.483 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:05.484 INFO:teuthology.orchestra.run.smithi002.stdout: "poolnum": 1, 2023-08-25T03:35:05.484 INFO:teuthology.orchestra.run.smithi002.stdout: "poolname": ".mgr" 2023-08-25T03:35:05.484 INFO:teuthology.orchestra.run.smithi002.stdout: }, 2023-08-25T03:35:05.485 INFO:teuthology.orchestra.run.smithi002.stdout: { 2023-08-25T03:35:05.485 INFO:teuthology.orchestra.run.smithi002.stdout: "poolnum": 2, 2023-08-25T03:35:05.485 INFO:teuthology.orchestra.run.smithi002.stdout: "poolname": "rbd" 2023-08-25T03:35:05.485 INFO:teuthology.orchestra.run.smithi002.stdout: } 2023-08-25T03:35:05.486 INFO:teuthology.orchestra.run.smithi002.stdout:] 2023-08-25T03:35:05.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.483+0000 7f97cdb53700 1 -- 172.21.15.2:0/4239009781 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f97b406a0e0 msgr2=0x7f97b406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:05.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.483+0000 7f97cdb53700 1 --2- 172.21.15.2:0/4239009781 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f97b406a0e0 0x7f97b406c5a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f97b8000f60 tx=0x7f97b8007c90 comp rx=0 tx=0).stop 2023-08-25T03:35:05.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.484+0000 7f97cdb53700 1 -- 172.21.15.2:0/4239009781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97c80ce000 msgr2=0x7f97c81588f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:05.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.484+0000 7f97cdb53700 1 --2- 172.21.15.2:0/4239009781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97c80ce000 0x7f97c81588f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f97c000b730 tx=0x7f97c0004ad0 comp rx=0 tx=0).stop 2023-08-25T03:35:05.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.484+0000 7f97cdb53700 1 -- 172.21.15.2:0/4239009781 shutdown_connections 2023-08-25T03:35:05.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.484+0000 7f97cdb53700 1 --2- 172.21.15.2:0/4239009781 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f97b406a0e0 0x7f97b406c5a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:05.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.484+0000 7f97cdb53700 1 --2- 172.21.15.2:0/4239009781 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97c80ce000 0x7f97c81588f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:05.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.484+0000 7f97cdb53700 1 --2- 172.21.15.2:0/4239009781 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f97c80b1fb0 0x7f97c80b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:05.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.484+0000 7f97cdb53700 1 --2- 172.21.15.2:0/4239009781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97c80cdb70 0x7f97c8157540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:05.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.484+0000 7f97cdb53700 1 -- 172.21.15.2:0/4239009781 >> 172.21.15.2:0/4239009781 conn(0x7f97c80cfea0 msgr2=0x7f97c815e150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:05.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.484+0000 7f97cdb53700 1 -- 172.21.15.2:0/4239009781 shutdown_connections 2023-08-25T03:35:05.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.484+0000 7f97cdb53700 1 -- 172.21.15.2:0/4239009781 wait complete. 2023-08-25T03:35:05.497 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-08-25T03:35:05.497 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-08-25T03:35:05.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.625+0000 7fbb80806700 1 Processor -- start 2023-08-25T03:35:05.626 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.626+0000 7fbb80806700 1 -- start start 2023-08-25T03:35:05.627 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.626+0000 7fbb80806700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbb7c0cadf0 0x7fbb7c0ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:05.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.626+0000 7fbb80806700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbb7c0cb1d0 0x7fbb7c160f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:05.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.626+0000 7fbb80806700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb7c0cddc0 con 0x7fbb7c1614d0 2023-08-25T03:35:05.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.626+0000 7fbb80806700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb7c161a50 con 0x7fbb7c0cb1d0 2023-08-25T03:35:05.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.626+0000 7fbb80806700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c161d30 con 0x7fbb7c0cadf0 2023-08-25T03:35:05.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.627+0000 7fbb79d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbb7c0cb1d0 0x7fbb7c160f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:05.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.627+0000 7fbb7a59c700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbb7c0cadf0 0x7fbb7c0ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:05.628 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.627+0000 7fbb7ad9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7fbb7c1614d0 0x7fbb7c06a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.2:55974/0 (socket says 172.21.15.2:55974) 2023-08-25T03:35:05.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.627+0000 7fbb7a59c700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbb7c0cadf0 0x7fbb7c0ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.2:33292/0 (socket says 172.21.15.2:33292) 2023-08-25T03:35:05.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.627+0000 7fbb7ad9d700 1 -- 172.21.15.2:0/1891214496 learned_addr learned my addr 172.21.15.2:0/1891214496 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:05.629 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.627+0000 7fbb79d9b700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fbb7c0cb1d0 0x7fbb7c160f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.2:52570/0 (socket says 172.21.15.2:52570) 2023-08-25T03:35:05.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.627+0000 7fbb79d9b700 1 -- 172.21.15.2:0/1891214496 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbb7c0cadf0 msgr2=0x7fbb7c0ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:05.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.627+0000 7fbb79d9b700 1 --2- 172.21.15.2:0/1891214496 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbb7c0cadf0 0x7fbb7c0ce860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:05.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.627+0000 7fbb79d9b700 1 -- 172.21.15.2:0/1891214496 >> v1:172.21.15.31:6789/0 conn(0x7fbb7c1614d0 legacy=0x7fbb7c06a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:05.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.627+0000 7fbb79d9b700 1 -- 172.21.15.2:0/1891214496 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c0ce080 con 0x7fbb7c0cb1d0 2023-08-25T03:35:05.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.627+0000 7fbb7959a700 1 -- 172.21.15.2:0/1891214496 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 104795491 0 0) 0x7fbb7c0cddc0 con 0x7fbb7c1614d0 2023-08-25T03:35:05.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.627+0000 7fbb7a59c700 1 --2- 172.21.15.2:0/1891214496 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbb7c0cadf0 0x7fbb7c0ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:35:05.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.628+0000 7fbb79d9b700 1 --2- 172.21.15.2:0/1891214496 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb7c0cb1d0 0x7fbb7c160f90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009840 tx=0x7fbb6c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3358c3f143edf047 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:05.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.628+0000 7fbb7959a700 1 -- 172.21.15.2:0/1891214496 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c011830 con 0x7fbb7c0cb1d0 2023-08-25T03:35:05.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.628+0000 7fbb7959a700 1 -- 172.21.15.2:0/1891214496 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6c011990 con 0x7fbb7c0cb1d0 2023-08-25T03:35:05.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.628+0000 7fbb7959a700 1 -- 172.21.15.2:0/1891214496 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c011d20 con 0x7fbb7c0cb1d0 2023-08-25T03:35:05.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.628+0000 7fbb80806700 1 -- 172.21.15.2:0/1891214496 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb7c0cb1d0 msgr2=0x7fbb7c160f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:05.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.628+0000 7fbb80806700 1 --2- 172.21.15.2:0/1891214496 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb7c0cb1d0 0x7fbb7c160f90 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009840 tx=0x7fbb6c009b50 comp rx=0 tx=0).stop 2023-08-25T03:35:05.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.628+0000 7fbb80806700 1 -- 172.21.15.2:0/1891214496 shutdown_connections 2023-08-25T03:35:05.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.628+0000 7fbb80806700 1 --2- 172.21.15.2:0/1891214496 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbb7c0cadf0 0x7fbb7c0ce860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:05.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.628+0000 7fbb80806700 1 --2- 172.21.15.2:0/1891214496 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb7c0cb1d0 0x7fbb7c160f90 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:05.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.628+0000 7fbb80806700 1 -- 172.21.15.2:0/1891214496 >> 172.21.15.2:0/1891214496 conn(0x7fbb7c0b1990 msgr2=0x7fbb7c0b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:05.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.628+0000 7fbb80806700 1 -- 172.21.15.2:0/1891214496 shutdown_connections 2023-08-25T03:35:05.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.628+0000 7fbb80806700 1 -- 172.21.15.2:0/1891214496 wait complete. 2023-08-25T03:35:05.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.630+0000 7fbb80806700 1 Processor -- start 2023-08-25T03:35:05.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.630+0000 7fbb80806700 1 -- start start 2023-08-25T03:35:05.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.630+0000 7fbb80806700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbb7c0cadf0 0x7fbb7c0c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:05.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.630+0000 7fbb80806700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb7c0cb1d0 0x7fbb7c1f6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:05.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.630+0000 7fbb80806700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb7c1614d0 0x7fbb7c1f65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:05.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.630+0000 7fbb80806700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c0b4840 con 0x7fbb7c0cb1d0 2023-08-25T03:35:05.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.630+0000 7fbb80806700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c0b4b20 con 0x7fbb7c1614d0 2023-08-25T03:35:05.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.630+0000 7fbb80806700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c0b4e00 con 0x7fbb7c0cadf0 2023-08-25T03:35:05.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.631+0000 7fbb79d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb7c0cb1d0 0x7fbb7c1f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:05.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.631+0000 7fbb79d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb7c0cb1d0 0x7fbb7c1f6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33056/0 (socket says 172.21.15.2:33056) 2023-08-25T03:35:05.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.631+0000 7fbb79d9b700 1 -- 172.21.15.2:0/1995069505 learned_addr learned my addr 172.21.15.2:0/1995069505 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:05.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.631+0000 7fbb7a59c700 1 --2- 172.21.15.2:0/1995069505 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbb7c0cadf0 0x7fbb7c0c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:05.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.631+0000 7fbb7ad9d700 1 --2- 172.21.15.2:0/1995069505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb7c1614d0 0x7fbb7c1f65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:05.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.631+0000 7fbb79d9b700 1 -- 172.21.15.2:0/1995069505 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbb7c0cadf0 msgr2=0x7fbb7c0c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:05.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.631+0000 7fbb79d9b700 1 --2- 172.21.15.2:0/1995069505 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbb7c0cadf0 0x7fbb7c0c8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:05.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.631+0000 7fbb79d9b700 1 -- 172.21.15.2:0/1995069505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb7c1614d0 msgr2=0x7fbb7c1f65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:05.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.631+0000 7fbb79d9b700 1 --2- 172.21.15.2:0/1995069505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb7c1614d0 0x7fbb7c1f65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:05.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.631+0000 7fbb79d9b700 1 -- 172.21.15.2:0/1995069505 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb700094a0 con 0x7fbb7c0cb1d0 2023-08-25T03:35:05.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.631+0000 7fbb79d9b700 1 --2- 172.21.15.2:0/1995069505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb7c0cb1d0 0x7fbb7c1f6080 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009fd0 tx=0x7fbb6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:05.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.631+0000 7fbb6affd700 1 -- 172.21.15.2:0/1995069505 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c0036a0 con 0x7fbb7c0cb1d0 2023-08-25T03:35:05.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.631+0000 7fbb6affd700 1 -- 172.21.15.2:0/1995069505 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb6c01f070 con 0x7fbb7c0cb1d0 2023-08-25T03:35:05.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.631+0000 7fbb6affd700 1 -- 172.21.15.2:0/1995069505 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb6c003a60 con 0x7fbb7c0cb1d0 2023-08-25T03:35:05.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.631+0000 7fbb80806700 1 -- 172.21.15.2:0/1995069505 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb6c0094a0 con 0x7fbb7c0cb1d0 2023-08-25T03:35:05.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.631+0000 7fbb80806700 1 -- 172.21.15.2:0/1995069505 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7c0b53c0 con 0x7fbb7c0cb1d0 2023-08-25T03:35:05.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.632+0000 7fbb80806700 1 -- 172.21.15.2:0/1995069505 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb7c066f70 con 0x7fbb7c0cb1d0 2023-08-25T03:35:05.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.633+0000 7fbb6affd700 1 -- 172.21.15.2:0/1995069505 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fbb6c003bc0 con 0x7fbb7c0cb1d0 2023-08-25T03:35:05.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.633+0000 7fbb6affd700 1 --2- 172.21.15.2:0/1995069505 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fbb5c06a0e0 0x7fbb5c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:05.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.633+0000 7fbb6affd700 1 -- 172.21.15.2:0/1995069505 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fbb6c021070 con 0x7fbb7c0cb1d0 2023-08-25T03:35:05.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.633+0000 7fbb7a59c700 1 --2- 172.21.15.2:0/1995069505 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fbb5c06a0e0 0x7fbb5c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:05.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.634+0000 7fbb7a59c700 1 --2- 172.21.15.2:0/1995069505 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fbb5c06a0e0 0x7fbb5c06c5a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbb70009d70 tx=0x7fbb70009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:05.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.638+0000 7fbb6affd700 1 -- 172.21.15.2:0/1995069505 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fbb6c055450 con 0x7fbb7c0cb1d0 2023-08-25T03:35:05.840 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.840+0000 7fbb80806700 1 -- 172.21.15.2:0/1995069505 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fbb7c0746d0 con 0x7fbb7c0cb1d0 2023-08-25T03:35:05.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.902+0000 7fbb6affd700 1 -- 172.21.15.2:0/1995069505 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v34) v1 ==== 138+0+0 (secure 0 0 0) 0x7fbb6c02b0b0 con 0x7fbb7c0cb1d0 2023-08-25T03:35:05.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.905+0000 7fbb80806700 1 -- 172.21.15.2:0/1995069505 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fbb5c06a0e0 msgr2=0x7fbb5c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:05.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.905+0000 7fbb80806700 1 --2- 172.21.15.2:0/1995069505 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fbb5c06a0e0 0x7fbb5c06c5a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbb70009d70 tx=0x7fbb70009210 comp rx=0 tx=0).stop 2023-08-25T03:35:05.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.905+0000 7fbb80806700 1 -- 172.21.15.2:0/1995069505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb7c0cb1d0 msgr2=0x7fbb7c1f6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:05.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.905+0000 7fbb80806700 1 --2- 172.21.15.2:0/1995069505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb7c0cb1d0 0x7fbb7c1f6080 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c009fd0 tx=0x7fbb6c004750 comp rx=0 tx=0).stop 2023-08-25T03:35:05.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.905+0000 7fbb80806700 1 -- 172.21.15.2:0/1995069505 shutdown_connections 2023-08-25T03:35:05.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.905+0000 7fbb80806700 1 --2- 172.21.15.2:0/1995069505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbb7c1614d0 0x7fbb7c1f65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:05.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.905+0000 7fbb80806700 1 --2- 172.21.15.2:0/1995069505 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbb7c0cadf0 0x7fbb7c0c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:05.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.905+0000 7fbb80806700 1 --2- 172.21.15.2:0/1995069505 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fbb5c06a0e0 0x7fbb5c06c5a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:05.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.905+0000 7fbb80806700 1 --2- 172.21.15.2:0/1995069505 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fbb7c0cb1d0 0x7fbb7c1f6080 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:05.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.905+0000 7fbb80806700 1 -- 172.21.15.2:0/1995069505 >> 172.21.15.2:0/1995069505 conn(0x7fbb7c0b1990 msgr2=0x7fbb7c0ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:05.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.905+0000 7fbb80806700 1 -- 172.21.15.2:0/1995069505 shutdown_connections 2023-08-25T03:35:05.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:05.906+0000 7fbb80806700 1 -- 172.21.15.2:0/1995069505 wait complete. 2023-08-25T03:35:05.910 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_metadata' created 2023-08-25T03:35:05.921 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-08-25T03:35:06.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.053+0000 7fa0ddb2e700 1 Processor -- start 2023-08-25T03:35:06.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.053+0000 7fa0ddb2e700 1 -- start start 2023-08-25T03:35:06.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.054+0000 7fa0ddb2e700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa0d806e7c0 0x7fa0d80d0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:06.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.054+0000 7fa0ddb2e700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0d806ec50 con 0x7fa0d806cea0 2023-08-25T03:35:06.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.054+0000 7fa0ddb2e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0d8161120 con 0x7fa0d8161c00 2023-08-25T03:35:06.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.054+0000 7fa0ddb2e700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d8161400 con 0x7fa0d806e7c0 2023-08-25T03:35:06.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.054+0000 7fa0d77fe700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fa0d806cea0 0x7fa0d806d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:47960/0 (socket says 172.21.15.2:47960) 2023-08-25T03:35:06.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.054+0000 7fa0d77fe700 1 -- 172.21.15.2:0/1099708896 learned_addr learned my addr 172.21.15.2:0/1099708896 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:06.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.054+0000 7fa0d6ffd700 1 --2- 172.21.15.2:0/1099708896 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa0d806e7c0 0x7fa0d80d0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:06.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.055+0000 7fa0d67fc700 1 -- 172.21.15.2:0/1099708896 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2814967237 0 0) 0x7fa0d806ec50 con 0x7fa0d806cea0 2023-08-25T03:35:06.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.055+0000 7fa0d67fc700 1 -- 172.21.15.2:0/1099708896 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0c00035b0 con 0x7fa0d806cea0 2023-08-25T03:35:06.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.055+0000 7fa0d67fc700 1 -- 172.21.15.2:0/1099708896 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3520656378 0 0) 0x7fa0c00035b0 con 0x7fa0d806cea0 2023-08-25T03:35:06.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.055+0000 7fa0d67fc700 1 -- 172.21.15.2:0/1099708896 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa0d806e7c0 msgr2=0x7fa0d80d0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:06.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.055+0000 7fa0d67fc700 1 --2- 172.21.15.2:0/1099708896 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa0d806e7c0 0x7fa0d80d0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:06.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.055+0000 7fa0d67fc700 1 -- 172.21.15.2:0/1099708896 >> v1:172.21.15.31:6789/0 conn(0x7fa0d8161c00 legacy=0x7fa0d806a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:06.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.055+0000 7fa0d67fc700 1 -- 172.21.15.2:0/1099708896 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d81616e0 con 0x7fa0d806cea0 2023-08-25T03:35:06.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.055+0000 7fa0d67fc700 1 -- 172.21.15.2:0/1099708896 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7fa0cc001670 con 0x7fa0d806cea0 2023-08-25T03:35:06.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.055+0000 7fa0d67fc700 1 -- 172.21.15.2:0/1099708896 >> v1:172.21.15.2:6789/0 conn(0x7fa0d806cea0 legacy=0x7fa0d806d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:06.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.055+0000 7fa0d67fc700 1 --2- 172.21.15.2:0/1099708896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0c0003a30 0x7fa0c0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:06.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.055+0000 7fa0d67fc700 1 --2- 172.21.15.2:0/1099708896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c00043e0 0x7fa0c00048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:06.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.055+0000 7fa0d67fc700 1 --2- 172.21.15.2:0/1099708896 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa0c0005150 0x7fa0c000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:06.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.055+0000 7fa0d67fc700 1 -- 172.21.15.2:0/1099708896 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d8161400 con 0x7fa0c0003a30 2023-08-25T03:35:06.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.055+0000 7fa0d67fc700 1 -- 172.21.15.2:0/1099708896 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c000bf50 con 0x7fa0c00043e0 2023-08-25T03:35:06.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.055+0000 7fa0d67fc700 1 -- 172.21.15.2:0/1099708896 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c000c090 con 0x7fa0c0005150 2023-08-25T03:35:06.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.055+0000 7fa0d77fe700 1 --2- 172.21.15.2:0/1099708896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0c0003a30 0x7fa0c0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:06.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.055+0000 7fa0d7fff700 1 --2- 172.21.15.2:0/1099708896 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa0c0005150 0x7fa0c000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:06.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.055+0000 7fa0d6ffd700 1 --2- 172.21.15.2:0/1099708896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c00043e0 0x7fa0c00048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:06.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.055+0000 7fa0d77fe700 1 -- 172.21.15.2:0/1099708896 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa0c0005150 msgr2=0x7fa0c000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:06.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.055+0000 7fa0d77fe700 1 --2- 172.21.15.2:0/1099708896 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa0c0005150 0x7fa0c000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:06.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.055+0000 7fa0d77fe700 1 -- 172.21.15.2:0/1099708896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c00043e0 msgr2=0x7fa0c00048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:06.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.055+0000 7fa0d77fe700 1 --2- 172.21.15.2:0/1099708896 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0c00043e0 0x7fa0c00048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:06.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.055+0000 7fa0d77fe700 1 -- 172.21.15.2:0/1099708896 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c00035b0 con 0x7fa0c0003a30 2023-08-25T03:35:06.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.056+0000 7fa0d77fe700 1 --2- 172.21.15.2:0/1099708896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0c0003a30 0x7fa0c0003e10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc00c240 tx=0x7fa0c0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482f42850cb49a16 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:06.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.056+0000 7fa0d67fc700 1 -- 172.21.15.2:0/1099708896 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc00cd80 con 0x7fa0c0003a30 2023-08-25T03:35:06.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.056+0000 7fa0d67fc700 1 -- 172.21.15.2:0/1099708896 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0cc005960 con 0x7fa0c0003a30 2023-08-25T03:35:06.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.056+0000 7fa0d67fc700 1 -- 172.21.15.2:0/1099708896 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc005d10 con 0x7fa0c0003a30 2023-08-25T03:35:06.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.056+0000 7fa0ddb2e700 1 -- 172.21.15.2:0/1099708896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0c0003a30 msgr2=0x7fa0c0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:06.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.056+0000 7fa0ddb2e700 1 --2- 172.21.15.2:0/1099708896 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0c0003a30 0x7fa0c0003e10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc00c240 tx=0x7fa0c0005880 comp rx=0 tx=0).stop 2023-08-25T03:35:06.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.056+0000 7fa0d7fff700 1 -- 172.21.15.2:0/1099708896 reap_dead start 2023-08-25T03:35:06.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.056+0000 7fa0ddb2e700 1 -- 172.21.15.2:0/1099708896 shutdown_connections 2023-08-25T03:35:06.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.056+0000 7fa0ddb2e700 1 -- 172.21.15.2:0/1099708896 >> 172.21.15.2:0/1099708896 conn(0x7fa0d80d0000 msgr2=0x7fa0d80d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:06.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.056+0000 7fa0ddb2e700 1 -- 172.21.15.2:0/1099708896 shutdown_connections 2023-08-25T03:35:06.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.056+0000 7fa0ddb2e700 1 -- 172.21.15.2:0/1099708896 wait complete. 2023-08-25T03:35:06.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.057+0000 7fa0ddb2e700 1 Processor -- start 2023-08-25T03:35:06.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.058+0000 7fa0ddb2e700 1 -- start start 2023-08-25T03:35:06.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.058+0000 7fa0ddb2e700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa0c0003a30 0x7fa0d81602c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:06.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.058+0000 7fa0ddb2e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0d81516d0 0x7fa0d80c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:06.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.058+0000 7fa0ddb2e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0d815fc10 0x7fa0d80ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:06.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.058+0000 7fa0ddb2e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d8160800 con 0x7fa0d81516d0 2023-08-25T03:35:06.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.058+0000 7fa0ddb2e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d80c5140 con 0x7fa0d815fc10 2023-08-25T03:35:06.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.058+0000 7fa0ddb2e700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d815eb70 con 0x7fa0c0003a30 2023-08-25T03:35:06.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.058+0000 7fa0d6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0d81516d0 0x7fa0d80c4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:06.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.059+0000 7fa0d6ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0d81516d0 0x7fa0d80c4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33076/0 (socket says 172.21.15.2:33076) 2023-08-25T03:35:06.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.059+0000 7fa0d6ffd700 1 -- 172.21.15.2:0/845221720 learned_addr learned my addr 172.21.15.2:0/845221720 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:06.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.059+0000 7fa0d6ffd700 1 -- 172.21.15.2:0/845221720 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa0c0003a30 msgr2=0x7fa0d81602c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:06.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.059+0000 7fa0d77fe700 1 --2- 172.21.15.2:0/845221720 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa0c0003a30 0x7fa0d81602c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:06.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.059+0000 7fa0d7fff700 1 --2- 172.21.15.2:0/845221720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0d815fc10 0x7fa0d80ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:06.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.059+0000 7fa0d6ffd700 1 --2- 172.21.15.2:0/845221720 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa0c0003a30 0x7fa0d81602c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:06.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.059+0000 7fa0d6ffd700 1 -- 172.21.15.2:0/845221720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0d815fc10 msgr2=0x7fa0d80ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:06.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.059+0000 7fa0d6ffd700 1 --2- 172.21.15.2:0/845221720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0d815fc10 0x7fa0d80ace10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:06.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.059+0000 7fa0d6ffd700 1 -- 172.21.15.2:0/845221720 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d815ee50 con 0x7fa0d81516d0 2023-08-25T03:35:06.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.059+0000 7fa0d6ffd700 1 --2- 172.21.15.2:0/845221720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0d81516d0 0x7fa0d80c4c00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8011aa0 tx=0x7fa0c8011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:06.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.059+0000 7fa0bffff700 1 -- 172.21.15.2:0/845221720 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c800f980 con 0x7fa0d81516d0 2023-08-25T03:35:06.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.059+0000 7fa0bffff700 1 -- 172.21.15.2:0/845221720 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c800fae0 con 0x7fa0d81516d0 2023-08-25T03:35:06.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.059+0000 7fa0ddb2e700 1 -- 172.21.15.2:0/845221720 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0d815f130 con 0x7fa0d81516d0 2023-08-25T03:35:06.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.059+0000 7fa0bffff700 1 -- 172.21.15.2:0/845221720 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c800fe90 con 0x7fa0d81516d0 2023-08-25T03:35:06.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.059+0000 7fa0ddb2e700 1 -- 172.21.15.2:0/845221720 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0d815f6f0 con 0x7fa0d81516d0 2023-08-25T03:35:06.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.060+0000 7fa0ddb2e700 1 -- 172.21.15.2:0/845221720 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0d8066f70 con 0x7fa0d81516d0 2023-08-25T03:35:06.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.061+0000 7fa0bffff700 1 -- 172.21.15.2:0/845221720 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa0c8014440 con 0x7fa0d81516d0 2023-08-25T03:35:06.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.061+0000 7fa0bffff700 1 --2- 172.21.15.2:0/845221720 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa0c406a0e0 0x7fa0c406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:06.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.061+0000 7fa0bffff700 1 -- 172.21.15.2:0/845221720 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5580+0+0 (secure 0 0 0) 0x7fa0c80889c0 con 0x7fa0d81516d0 2023-08-25T03:35:06.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.061+0000 7fa0d77fe700 1 --2- 172.21.15.2:0/845221720 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa0c406a0e0 0x7fa0c406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:06.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.062+0000 7fa0d77fe700 1 --2- 172.21.15.2:0/845221720 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa0c406a0e0 0x7fa0c406c5a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc015040 tx=0x7fa0cc00a000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:06.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.066+0000 7fa0bffff700 1 -- 172.21.15.2:0/845221720 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa0c80543e0 con 0x7fa0d81516d0 2023-08-25T03:35:06.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.266+0000 7fa0ddb2e700 1 -- 172.21.15.2:0/845221720 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa0d809e190 con 0x7fa0d81516d0 2023-08-25T03:35:06.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.905+0000 7fa0bffff700 1 -- 172.21.15.2:0/845221720 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v35) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa0c8057a00 con 0x7fa0d81516d0 2023-08-25T03:35:06.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.917+0000 7fa0ddb2e700 1 -- 172.21.15.2:0/845221720 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa0c406a0e0 msgr2=0x7fa0c406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:06.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.917+0000 7fa0ddb2e700 1 --2- 172.21.15.2:0/845221720 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa0c406a0e0 0x7fa0c406c5a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc015040 tx=0x7fa0cc00a000 comp rx=0 tx=0).stop 2023-08-25T03:35:06.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.917+0000 7fa0ddb2e700 1 -- 172.21.15.2:0/845221720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0d81516d0 msgr2=0x7fa0d80c4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:06.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.917+0000 7fa0ddb2e700 1 --2- 172.21.15.2:0/845221720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0d81516d0 0x7fa0d80c4c00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8011aa0 tx=0x7fa0c8011e60 comp rx=0 tx=0).stop 2023-08-25T03:35:06.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.917+0000 7fa0ddb2e700 1 -- 172.21.15.2:0/845221720 shutdown_connections 2023-08-25T03:35:06.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.917+0000 7fa0ddb2e700 1 --2- 172.21.15.2:0/845221720 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa0d815fc10 0x7fa0d80ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:06.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.917+0000 7fa0ddb2e700 1 --2- 172.21.15.2:0/845221720 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa0c0003a30 0x7fa0d81602c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:06.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.917+0000 7fa0ddb2e700 1 --2- 172.21.15.2:0/845221720 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa0c406a0e0 0x7fa0c406c5a0 secure :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc015040 tx=0x7fa0cc00a000 comp rx=0 tx=0).stop 2023-08-25T03:35:06.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.917+0000 7fa0ddb2e700 1 --2- 172.21.15.2:0/845221720 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa0d81516d0 0x7fa0d80c4c00 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:06.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.917+0000 7fa0ddb2e700 1 -- 172.21.15.2:0/845221720 >> 172.21.15.2:0/845221720 conn(0x7fa0d80d0000 msgr2=0x7fa0d815cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:06.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.919+0000 7fa0ddb2e700 1 -- 172.21.15.2:0/845221720 shutdown_connections 2023-08-25T03:35:06.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:06.920+0000 7fa0ddb2e700 1 -- 172.21.15.2:0/845221720 wait complete. 2023-08-25T03:35:06.926 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_data' created 2023-08-25T03:35:06.941 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-08-25T03:35:07.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.074+0000 7f6d07c18700 1 Processor -- start 2023-08-25T03:35:07.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.075+0000 7f6d07c18700 1 -- start start 2023-08-25T03:35:07.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.075+0000 7f6d07c18700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f6d001607b0 0x7f6d000cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:07.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.075+0000 7f6d07c18700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6d000d1510 0x7f6d000ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:07.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.075+0000 7f6d07c18700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6d000d2b20 0x7f6d0006c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:07.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.075+0000 7f6d07c18700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d00160b90 con 0x7f6d001607b0 2023-08-25T03:35:07.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.075+0000 7f6d07c18700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d000d1a80 con 0x7f6d000d2b20 2023-08-25T03:35:07.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.075+0000 7f6d07c18700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d000d1d60 con 0x7f6d000d1510 2023-08-25T03:35:07.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.076+0000 7f6d059b4700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f6d001607b0 0x7f6d000cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:07.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.076+0000 7f6d059b4700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f6d001607b0 0x7f6d000cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33082/0 (socket says 172.21.15.2:33082) 2023-08-25T03:35:07.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.076+0000 7f6d059b4700 1 -- 172.21.15.2:0/836144375 learned_addr learned my addr 172.21.15.2:0/836144375 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:07.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.076+0000 7f6d061b5700 1 --2- 172.21.15.2:0/836144375 >> v2:172.21.15.31:3300/0 conn(0x7f6d000d2b20 0x7f6d0006c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:07.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.076+0000 7f6d051b3700 1 --2- 172.21.15.2:0/836144375 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6d000d1510 0x7f6d000ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:07.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.076+0000 7f6d059b4700 1 -- 172.21.15.2:0/836144375 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6d000d1510 msgr2=0x7f6d000ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:07.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.076+0000 7f6d059b4700 1 --2- 172.21.15.2:0/836144375 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6d000d1510 0x7f6d000ce300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:07.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.076+0000 7f6d059b4700 1 -- 172.21.15.2:0/836144375 >> v2:172.21.15.31:3300/0 conn(0x7f6d000d2b20 msgr2=0x7f6d0006c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:07.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.076+0000 7f6d059b4700 1 --2- 172.21.15.2:0/836144375 >> v2:172.21.15.31:3300/0 conn(0x7f6d000d2b20 0x7f6d0006c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:07.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.076+0000 7f6d059b4700 1 -- 172.21.15.2:0/836144375 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d000d2040 con 0x7f6d001607b0 2023-08-25T03:35:07.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.076+0000 7f6d061b5700 1 --2- 172.21.15.2:0/836144375 >> v2:172.21.15.31:3300/0 conn(0x7f6d000d2b20 0x7f6d0006c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:07.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.076+0000 7f6d059b4700 1 --2- 172.21.15.2:0/836144375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d001607b0 0x7f6d000cddc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6d000b9240 tx=0x7f6cfc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5587460171ee5f43 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:07.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.076+0000 7f6d049b2700 1 -- 172.21.15.2:0/836144375 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc00b870 con 0x7f6d001607b0 2023-08-25T03:35:07.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.076+0000 7f6d049b2700 1 -- 172.21.15.2:0/836144375 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cfc00b9d0 con 0x7f6d001607b0 2023-08-25T03:35:07.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.076+0000 7f6d049b2700 1 -- 172.21.15.2:0/836144375 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc00bd60 con 0x7f6d001607b0 2023-08-25T03:35:07.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.077+0000 7f6d07c18700 1 -- 172.21.15.2:0/836144375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d001607b0 msgr2=0x7f6d000cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:07.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.077+0000 7f6d07c18700 1 --2- 172.21.15.2:0/836144375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d001607b0 0x7f6d000cddc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f6d000b9240 tx=0x7f6cfc009bd0 comp rx=0 tx=0).stop 2023-08-25T03:35:07.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.077+0000 7f6d07c18700 1 -- 172.21.15.2:0/836144375 shutdown_connections 2023-08-25T03:35:07.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.077+0000 7f6d07c18700 1 --2- 172.21.15.2:0/836144375 >> v2:172.21.15.31:3300/0 conn(0x7f6d000d2b20 0x7f6d0006c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:07.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.077+0000 7f6d07c18700 1 --2- 172.21.15.2:0/836144375 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d001607b0 0x7f6d000cddc0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:07.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.077+0000 7f6d07c18700 1 --2- 172.21.15.2:0/836144375 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6d000d1510 0x7f6d000ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:07.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.077+0000 7f6d07c18700 1 -- 172.21.15.2:0/836144375 >> 172.21.15.2:0/836144375 conn(0x7f6d000cfb70 msgr2=0x7f6d000cff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:07.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.077+0000 7f6d07c18700 1 -- 172.21.15.2:0/836144375 shutdown_connections 2023-08-25T03:35:07.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.077+0000 7f6d07c18700 1 -- 172.21.15.2:0/836144375 wait complete. 2023-08-25T03:35:07.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.078+0000 7f6d07c18700 1 Processor -- start 2023-08-25T03:35:07.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.078+0000 7f6d07c18700 1 -- start start 2023-08-25T03:35:07.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.079+0000 7f6d07c18700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d000d1510 0x7f6d0015f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:07.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.079+0000 7f6d07c18700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d000d2b20 0x7f6d000cab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:07.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.079+0000 7f6d07c18700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6d001607b0 0x7f6d00156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:07.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.079+0000 7f6d07c18700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0015fcd0 con 0x7f6d000d1510 2023-08-25T03:35:07.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.079+0000 7f6d07c18700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0015de60 con 0x7f6d000d2b20 2023-08-25T03:35:07.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.079+0000 7f6d07c18700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0015e140 con 0x7f6d001607b0 2023-08-25T03:35:07.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.079+0000 7f6d059b4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d000d1510 0x7f6d0015f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:07.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.079+0000 7f6d059b4700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d000d1510 0x7f6d0015f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33088/0 (socket says 172.21.15.2:33088) 2023-08-25T03:35:07.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.079+0000 7f6d059b4700 1 -- 172.21.15.2:0/156748672 learned_addr learned my addr 172.21.15.2:0/156748672 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:07.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.079+0000 7f6d061b5700 1 --2- 172.21.15.2:0/156748672 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6d001607b0 0x7f6d00156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:07.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.079+0000 7f6d051b3700 1 --2- 172.21.15.2:0/156748672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d000d2b20 0x7f6d000cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:07.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.079+0000 7f6d059b4700 1 -- 172.21.15.2:0/156748672 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6d001607b0 msgr2=0x7f6d00156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:07.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.079+0000 7f6d059b4700 1 --2- 172.21.15.2:0/156748672 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6d001607b0 0x7f6d00156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:07.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.079+0000 7f6d059b4700 1 -- 172.21.15.2:0/156748672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d000d2b20 msgr2=0x7f6d000cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:07.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.079+0000 7f6d059b4700 1 --2- 172.21.15.2:0/156748672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d000d2b20 0x7f6d000cab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:07.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.079+0000 7f6d059b4700 1 -- 172.21.15.2:0/156748672 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cfc009550 con 0x7f6d000d1510 2023-08-25T03:35:07.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.080+0000 7f6d059b4700 1 --2- 172.21.15.2:0/156748672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d000d1510 0x7f6d0015f790 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc000c00 tx=0x7f6cfc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:07.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.080+0000 7f6cf67fc700 1 -- 172.21.15.2:0/156748672 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc004310 con 0x7f6d000d1510 2023-08-25T03:35:07.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.080+0000 7f6cf67fc700 1 -- 172.21.15.2:0/156748672 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cfc01e070 con 0x7f6d000d1510 2023-08-25T03:35:07.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.080+0000 7f6d07c18700 1 -- 172.21.15.2:0/156748672 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d0015e420 con 0x7f6d000d1510 2023-08-25T03:35:07.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.080+0000 7f6cf67fc700 1 -- 172.21.15.2:0/156748672 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc0036a0 con 0x7f6d000d1510 2023-08-25T03:35:07.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.080+0000 7f6d07c18700 1 -- 172.21.15.2:0/156748672 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d0015f1f0 con 0x7f6d000d1510 2023-08-25T03:35:07.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.081+0000 7f6d07c18700 1 -- 172.21.15.2:0/156748672 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d0015e940 con 0x7f6d000d1510 2023-08-25T03:35:07.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.082+0000 7f6cf67fc700 1 -- 172.21.15.2:0/156748672 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6cfc020030 con 0x7f6d000d1510 2023-08-25T03:35:07.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.082+0000 7f6cf67fc700 1 --2- 172.21.15.2:0/156748672 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6ce406a0e0 0x7f6ce406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:07.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.082+0000 7f6cf67fc700 1 -- 172.21.15.2:0/156748672 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f6cfc0893f0 con 0x7f6d000d1510 2023-08-25T03:35:07.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.082+0000 7f6d051b3700 1 --2- 172.21.15.2:0/156748672 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6ce406a0e0 0x7f6ce406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:07.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.083+0000 7f6d051b3700 1 --2- 172.21.15.2:0/156748672 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6ce406a0e0 0x7f6ce406c5a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6d0006aa90 tx=0x7f6cf0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:07.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.087+0000 7f6cf67fc700 1 -- 172.21.15.2:0/156748672 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6d0015e940 con 0x7f6d000d1510 2023-08-25T03:35:07.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.332+0000 7f6d07c18700 1 -- 172.21.15.2:0/156748672 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f6d0015e940 con 0x7f6d000d1510 2023-08-25T03:35:07.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.915+0000 7f6cf67fc700 1 -- 172.21.15.2:0/156748672 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f6d0015e940 con 0x7f6d000d1510 2023-08-25T03:35:07.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.918+0000 7f6d07c18700 1 -- 172.21.15.2:0/156748672 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6ce406a0e0 msgr2=0x7f6ce406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:07.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.918+0000 7f6d07c18700 1 --2- 172.21.15.2:0/156748672 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6ce406a0e0 0x7f6ce406c5a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f6d0006aa90 tx=0x7f6cf0008040 comp rx=0 tx=0).stop 2023-08-25T03:35:07.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.918+0000 7f6d07c18700 1 -- 172.21.15.2:0/156748672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d000d1510 msgr2=0x7f6d0015f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:07.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.918+0000 7f6d07c18700 1 --2- 172.21.15.2:0/156748672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d000d1510 0x7f6d0015f790 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc000c00 tx=0x7f6cfc004750 comp rx=0 tx=0).stop 2023-08-25T03:35:07.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.919+0000 7f6d07c18700 1 -- 172.21.15.2:0/156748672 shutdown_connections 2023-08-25T03:35:07.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.919+0000 7f6d07c18700 1 --2- 172.21.15.2:0/156748672 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6ce406a0e0 0x7f6ce406c5a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:07.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.919+0000 7f6d07c18700 1 --2- 172.21.15.2:0/156748672 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6d000d1510 0x7f6d0015f790 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:07.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.919+0000 7f6d07c18700 1 --2- 172.21.15.2:0/156748672 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6d001607b0 0x7f6d00156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:07.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.919+0000 7f6d07c18700 1 --2- 172.21.15.2:0/156748672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6d000d2b20 0x7f6d000cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:07.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.919+0000 7f6d07c18700 1 -- 172.21.15.2:0/156748672 >> 172.21.15.2:0/156748672 conn(0x7f6d000cfb70 msgr2=0x7f6d000c8fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:07.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.919+0000 7f6d07c18700 1 -- 172.21.15.2:0/156748672 shutdown_connections 2023-08-25T03:35:07.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:07.921+0000 7f6d07c18700 1 -- 172.21.15.2:0/156748672 wait complete. 2023-08-25T03:35:07.924 INFO:teuthology.orchestra.run.smithi002.stderr:new fs with metadata pool 5 and data pool 6 2023-08-25T03:35:07.938 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:35:08.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.068+0000 7fe9a23e3700 1 Processor -- start 2023-08-25T03:35:08.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.068+0000 7fe9a23e3700 1 -- start start 2023-08-25T03:35:08.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.069+0000 7fe9a23e3700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe99c0cdb70 0x7fe99c06b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:08.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.069+0000 7fe9a23e3700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe99c0ce000 0x7fe99c06be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:08.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.069+0000 7fe9a23e3700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe99c161620 con 0x7fe99c0b1fb0 2023-08-25T03:35:08.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.069+0000 7fe9a23e3700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe99c06d450 con 0x7fe99c0cdb70 2023-08-25T03:35:08.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.069+0000 7fe9a23e3700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c06d730 con 0x7fe99c0ce000 2023-08-25T03:35:08.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.069+0000 7fe99b7fe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe99c0cdb70 0x7fe99c06b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:08.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.069+0000 7fe99b7fe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe99c0cdb70 0x7fe99c06b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33098/0 (socket says 172.21.15.2:33098) 2023-08-25T03:35:08.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.069+0000 7fe99b7fe700 1 -- 172.21.15.2:0/3398094512 learned_addr learned my addr 172.21.15.2:0/3398094512 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:08.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.069+0000 7fe9a0980700 1 --2- 172.21.15.2:0/3398094512 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe99c0ce000 0x7fe99c06be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:08.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.070+0000 7fe99b7fe700 1 -- 172.21.15.2:0/3398094512 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe99c0ce000 msgr2=0x7fe99c06be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:08.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.070+0000 7fe99b7fe700 1 --2- 172.21.15.2:0/3398094512 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe99c0ce000 0x7fe99c06be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:08.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.070+0000 7fe99b7fe700 1 -- 172.21.15.2:0/3398094512 >> v1:172.21.15.31:6789/0 conn(0x7fe99c0b1fb0 legacy=0x7fe99c160f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:08.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.070+0000 7fe99b7fe700 1 -- 172.21.15.2:0/3398094512 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe99c06da10 con 0x7fe99c0cdb70 2023-08-25T03:35:08.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.070+0000 7fe99b7fe700 1 --2- 172.21.15.2:0/3398094512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe99c0cdb70 0x7fe99c06b8c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe9840098f0 tx=0x7fe984009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41dbf82d19960e29 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:08.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.070+0000 7fe99affd700 1 -- 172.21.15.2:0/3398094512 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98400b8a0 con 0x7fe99c0cdb70 2023-08-25T03:35:08.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.070+0000 7fe99affd700 1 -- 172.21.15.2:0/3398094512 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98400ba00 con 0x7fe99c0cdb70 2023-08-25T03:35:08.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.070+0000 7fe99affd700 1 -- 172.21.15.2:0/3398094512 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98400bd90 con 0x7fe99c0cdb70 2023-08-25T03:35:08.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.070+0000 7fe9a23e3700 1 -- 172.21.15.2:0/3398094512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe99c0cdb70 msgr2=0x7fe99c06b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:08.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.070+0000 7fe9a23e3700 1 --2- 172.21.15.2:0/3398094512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe99c0cdb70 0x7fe99c06b8c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe9840098f0 tx=0x7fe984009c00 comp rx=0 tx=0).stop 2023-08-25T03:35:08.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.070+0000 7fe9a23e3700 1 -- 172.21.15.2:0/3398094512 shutdown_connections 2023-08-25T03:35:08.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.070+0000 7fe9a23e3700 1 --2- 172.21.15.2:0/3398094512 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe99c0ce000 0x7fe99c06be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:08.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.070+0000 7fe9a23e3700 1 --2- 172.21.15.2:0/3398094512 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe99c0cdb70 0x7fe99c06b8c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:08.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.070+0000 7fe9a23e3700 1 -- 172.21.15.2:0/3398094512 >> 172.21.15.2:0/3398094512 conn(0x7fe99c0cfea0 msgr2=0x7fe99c0d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:08.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.071+0000 7fe9a23e3700 1 -- 172.21.15.2:0/3398094512 shutdown_connections 2023-08-25T03:35:08.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.071+0000 7fe9a23e3700 1 -- 172.21.15.2:0/3398094512 wait complete. 2023-08-25T03:35:08.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.072+0000 7fe9a23e3700 1 Processor -- start 2023-08-25T03:35:08.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.072+0000 7fe9a23e3700 1 -- start start 2023-08-25T03:35:08.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.073+0000 7fe9a23e3700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe99c0b1fb0 0x7fe99c151670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:08.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.073+0000 7fe9a23e3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe99c0cdb70 0x7fe99c156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:08.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.073+0000 7fe9a23e3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe99c0ce000 0x7fe99c156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:08.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.073+0000 7fe9a23e3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c159810 con 0x7fe99c0ce000 2023-08-25T03:35:08.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.073+0000 7fe9a23e3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c1f3c40 con 0x7fe99c0cdb70 2023-08-25T03:35:08.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.073+0000 7fe9a23e3700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe99c1f3660 con 0x7fe99c0b1fb0 2023-08-25T03:35:08.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.073+0000 7fe9a0980700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe99c0ce000 0x7fe99c156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:08.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.073+0000 7fe9a0980700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe99c0ce000 0x7fe99c156880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33106/0 (socket says 172.21.15.2:33106) 2023-08-25T03:35:08.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.073+0000 7fe9a0980700 1 -- 172.21.15.2:0/3219612815 learned_addr learned my addr 172.21.15.2:0/3219612815 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:08.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.073+0000 7fe99bfff700 1 --2- 172.21.15.2:0/3219612815 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe99c0b1fb0 0x7fe99c151670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:08.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.073+0000 7fe99b7fe700 1 --2- 172.21.15.2:0/3219612815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe99c0cdb70 0x7fe99c156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:08.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.073+0000 7fe9a0980700 1 -- 172.21.15.2:0/3219612815 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe99c0b1fb0 msgr2=0x7fe99c151670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:08.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.073+0000 7fe9a0980700 1 --2- 172.21.15.2:0/3219612815 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe99c0b1fb0 0x7fe99c151670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:08.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.073+0000 7fe9a0980700 1 -- 172.21.15.2:0/3219612815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe99c0cdb70 msgr2=0x7fe99c156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:08.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.073+0000 7fe9a0980700 1 --2- 172.21.15.2:0/3219612815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe99c0cdb70 0x7fe99c156340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:08.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.073+0000 7fe9a0980700 1 -- 172.21.15.2:0/3219612815 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe984009550 con 0x7fe99c0ce000 2023-08-25T03:35:08.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.073+0000 7fe9a0980700 1 --2- 172.21.15.2:0/3219612815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe99c0ce000 0x7fe99c156880 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe994007e40 tx=0x7fe99400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:08.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.073+0000 7fe998ff9700 1 -- 172.21.15.2:0/3219612815 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe99400eea0 con 0x7fe99c0ce000 2023-08-25T03:35:08.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.074+0000 7fe998ff9700 1 -- 172.21.15.2:0/3219612815 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe994014420 con 0x7fe99c0ce000 2023-08-25T03:35:08.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.074+0000 7fe9a23e3700 1 -- 172.21.15.2:0/3219612815 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe99c1f3940 con 0x7fe99c0ce000 2023-08-25T03:35:08.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.074+0000 7fe998ff9700 1 -- 172.21.15.2:0/3219612815 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9940147b0 con 0x7fe99c0ce000 2023-08-25T03:35:08.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.074+0000 7fe9a23e3700 1 -- 172.21.15.2:0/3219612815 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe99c159430 con 0x7fe99c0ce000 2023-08-25T03:35:08.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.074+0000 7fe9a23e3700 1 -- 172.21.15.2:0/3219612815 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe99c066f70 con 0x7fe99c0ce000 2023-08-25T03:35:08.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.075+0000 7fe998ff9700 1 -- 172.21.15.2:0/3219612815 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe994014910 con 0x7fe99c0ce000 2023-08-25T03:35:08.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.075+0000 7fe998ff9700 1 --2- 172.21.15.2:0/3219612815 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe97c06a0e0 0x7fe97c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:08.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.076+0000 7fe99bfff700 1 --2- 172.21.15.2:0/3219612815 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe97c06a0e0 0x7fe97c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:08.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.076+0000 7fe998ff9700 1 -- 172.21.15.2:0/3219612815 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe994088da0 con 0x7fe99c0ce000 2023-08-25T03:35:08.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.076+0000 7fe99bfff700 1 --2- 172.21.15.2:0/3219612815 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe97c06a0e0 0x7fe97c06c5a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe98c000f60 tx=0x7fe98c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:08.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.080+0000 7fe998ff9700 1 -- 172.21.15.2:0/3219612815 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe994054480 con 0x7fe99c0ce000 2023-08-25T03:35:08.283 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.282+0000 7fe9a23e3700 1 -- 172.21.15.2:0/3219612815 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe99c0746d0 con 0x7fe99c0ce000 2023-08-25T03:35:08.285 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.284+0000 7fe998ff9700 1 -- 172.21.15.2:0/3219612815 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+18220 (secure 0 0 0) 0x7fe994057aa0 con 0x7fe99c0ce000 2023-08-25T03:35:08.285 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:35:08.286 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":36,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:35:07.333527+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:35:05.870790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-08-25T03:35:06.296959+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:35:08.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.286+0000 7fe9a23e3700 1 -- 172.21.15.2:0/3219612815 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe97c06a0e0 msgr2=0x7fe97c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:08.287 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.286+0000 7fe9a23e3700 1 --2- 172.21.15.2:0/3219612815 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe97c06a0e0 0x7fe97c06c5a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe98c000f60 tx=0x7fe98c009040 comp rx=0 tx=0).stop 2023-08-25T03:35:08.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.286+0000 7fe9a23e3700 1 -- 172.21.15.2:0/3219612815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe99c0ce000 msgr2=0x7fe99c156880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:08.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.286+0000 7fe9a23e3700 1 --2- 172.21.15.2:0/3219612815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe99c0ce000 0x7fe99c156880 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe994007e40 tx=0x7fe99400e3c0 comp rx=0 tx=0).stop 2023-08-25T03:35:08.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.286+0000 7fe9a23e3700 1 -- 172.21.15.2:0/3219612815 shutdown_connections 2023-08-25T03:35:08.288 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.286+0000 7fe9a23e3700 1 --2- 172.21.15.2:0/3219612815 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe97c06a0e0 0x7fe97c06c5a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:08.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.286+0000 7fe9a23e3700 1 --2- 172.21.15.2:0/3219612815 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe99c0ce000 0x7fe99c156880 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:08.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.286+0000 7fe9a23e3700 1 --2- 172.21.15.2:0/3219612815 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe99c0b1fb0 0x7fe99c151670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:08.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.286+0000 7fe9a23e3700 1 --2- 172.21.15.2:0/3219612815 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe99c0cdb70 0x7fe99c156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:08.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.286+0000 7fe9a23e3700 1 -- 172.21.15.2:0/3219612815 >> 172.21.15.2:0/3219612815 conn(0x7fe99c0cfea0 msgr2=0x7fe99c0cadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:08.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.286+0000 7fe9a23e3700 1 -- 172.21.15.2:0/3219612815 shutdown_connections 2023-08-25T03:35:08.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.286+0000 7fe9a23e3700 1 -- 172.21.15.2:0/3219612815 wait complete. 2023-08-25T03:35:08.299 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:35:08.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.427+0000 7feef03e9700 1 Processor -- start 2023-08-25T03:35:08.428 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.427+0000 7feef03e9700 1 -- start start 2023-08-25T03:35:08.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.428+0000 7feef03e9700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feee8157850 0x7feee80ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:08.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.428+0000 7feef03e9700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7feee806b900 0x7feee80b2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:08.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.428+0000 7feef03e9700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feee815dd80 con 0x7feee80cd7b0 2023-08-25T03:35:08.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.428+0000 7feef03e9700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feee80b24e0 con 0x7feee806b900 2023-08-25T03:35:08.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.428+0000 7feef03e9700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee815e060 con 0x7feee8157850 2023-08-25T03:35:08.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.428+0000 7feeee986700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7feee806b900 0x7feee80b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:08.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.428+0000 7feeed984700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7feee80cd7b0 0x7feee806b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48010/0 (socket says 172.21.15.2:48010) 2023-08-25T03:35:08.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.428+0000 7feeee986700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7feee806b900 0x7feee80b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33114/0 (socket says 172.21.15.2:33114) 2023-08-25T03:35:08.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.428+0000 7feeed984700 1 -- 172.21.15.2:0/1867235170 learned_addr learned my addr 172.21.15.2:0/1867235170 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:08.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.428+0000 7feeee185700 1 --2- 172.21.15.2:0/1867235170 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feee8157850 0x7feee80ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:08.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.428+0000 7feeee986700 1 -- 172.21.15.2:0/1867235170 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feee8157850 msgr2=0x7feee80ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:08.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.428+0000 7feeee986700 1 --2- 172.21.15.2:0/1867235170 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feee8157850 0x7feee80ceee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:08.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.428+0000 7feeee986700 1 -- 172.21.15.2:0/1867235170 >> v1:172.21.15.2:6789/0 conn(0x7feee80cd7b0 legacy=0x7feee806b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:08.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.428+0000 7feeee986700 1 -- 172.21.15.2:0/1867235170 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee815e340 con 0x7feee806b900 2023-08-25T03:35:08.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.428+0000 7feeed183700 1 -- 172.21.15.2:0/1867235170 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3386149784 0 0) 0x7feee815dd80 con 0x7feee80cd7b0 2023-08-25T03:35:08.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.428+0000 7feeee185700 1 --2- 172.21.15.2:0/1867235170 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feee8157850 0x7feee80ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:08.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.428+0000 7feeee986700 1 --2- 172.21.15.2:0/1867235170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feee806b900 0x7feee80b2930 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7feee0009d90 tx=0x7feee000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1211f2fa694d3c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:08.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.429+0000 7feeed183700 1 -- 172.21.15.2:0/1867235170 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee000e070 con 0x7feee806b900 2023-08-25T03:35:08.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.429+0000 7feeed183700 1 -- 172.21.15.2:0/1867235170 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee000bbd0 con 0x7feee806b900 2023-08-25T03:35:08.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.429+0000 7feeed183700 1 -- 172.21.15.2:0/1867235170 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee0012430 con 0x7feee806b900 2023-08-25T03:35:08.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.429+0000 7feef03e9700 1 -- 172.21.15.2:0/1867235170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feee806b900 msgr2=0x7feee80b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:08.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.429+0000 7feef03e9700 1 --2- 172.21.15.2:0/1867235170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feee806b900 0x7feee80b2930 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7feee0009d90 tx=0x7feee000b330 comp rx=0 tx=0).stop 2023-08-25T03:35:08.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.429+0000 7feef03e9700 1 -- 172.21.15.2:0/1867235170 shutdown_connections 2023-08-25T03:35:08.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.429+0000 7feef03e9700 1 --2- 172.21.15.2:0/1867235170 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feee806b900 0x7feee80b2930 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:08.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.429+0000 7feef03e9700 1 --2- 172.21.15.2:0/1867235170 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feee8157850 0x7feee80ceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:08.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.429+0000 7feef03e9700 1 -- 172.21.15.2:0/1867235170 >> 172.21.15.2:0/1867235170 conn(0x7feee80cfd10 msgr2=0x7feee806ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:08.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.429+0000 7feef03e9700 1 -- 172.21.15.2:0/1867235170 shutdown_connections 2023-08-25T03:35:08.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.429+0000 7feef03e9700 1 -- 172.21.15.2:0/1867235170 wait complete. 2023-08-25T03:35:08.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.430+0000 7feef03e9700 1 Processor -- start 2023-08-25T03:35:08.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.430+0000 7feef03e9700 1 -- start start 2023-08-25T03:35:08.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.431+0000 7feef03e9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feee806b900 0x7feee814a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:08.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.431+0000 7feef03e9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feee80cd7b0 0x7feee8156a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:08.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.431+0000 7feef03e9700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feee8157850 0x7feee8152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:08.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.431+0000 7feef03e9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee814aa80 con 0x7feee80cd7b0 2023-08-25T03:35:08.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.431+0000 7feef03e9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feee814c2d0 con 0x7feee806b900 2023-08-25T03:35:08.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.431+0000 7feef03e9700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee814e770 con 0x7feee8157850 2023-08-25T03:35:08.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.431+0000 7feeed984700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feee80cd7b0 0x7feee8156a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:08.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.431+0000 7feeed984700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feee80cd7b0 0x7feee8156a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33118/0 (socket says 172.21.15.2:33118) 2023-08-25T03:35:08.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.431+0000 7feeed984700 1 -- 172.21.15.2:0/3716299916 learned_addr learned my addr 172.21.15.2:0/3716299916 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:08.437 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.431+0000 7feeee986700 1 --2- 172.21.15.2:0/3716299916 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feee8157850 0x7feee8152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:08.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.431+0000 7feeee185700 1 --2- 172.21.15.2:0/3716299916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feee806b900 0x7feee814a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:08.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.432+0000 7feeed984700 1 -- 172.21.15.2:0/3716299916 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feee8157850 msgr2=0x7feee8152910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:08.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.432+0000 7feeed984700 1 --2- 172.21.15.2:0/3716299916 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feee8157850 0x7feee8152910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:08.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.432+0000 7feeed984700 1 -- 172.21.15.2:0/3716299916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feee806b900 msgr2=0x7feee814a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:08.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.432+0000 7feeed984700 1 --2- 172.21.15.2:0/3716299916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feee806b900 0x7feee814a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:08.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.432+0000 7feeed984700 1 -- 172.21.15.2:0/3716299916 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee0009a70 con 0x7feee80cd7b0 2023-08-25T03:35:08.438 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.432+0000 7feeed984700 1 --2- 172.21.15.2:0/3716299916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feee80cd7b0 0x7feee8156a50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7feed80011e0 tx=0x7feed800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:08.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.432+0000 7feedeffd700 1 -- 172.21.15.2:0/3716299916 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed8004c80 con 0x7feee80cd7b0 2023-08-25T03:35:08.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.432+0000 7feedeffd700 1 -- 172.21.15.2:0/3716299916 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed8004e00 con 0x7feee80cd7b0 2023-08-25T03:35:08.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.432+0000 7feef03e9700 1 -- 172.21.15.2:0/3716299916 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feee814ea50 con 0x7feee80cd7b0 2023-08-25T03:35:08.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.432+0000 7feedeffd700 1 -- 172.21.15.2:0/3716299916 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed8007310 con 0x7feee80cd7b0 2023-08-25T03:35:08.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.432+0000 7feef03e9700 1 -- 172.21.15.2:0/3716299916 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feee814f010 con 0x7feee80cd7b0 2023-08-25T03:35:08.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.433+0000 7feef03e9700 1 -- 172.21.15.2:0/3716299916 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feee814c4a0 con 0x7feee80cd7b0 2023-08-25T03:35:08.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.433+0000 7feedeffd700 1 -- 172.21.15.2:0/3716299916 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7feed8007470 con 0x7feee80cd7b0 2023-08-25T03:35:08.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.434+0000 7feedeffd700 1 --2- 172.21.15.2:0/3716299916 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7feecc06a0e0 0x7feecc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:08.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.434+0000 7feedeffd700 1 -- 172.21.15.2:0/3716299916 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7feed80881a0 con 0x7feee80cd7b0 2023-08-25T03:35:08.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.434+0000 7feeee185700 1 --2- 172.21.15.2:0/3716299916 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7feecc06a0e0 0x7feecc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:08.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.434+0000 7feeee185700 1 --2- 172.21.15.2:0/3716299916 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7feecc06a0e0 0x7feecc06c5a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7feee400c940 tx=0x7feee4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:08.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.438+0000 7feedeffd700 1 -- 172.21.15.2:0/3716299916 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7feee814c4a0 con 0x7feee80cd7b0 2023-08-25T03:35:08.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.638+0000 7feef03e9700 1 -- 172.21.15.2:0/3716299916 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feee815d850 con 0x7feee80cd7b0 2023-08-25T03:35:08.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.639+0000 7feedeffd700 1 -- 172.21.15.2:0/3716299916 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18207 (secure 0 0 0) 0x7feed8053920 con 0x7feee80cd7b0 2023-08-25T03:35:08.640 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:35:08.640 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":37,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:35:08.407381+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:35:05.870790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-08-25T03:35:06.296959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:35:08.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.641+0000 7feef03e9700 1 -- 172.21.15.2:0/3716299916 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7feecc06a0e0 msgr2=0x7feecc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:08.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.641+0000 7feef03e9700 1 --2- 172.21.15.2:0/3716299916 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7feecc06a0e0 0x7feecc06c5a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7feee400c940 tx=0x7feee4008040 comp rx=0 tx=0).stop 2023-08-25T03:35:08.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.641+0000 7feef03e9700 1 -- 172.21.15.2:0/3716299916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feee80cd7b0 msgr2=0x7feee8156a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:08.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.641+0000 7feef03e9700 1 --2- 172.21.15.2:0/3716299916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feee80cd7b0 0x7feee8156a50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7feed80011e0 tx=0x7feed800ca30 comp rx=0 tx=0).stop 2023-08-25T03:35:08.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.641+0000 7feef03e9700 1 -- 172.21.15.2:0/3716299916 shutdown_connections 2023-08-25T03:35:08.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.641+0000 7feef03e9700 1 --2- 172.21.15.2:0/3716299916 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feee8157850 0x7feee8152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:08.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.641+0000 7feef03e9700 1 --2- 172.21.15.2:0/3716299916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feee806b900 0x7feee814a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:08.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.641+0000 7feef03e9700 1 --2- 172.21.15.2:0/3716299916 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7feecc06a0e0 0x7feecc06c5a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:08.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.641+0000 7feef03e9700 1 --2- 172.21.15.2:0/3716299916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7feee80cd7b0 0x7feee8156a50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:08.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.641+0000 7feef03e9700 1 -- 172.21.15.2:0/3716299916 >> 172.21.15.2:0/3716299916 conn(0x7feee80cfd10 msgr2=0x7feee815d010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:08.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.641+0000 7feef03e9700 1 -- 172.21.15.2:0/3716299916 shutdown_connections 2023-08-25T03:35:08.644 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.641+0000 7feef03e9700 1 -- 172.21.15.2:0/3716299916 wait complete. 2023-08-25T03:35:08.655 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-08-25T03:35:08.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.783+0000 7f3f9ff0c700 1 Processor -- start 2023-08-25T03:35:08.785 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.784+0000 7f3f9ff0c700 1 -- start start 2023-08-25T03:35:08.786 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.785+0000 7f3f9ff0c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f3f9806a8f0 0x7f3f98070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:08.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.785+0000 7f3f9ff0c700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f98160bd0 con 0x7f3f980af180 2023-08-25T03:35:08.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.785+0000 7f3f9ff0c700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f980cdd00 con 0x7f3f981607f0 2023-08-25T03:35:08.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.785+0000 7f3f9ff0c700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f980713a0 con 0x7f3f9806a8f0 2023-08-25T03:35:08.787 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.785+0000 7f3f9d4a7700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f3f980af180 0x7f3f9806a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48024/0 (socket says 172.21.15.2:48024) 2023-08-25T03:35:08.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.785+0000 7f3f9d4a7700 1 -- 172.21.15.2:0/3704563370 learned_addr learned my addr 172.21.15.2:0/3704563370 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:08.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.785+0000 7f3f9e4a9700 1 --2- 172.21.15.2:0/3704563370 >> v2:172.21.15.31:3300/0 conn(0x7f3f9806a8f0 0x7f3f98070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:08.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.785+0000 7f3f9cca6700 1 -- 172.21.15.2:0/3704563370 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 347148275 0 0) 0x7f3f98160bd0 con 0x7f3f980af180 2023-08-25T03:35:08.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.785+0000 7f3f9cca6700 1 -- 172.21.15.2:0/3704563370 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f840035b0 con 0x7f3f980af180 2023-08-25T03:35:08.788 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.785+0000 7f3f9cca6700 1 -- 172.21.15.2:0/3704563370 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3960459981 0 0) 0x7f3f840035b0 con 0x7f3f980af180 2023-08-25T03:35:08.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.785+0000 7f3f9cca6700 1 -- 172.21.15.2:0/3704563370 >> v2:172.21.15.31:3300/0 conn(0x7f3f9806a8f0 msgr2=0x7f3f98070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:08.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.785+0000 7f3f9cca6700 1 --2- 172.21.15.2:0/3704563370 >> v2:172.21.15.31:3300/0 conn(0x7f3f9806a8f0 0x7f3f98070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:08.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.785+0000 7f3f9cca6700 1 -- 172.21.15.2:0/3704563370 >> v1:172.21.15.31:6789/0 conn(0x7f3f981607f0 legacy=0x7f3f980ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:08.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.785+0000 7f3f9cca6700 1 -- 172.21.15.2:0/3704563370 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f980cda00 con 0x7f3f980af180 2023-08-25T03:35:08.789 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.785+0000 7f3f9cca6700 1 -- 172.21.15.2:0/3704563370 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f3f8c001670 con 0x7f3f980af180 2023-08-25T03:35:08.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.785+0000 7f3f9cca6700 1 -- 172.21.15.2:0/3704563370 >> v1:172.21.15.2:6789/0 conn(0x7f3f980af180 legacy=0x7f3f9806a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:08.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.785+0000 7f3f9cca6700 1 --2- 172.21.15.2:0/3704563370 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3f84003a00 0x7f3f84003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:08.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.785+0000 7f3f9cca6700 1 --2- 172.21.15.2:0/3704563370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f84004850 0x7f3f84004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:08.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.785+0000 7f3f9cca6700 1 --2- 172.21.15.2:0/3704563370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f84005210 0x7f3f8400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:08.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.785+0000 7f3f9cca6700 1 -- 172.21.15.2:0/3704563370 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8400c180 con 0x7f3f84005210 2023-08-25T03:35:08.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.785+0000 7f3f9cca6700 1 -- 172.21.15.2:0/3704563370 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8400c2f0 con 0x7f3f84004850 2023-08-25T03:35:08.790 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.785+0000 7f3f9cca6700 1 -- 172.21.15.2:0/3704563370 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8400c4b0 con 0x7f3f84003a00 2023-08-25T03:35:08.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.785+0000 7f3f9e4a9700 1 --2- 172.21.15.2:0/3704563370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f84005210 0x7f3f8400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:08.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.786+0000 7f3f9d4a7700 1 --2- 172.21.15.2:0/3704563370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f84004850 0x7f3f84004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:08.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.786+0000 7f3f9dca8700 1 --2- 172.21.15.2:0/3704563370 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3f84003a00 0x7f3f84003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:08.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.786+0000 7f3f9e4a9700 1 -- 172.21.15.2:0/3704563370 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3f84003a00 msgr2=0x7f3f84003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:08.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.786+0000 7f3f9e4a9700 1 --2- 172.21.15.2:0/3704563370 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3f84003a00 0x7f3f84003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:08.791 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.786+0000 7f3f9e4a9700 1 -- 172.21.15.2:0/3704563370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f84004850 msgr2=0x7f3f84004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:08.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.786+0000 7f3f9e4a9700 1 --2- 172.21.15.2:0/3704563370 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f84004850 0x7f3f84004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:08.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.786+0000 7f3f9e4a9700 1 -- 172.21.15.2:0/3704563370 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f840035b0 con 0x7f3f84005210 2023-08-25T03:35:08.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.786+0000 7f3f9e4a9700 1 -- 172.21.15.2:0/3704563370 reap_dead start 2023-08-25T03:35:08.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.786+0000 7f3f9e4a9700 1 --2- 172.21.15.2:0/3704563370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f84005210 0x7f3f8400ba10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3f90009fc0 tx=0x7f3f84005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328bc3692165ee7f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:08.792 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.786+0000 7f3f9cca6700 1 -- 172.21.15.2:0/3704563370 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f90003a70 con 0x7f3f84005210 2023-08-25T03:35:08.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.786+0000 7f3f9cca6700 1 -- 172.21.15.2:0/3704563370 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f9000f040 con 0x7f3f84005210 2023-08-25T03:35:08.793 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.786+0000 7f3f9cca6700 1 -- 172.21.15.2:0/3704563370 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f90003e30 con 0x7f3f84005210 2023-08-25T03:35:08.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.786+0000 7f3f9ff0c700 1 -- 172.21.15.2:0/3704563370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f84005210 msgr2=0x7f3f8400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:08.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.786+0000 7f3f9ff0c700 1 --2- 172.21.15.2:0/3704563370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f84005210 0x7f3f8400ba10 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3f90009fc0 tx=0x7f3f84005880 comp rx=0 tx=0).stop 2023-08-25T03:35:08.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.786+0000 7f3f9ff0c700 1 -- 172.21.15.2:0/3704563370 shutdown_connections 2023-08-25T03:35:08.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.786+0000 7f3f9ff0c700 1 --2- 172.21.15.2:0/3704563370 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f84005210 0x7f3f8400ba10 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:08.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.786+0000 7f3f9ff0c700 1 -- 172.21.15.2:0/3704563370 >> 172.21.15.2:0/3704563370 conn(0x7f3f980cfd90 msgr2=0x7f3f980d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:08.796 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.787+0000 7f3f9ff0c700 1 -- 172.21.15.2:0/3704563370 shutdown_connections 2023-08-25T03:35:08.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.787+0000 7f3f9ff0c700 1 -- 172.21.15.2:0/3704563370 wait complete. 2023-08-25T03:35:08.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.788+0000 7f3f9ff0c700 1 Processor -- start 2023-08-25T03:35:08.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.788+0000 7f3f9ff0c700 1 -- start start 2023-08-25T03:35:08.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.789+0000 7f3f9ff0c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f84005210 0x7f3f980c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:08.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.789+0000 7f3f9ff0c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f9815fcf0 0x7f3f9815a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:08.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.789+0000 7f3f9ff0c700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3f980ad420 0x7f3f980ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:08.797 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.789+0000 7f3f9ff0c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f980addf0 con 0x7f3f84005210 2023-08-25T03:35:08.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.789+0000 7f3f9ff0c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9815ccb0 con 0x7f3f9815fcf0 2023-08-25T03:35:08.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.789+0000 7f3f9ff0c700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f980c4710 con 0x7f3f980ad420 2023-08-25T03:35:08.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.789+0000 7f3f9dca8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f84005210 0x7f3f980c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:08.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.789+0000 7f3f9dca8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f84005210 0x7f3f980c3ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33136/0 (socket says 172.21.15.2:33136) 2023-08-25T03:35:08.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.789+0000 7f3f9dca8700 1 -- 172.21.15.2:0/4283101087 learned_addr learned my addr 172.21.15.2:0/4283101087 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:08.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.789+0000 7f3f9dca8700 1 -- 172.21.15.2:0/4283101087 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3f980ad420 msgr2=0x7f3f980ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:08.798 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.789+0000 7f3f9dca8700 1 --2- 172.21.15.2:0/4283101087 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3f980ad420 0x7f3f980ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:08.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.789+0000 7f3f9dca8700 1 -- 172.21.15.2:0/4283101087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f9815fcf0 msgr2=0x7f3f9815a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:08.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.789+0000 7f3f9dca8700 1 --2- 172.21.15.2:0/4283101087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f9815fcf0 0x7f3f9815a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:08.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.789+0000 7f3f9dca8700 1 -- 172.21.15.2:0/4283101087 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f9815cf50 con 0x7f3f84005210 2023-08-25T03:35:08.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.789+0000 7f3f9dca8700 1 --2- 172.21.15.2:0/4283101087 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f84005210 0x7f3f980c3ee0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3f9400e550 tx=0x7f3f9400e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:08.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.789+0000 7f3f8a7fc700 1 -- 172.21.15.2:0/4283101087 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f940052e0 con 0x7f3f84005210 2023-08-25T03:35:08.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.789+0000 7f3f8a7fc700 1 -- 172.21.15.2:0/4283101087 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f94005440 con 0x7f3f84005210 2023-08-25T03:35:08.799 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.789+0000 7f3f9ff0c700 1 -- 172.21.15.2:0/4283101087 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f9815d230 con 0x7f3f84005210 2023-08-25T03:35:08.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.789+0000 7f3f8a7fc700 1 -- 172.21.15.2:0/4283101087 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f940038a0 con 0x7f3f84005210 2023-08-25T03:35:08.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.789+0000 7f3f9ff0c700 1 -- 172.21.15.2:0/4283101087 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f980c49f0 con 0x7f3f84005210 2023-08-25T03:35:08.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.790+0000 7f3f9ff0c700 1 -- 172.21.15.2:0/4283101087 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f980746d0 con 0x7f3f84005210 2023-08-25T03:35:08.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.791+0000 7f3f8a7fc700 1 -- 172.21.15.2:0/4283101087 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3f94003a00 con 0x7f3f84005210 2023-08-25T03:35:08.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.791+0000 7f3f8a7fc700 1 --2- 172.21.15.2:0/4283101087 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3f7c06a0e0 0x7f3f7c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:08.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.791+0000 7f3f8a7fc700 1 -- 172.21.15.2:0/4283101087 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f3f94087c00 con 0x7f3f84005210 2023-08-25T03:35:08.800 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.791+0000 7f3f9d4a7700 1 --2- 172.21.15.2:0/4283101087 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3f7c06a0e0 0x7f3f7c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:08.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.792+0000 7f3f9d4a7700 1 --2- 172.21.15.2:0/4283101087 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3f7c06a0e0 0x7f3f7c06c5a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c005940 tx=0x7f3f8c000ef0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:08.801 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:08.796+0000 7f3f8a7fc700 1 -- 172.21.15.2:0/4283101087 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3f9408e080 con 0x7f3f84005210 2023-08-25T03:35:09.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:09.043+0000 7f3f9ff0c700 1 -- 172.21.15.2:0/4283101087 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3f980ae800 con 0x7f3f84005210 2023-08-25T03:35:09.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:09.933+0000 7f3f8a7fc700 1 -- 172.21.15.2:0/4283101087 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3f94053470 con 0x7f3f84005210 2023-08-25T03:35:09.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:09.936+0000 7f3f9ff0c700 1 -- 172.21.15.2:0/4283101087 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3f7c06a0e0 msgr2=0x7f3f7c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:09.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:09.936+0000 7f3f9ff0c700 1 --2- 172.21.15.2:0/4283101087 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3f7c06a0e0 0x7f3f7c06c5a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c005940 tx=0x7f3f8c000ef0 comp rx=0 tx=0).stop 2023-08-25T03:35:09.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:09.936+0000 7f3f9ff0c700 1 -- 172.21.15.2:0/4283101087 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f84005210 msgr2=0x7f3f980c3ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:09.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:09.936+0000 7f3f9ff0c700 1 --2- 172.21.15.2:0/4283101087 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f84005210 0x7f3f980c3ee0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3f9400e550 tx=0x7f3f9400e910 comp rx=0 tx=0).stop 2023-08-25T03:35:09.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:09.936+0000 7f3f9ff0c700 1 -- 172.21.15.2:0/4283101087 shutdown_connections 2023-08-25T03:35:09.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:09.936+0000 7f3f9ff0c700 1 --2- 172.21.15.2:0/4283101087 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3f7c06a0e0 0x7f3f7c06c5a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:09.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:09.936+0000 7f3f9ff0c700 1 --2- 172.21.15.2:0/4283101087 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3f84005210 0x7f3f980c3ee0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:09.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:09.936+0000 7f3f9ff0c700 1 --2- 172.21.15.2:0/4283101087 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3f980ad420 0x7f3f980ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:09.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:09.936+0000 7f3f9ff0c700 1 --2- 172.21.15.2:0/4283101087 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f9815fcf0 0x7f3f9815a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:09.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:09.936+0000 7f3f9ff0c700 1 -- 172.21.15.2:0/4283101087 >> 172.21.15.2:0/4283101087 conn(0x7f3f980cfd90 msgr2=0x7f3f980af860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:09.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:09.936+0000 7f3f9ff0c700 1 -- 172.21.15.2:0/4283101087 shutdown_connections 2023-08-25T03:35:09.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:09.936+0000 7f3f9ff0c700 1 -- 172.21.15.2:0/4283101087 wait complete. 2023-08-25T03:35:09.951 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-08-25T03:35:09.952 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-08-25T03:35:10.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.086+0000 7fefc20c0700 1 Processor -- start 2023-08-25T03:35:10.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.086+0000 7fefc20c0700 1 -- start start 2023-08-25T03:35:10.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.087+0000 7fefc20c0700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fefbc1607b0 0x7fefbc0cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:10.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.087+0000 7fefc20c0700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fefbc0d2b20 0x7fefbc0ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:10.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.087+0000 7fefc20c0700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefbc160b90 con 0x7fefbc0d1510 2023-08-25T03:35:10.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.087+0000 7fefc20c0700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefbc0d1a80 con 0x7fefbc1607b0 2023-08-25T03:35:10.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.087+0000 7fefc20c0700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc0d1d60 con 0x7fefbc0d2b20 2023-08-25T03:35:10.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.087+0000 7fefbb7fe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fefbc1607b0 0x7fefbc0cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:10.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.087+0000 7fefbaffd700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fefbc0d1510 0x7fefbc06c260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48044/0 (socket says 172.21.15.2:48044) 2023-08-25T03:35:10.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.087+0000 7fefbaffd700 1 -- 172.21.15.2:0/114288201 learned_addr learned my addr 172.21.15.2:0/114288201 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:10.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.087+0000 7fefbb7fe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fefbc1607b0 0x7fefbc0cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33144/0 (socket says 172.21.15.2:33144) 2023-08-25T03:35:10.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.088+0000 7fefbbfff700 1 --2- 172.21.15.2:0/114288201 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fefbc0d2b20 0x7fefbc0ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:10.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.088+0000 7fefbb7fe700 1 -- 172.21.15.2:0/114288201 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fefbc0d2b20 msgr2=0x7fefbc0ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:10.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.088+0000 7fefbb7fe700 1 --2- 172.21.15.2:0/114288201 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fefbc0d2b20 0x7fefbc0ce300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:10.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.088+0000 7fefbb7fe700 1 -- 172.21.15.2:0/114288201 >> v1:172.21.15.2:6789/0 conn(0x7fefbc0d1510 legacy=0x7fefbc06c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:10.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.088+0000 7fefba7fc700 1 -- 172.21.15.2:0/114288201 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2253966895 0 0) 0x7fefbc160b90 con 0x7fefbc0d1510 2023-08-25T03:35:10.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.088+0000 7fefbb7fe700 1 -- 172.21.15.2:0/114288201 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefbc0d2040 con 0x7fefbc1607b0 2023-08-25T03:35:10.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.088+0000 7fefbbfff700 1 --2- 172.21.15.2:0/114288201 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fefbc0d2b20 0x7fefbc0ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:35:10.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.088+0000 7fefbb7fe700 1 --2- 172.21.15.2:0/114288201 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefbc1607b0 0x7fefbc0cddc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fefac009840 tx=0x7fefac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e828cecffda3f866 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:10.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.088+0000 7fefba7fc700 1 -- 172.21.15.2:0/114288201 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac011830 con 0x7fefbc1607b0 2023-08-25T03:35:10.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.088+0000 7fefba7fc700 1 -- 172.21.15.2:0/114288201 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefac011990 con 0x7fefbc1607b0 2023-08-25T03:35:10.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.088+0000 7fefba7fc700 1 -- 172.21.15.2:0/114288201 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac011d20 con 0x7fefbc1607b0 2023-08-25T03:35:10.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.088+0000 7fefc20c0700 1 -- 172.21.15.2:0/114288201 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefbc1607b0 msgr2=0x7fefbc0cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:10.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.088+0000 7fefc20c0700 1 --2- 172.21.15.2:0/114288201 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefbc1607b0 0x7fefbc0cddc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fefac009840 tx=0x7fefac009b50 comp rx=0 tx=0).stop 2023-08-25T03:35:10.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.089+0000 7fefc20c0700 1 -- 172.21.15.2:0/114288201 shutdown_connections 2023-08-25T03:35:10.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.089+0000 7fefc20c0700 1 --2- 172.21.15.2:0/114288201 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fefbc0d2b20 0x7fefbc0ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:10.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.089+0000 7fefc20c0700 1 --2- 172.21.15.2:0/114288201 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefbc1607b0 0x7fefbc0cddc0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:10.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.089+0000 7fefc20c0700 1 -- 172.21.15.2:0/114288201 >> 172.21.15.2:0/114288201 conn(0x7fefbc0cfb70 msgr2=0x7fefbc0cff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:10.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.089+0000 7fefc20c0700 1 -- 172.21.15.2:0/114288201 shutdown_connections 2023-08-25T03:35:10.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.089+0000 7fefc20c0700 1 -- 172.21.15.2:0/114288201 wait complete. 2023-08-25T03:35:10.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.090+0000 7fefc20c0700 1 Processor -- start 2023-08-25T03:35:10.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.090+0000 7fefc20c0700 1 -- start start 2023-08-25T03:35:10.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.091+0000 7fefc20c0700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fefbc0d1510 0x7fefbc146510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:10.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.091+0000 7fefc20c0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefbc0d2b20 0x7fefbc1582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:10.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.091+0000 7fefc20c0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefbc1607b0 0x7fefbc0cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:10.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.091+0000 7fefc20c0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc1587f0 con 0x7fefbc0d2b20 2023-08-25T03:35:10.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.091+0000 7fefc20c0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc1f2b90 con 0x7fefbc1607b0 2023-08-25T03:35:10.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.091+0000 7fefc20c0700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefbc149840 con 0x7fefbc0d1510 2023-08-25T03:35:10.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.091+0000 7fefbaffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefbc0d2b20 0x7fefbc1582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:10.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.091+0000 7fefbaffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefbc0d2b20 0x7fefbc1582b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33152/0 (socket says 172.21.15.2:33152) 2023-08-25T03:35:10.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.091+0000 7fefbaffd700 1 -- 172.21.15.2:0/347181770 learned_addr learned my addr 172.21.15.2:0/347181770 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:10.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.091+0000 7fefbb7fe700 1 --2- 172.21.15.2:0/347181770 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fefbc0d1510 0x7fefbc146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:10.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.091+0000 7fefbbfff700 1 --2- 172.21.15.2:0/347181770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefbc1607b0 0x7fefbc0cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:10.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.091+0000 7fefbaffd700 1 -- 172.21.15.2:0/347181770 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fefbc0d1510 msgr2=0x7fefbc146510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:10.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.091+0000 7fefbaffd700 1 --2- 172.21.15.2:0/347181770 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fefbc0d1510 0x7fefbc146510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:10.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.091+0000 7fefbaffd700 1 -- 172.21.15.2:0/347181770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefbc1607b0 msgr2=0x7fefbc0cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:10.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.091+0000 7fefbaffd700 1 --2- 172.21.15.2:0/347181770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefbc1607b0 0x7fefbc0cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:10.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.091+0000 7fefbaffd700 1 -- 172.21.15.2:0/347181770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefac0094a0 con 0x7fefbc0d2b20 2023-08-25T03:35:10.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.091+0000 7fefbaffd700 1 --2- 172.21.15.2:0/347181770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefbc0d2b20 0x7fefbc1582b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fefa40011e0 tx=0x7fefa400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:10.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.091+0000 7fefb3fff700 1 -- 172.21.15.2:0/347181770 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa4004c80 con 0x7fefbc0d2b20 2023-08-25T03:35:10.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.091+0000 7fefc20c0700 1 -- 172.21.15.2:0/347181770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefbc146ca0 con 0x7fefbc0d2b20 2023-08-25T03:35:10.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.091+0000 7fefb3fff700 1 -- 172.21.15.2:0/347181770 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa4004e00 con 0x7fefbc0d2b20 2023-08-25T03:35:10.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.091+0000 7fefb3fff700 1 -- 172.21.15.2:0/347181770 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa4007310 con 0x7fefbc0d2b20 2023-08-25T03:35:10.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.092+0000 7fefc20c0700 1 -- 172.21.15.2:0/347181770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefbc147260 con 0x7fefbc0d2b20 2023-08-25T03:35:10.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.092+0000 7fefc20c0700 1 -- 172.21.15.2:0/347181770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefbc0746d0 con 0x7fefbc0d2b20 2023-08-25T03:35:10.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.093+0000 7fefb3fff700 1 -- 172.21.15.2:0/347181770 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fefa4007470 con 0x7fefbc0d2b20 2023-08-25T03:35:10.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.093+0000 7fefb3fff700 1 --2- 172.21.15.2:0/347181770 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fef9c06a0e0 0x7fef9c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:10.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.093+0000 7fefb3fff700 1 -- 172.21.15.2:0/347181770 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fefa40882d0 con 0x7fefbc0d2b20 2023-08-25T03:35:10.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.093+0000 7fefbb7fe700 1 --2- 172.21.15.2:0/347181770 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fef9c06a0e0 0x7fef9c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:10.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.094+0000 7fefbb7fe700 1 --2- 172.21.15.2:0/347181770 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fef9c06a0e0 0x7fef9c06c5a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fefac000c00 tx=0x7fefac008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:10.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.098+0000 7fefb3fff700 1 -- 172.21.15.2:0/347181770 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fefa40539d0 con 0x7fefbc0d2b20 2023-08-25T03:35:10.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.341+0000 7fefc20c0700 1 -- 172.21.15.2:0/347181770 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fefbc15d300 con 0x7fefbc0d2b20 2023-08-25T03:35:10.935 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.935+0000 7fefb3fff700 1 -- 172.21.15.2:0/347181770 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7fefa4056ff0 con 0x7fefbc0d2b20 2023-08-25T03:35:10.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.938+0000 7fefc20c0700 1 -- 172.21.15.2:0/347181770 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fef9c06a0e0 msgr2=0x7fef9c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:10.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.938+0000 7fefc20c0700 1 --2- 172.21.15.2:0/347181770 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fef9c06a0e0 0x7fef9c06c5a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fefac000c00 tx=0x7fefac008030 comp rx=0 tx=0).stop 2023-08-25T03:35:10.941 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.938+0000 7fefc20c0700 1 -- 172.21.15.2:0/347181770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefbc0d2b20 msgr2=0x7fefbc1582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:10.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.938+0000 7fefc20c0700 1 --2- 172.21.15.2:0/347181770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefbc0d2b20 0x7fefbc1582b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fefa40011e0 tx=0x7fefa400cae0 comp rx=0 tx=0).stop 2023-08-25T03:35:10.942 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.938+0000 7fefc20c0700 1 -- 172.21.15.2:0/347181770 shutdown_connections 2023-08-25T03:35:10.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.938+0000 7fefc20c0700 1 --2- 172.21.15.2:0/347181770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fefbc1607b0 0x7fefbc0cd970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:10.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.938+0000 7fefc20c0700 1 --2- 172.21.15.2:0/347181770 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fefbc0d1510 0x7fefbc146510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:10.943 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.938+0000 7fefc20c0700 1 --2- 172.21.15.2:0/347181770 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fef9c06a0e0 0x7fef9c06c5a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:10.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.938+0000 7fefc20c0700 1 --2- 172.21.15.2:0/347181770 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fefbc0d2b20 0x7fefbc1582b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:10.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.938+0000 7fefc20c0700 1 -- 172.21.15.2:0/347181770 >> 172.21.15.2:0/347181770 conn(0x7fefbc0cfb70 msgr2=0x7fefbc0c8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:10.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.938+0000 7fefc20c0700 1 -- 172.21.15.2:0/347181770 shutdown_connections 2023-08-25T03:35:10.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:10.938+0000 7fefc20c0700 1 -- 172.21.15.2:0/347181770 wait complete. 2023-08-25T03:35:10.954 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-08-25T03:35:10.955 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T03:35:11.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.087+0000 7efeb60c3700 1 Processor -- start 2023-08-25T03:35:11.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.087+0000 7efeb60c3700 1 -- start start 2023-08-25T03:35:11.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.088+0000 7efeb60c3700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efeb00ccd50 0x7efeb00cd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:11.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.088+0000 7efeb60c3700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb0071dc0 con 0x7efeb015a410 2023-08-25T03:35:11.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.088+0000 7efeb60c3700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb00720a0 con 0x7efeb015a030 2023-08-25T03:35:11.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.088+0000 7efeb60c3700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb00cd720 con 0x7efeb00ccd50 2023-08-25T03:35:11.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.088+0000 7efeaeffd700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7efeb015a410 0x7efeb00ce170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48056/0 (socket says 172.21.15.2:48056) 2023-08-25T03:35:11.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.088+0000 7efeaeffd700 1 -- 172.21.15.2:0/3829978662 learned_addr learned my addr 172.21.15.2:0/3829978662 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:11.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.088+0000 7efeaffff700 1 --2- 172.21.15.2:0/3829978662 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efeb00ccd50 0x7efeb00cd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:11.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.088+0000 7efeae7fc700 1 -- 172.21.15.2:0/3829978662 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 793410053 0 0) 0x7efeb0071dc0 con 0x7efeb015a410 2023-08-25T03:35:11.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.088+0000 7efeae7fc700 1 -- 172.21.15.2:0/3829978662 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe9c0035b0 con 0x7efeb015a410 2023-08-25T03:35:11.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.088+0000 7efeae7fc700 1 -- 172.21.15.2:0/3829978662 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 98138347 0 0) 0x7efe9c0035b0 con 0x7efeb015a410 2023-08-25T03:35:11.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.088+0000 7efeae7fc700 1 -- 172.21.15.2:0/3829978662 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efeb00ccd50 msgr2=0x7efeb00cd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:11.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.088+0000 7efeae7fc700 1 --2- 172.21.15.2:0/3829978662 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efeb00ccd50 0x7efeb00cd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:11.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.088+0000 7efeae7fc700 1 -- 172.21.15.2:0/3829978662 >> v1:172.21.15.31:6789/0 conn(0x7efeb015a030 legacy=0x7efeb00cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:11.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.089+0000 7efeae7fc700 1 -- 172.21.15.2:0/3829978662 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb00cb4b0 con 0x7efeb015a410 2023-08-25T03:35:11.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.089+0000 7efeae7fc700 1 -- 172.21.15.2:0/3829978662 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7efe98001670 con 0x7efeb015a410 2023-08-25T03:35:11.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.089+0000 7efeae7fc700 1 -- 172.21.15.2:0/3829978662 >> v1:172.21.15.2:6789/0 conn(0x7efeb015a410 legacy=0x7efeb00ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:11.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.089+0000 7efeae7fc700 1 --2- 172.21.15.2:0/3829978662 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efeb00ccd50 0x7efe9c003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:11.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.089+0000 7efeae7fc700 1 --2- 172.21.15.2:0/3829978662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe9c003fb0 0x7efe9c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:11.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.089+0000 7efeae7fc700 1 --2- 172.21.15.2:0/3829978662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe9c004d90 0x7efe9c005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:11.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.089+0000 7efeae7fc700 1 -- 172.21.15.2:0/3829978662 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb00cd720 con 0x7efe9c004d90 2023-08-25T03:35:11.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.089+0000 7efeae7fc700 1 -- 172.21.15.2:0/3829978662 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c00ddb0 con 0x7efe9c003fb0 2023-08-25T03:35:11.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.089+0000 7efeae7fc700 1 -- 172.21.15.2:0/3829978662 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c00df70 con 0x7efeb00ccd50 2023-08-25T03:35:11.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.089+0000 7efeaffff700 1 --2- 172.21.15.2:0/3829978662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe9c004d90 0x7efe9c005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:11.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.089+0000 7efeaeffd700 1 --2- 172.21.15.2:0/3829978662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe9c003fb0 0x7efe9c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:11.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.089+0000 7efeaf7fe700 1 --2- 172.21.15.2:0/3829978662 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efeb00ccd50 0x7efe9c003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:11.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.089+0000 7efeaffff700 1 -- 172.21.15.2:0/3829978662 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efeb00ccd50 msgr2=0x7efe9c003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:11.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.089+0000 7efeaffff700 1 --2- 172.21.15.2:0/3829978662 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efeb00ccd50 0x7efe9c003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:11.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.089+0000 7efeaffff700 1 -- 172.21.15.2:0/3829978662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe9c003fb0 msgr2=0x7efe9c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:11.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.089+0000 7efeaffff700 1 --2- 172.21.15.2:0/3829978662 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe9c003fb0 0x7efe9c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:11.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.089+0000 7efeaffff700 1 -- 172.21.15.2:0/3829978662 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9c0035b0 con 0x7efe9c004d90 2023-08-25T03:35:11.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.089+0000 7efeaffff700 1 --2- 172.21.15.2:0/3829978662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe9c004d90 0x7efe9c005210 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efea8009fc0 tx=0x7efe9c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e8673377fe8d14 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:11.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.089+0000 7efeae7fc700 1 -- 172.21.15.2:0/3829978662 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea800ec30 con 0x7efe9c004d90 2023-08-25T03:35:11.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.089+0000 7efeae7fc700 1 -- 172.21.15.2:0/3829978662 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea800f040 con 0x7efe9c004d90 2023-08-25T03:35:11.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.089+0000 7efeae7fc700 1 -- 172.21.15.2:0/3829978662 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea8014480 con 0x7efe9c004d90 2023-08-25T03:35:11.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.090+0000 7efeb60c3700 1 -- 172.21.15.2:0/3829978662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe9c004d90 msgr2=0x7efe9c005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:11.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.090+0000 7efeb60c3700 1 --2- 172.21.15.2:0/3829978662 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe9c004d90 0x7efe9c005210 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7efea8009fc0 tx=0x7efe9c005880 comp rx=0 tx=0).stop 2023-08-25T03:35:11.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.090+0000 7efeaffff700 1 -- 172.21.15.2:0/3829978662 reap_dead start 2023-08-25T03:35:11.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.090+0000 7efeb60c3700 1 -- 172.21.15.2:0/3829978662 shutdown_connections 2023-08-25T03:35:11.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.090+0000 7efeb60c3700 1 -- 172.21.15.2:0/3829978662 >> 172.21.15.2:0/3829978662 conn(0x7efeb00b1cf0 msgr2=0x7efeb00b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:11.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.090+0000 7efeb60c3700 1 -- 172.21.15.2:0/3829978662 shutdown_connections 2023-08-25T03:35:11.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.090+0000 7efeb60c3700 1 -- 172.21.15.2:0/3829978662 wait complete. 2023-08-25T03:35:11.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.091+0000 7efeb60c3700 1 Processor -- start 2023-08-25T03:35:11.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.091+0000 7efeb60c3700 1 -- start start 2023-08-25T03:35:11.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.092+0000 7efeb60c3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe9c004d90 0x7efeb0151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:11.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.092+0000 7efeb60c3700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efeb015bc40 0x7efeb01ee430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:11.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.092+0000 7efeb60c3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efeb01f23f0 0x7efeb0154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:11.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.092+0000 7efeb60c3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb00b3530 con 0x7efe9c004d90 2023-08-25T03:35:11.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.092+0000 7efeb60c3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb006a680 con 0x7efeb01f23f0 2023-08-25T03:35:11.107 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.092+0000 7efeb60c3700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb01eea40 con 0x7efeb015bc40 2023-08-25T03:35:11.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.092+0000 7efeaf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe9c004d90 0x7efeb0151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:11.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.092+0000 7efeaf7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe9c004d90 0x7efeb0151fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33170/0 (socket says 172.21.15.2:33170) 2023-08-25T03:35:11.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.092+0000 7efeaf7fe700 1 -- 172.21.15.2:0/1216619129 learned_addr learned my addr 172.21.15.2:0/1216619129 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:11.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.092+0000 7efeaeffd700 1 --2- 172.21.15.2:0/1216619129 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efeb015bc40 0x7efeb01ee430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:11.108 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.092+0000 7efeaffff700 1 --2- 172.21.15.2:0/1216619129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efeb01f23f0 0x7efeb0154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:11.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.093+0000 7efeaf7fe700 1 -- 172.21.15.2:0/1216619129 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efeb015bc40 msgr2=0x7efeb01ee430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:11.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.093+0000 7efeaf7fe700 1 --2- 172.21.15.2:0/1216619129 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efeb015bc40 0x7efeb01ee430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:11.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.093+0000 7efeaf7fe700 1 -- 172.21.15.2:0/1216619129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efeb01f23f0 msgr2=0x7efeb0154f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:11.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.093+0000 7efeaf7fe700 1 --2- 172.21.15.2:0/1216619129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efeb01f23f0 0x7efeb0154f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:11.109 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.093+0000 7efeaf7fe700 1 -- 172.21.15.2:0/1216619129 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb01eed20 con 0x7efe9c004d90 2023-08-25T03:35:11.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.093+0000 7efeaf7fe700 1 --2- 172.21.15.2:0/1216619129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe9c004d90 0x7efeb0151fb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7efea000f7f0 tx=0x7efea000fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:11.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.093+0000 7efea7fff700 1 -- 172.21.15.2:0/1216619129 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea00052e0 con 0x7efe9c004d90 2023-08-25T03:35:11.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.093+0000 7efea7fff700 1 -- 172.21.15.2:0/1216619129 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea0005440 con 0x7efe9c004d90 2023-08-25T03:35:11.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.093+0000 7efeb60c3700 1 -- 172.21.15.2:0/1216619129 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb01ef000 con 0x7efe9c004d90 2023-08-25T03:35:11.110 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.093+0000 7efea7fff700 1 -- 172.21.15.2:0/1216619129 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea00038c0 con 0x7efe9c004d90 2023-08-25T03:35:11.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.093+0000 7efeb60c3700 1 -- 172.21.15.2:0/1216619129 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb01ef2e0 con 0x7efe9c004d90 2023-08-25T03:35:11.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.094+0000 7efeb60c3700 1 -- 172.21.15.2:0/1216619129 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb0066f70 con 0x7efe9c004d90 2023-08-25T03:35:11.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.094+0000 7efea7fff700 1 -- 172.21.15.2:0/1216619129 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7efea0003a20 con 0x7efe9c004d90 2023-08-25T03:35:11.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.095+0000 7efea7fff700 1 --2- 172.21.15.2:0/1216619129 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7efe9006a0e0 0x7efe9006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:11.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.095+0000 7efea7fff700 1 -- 172.21.15.2:0/1216619129 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7efea00881a0 con 0x7efe9c004d90 2023-08-25T03:35:11.111 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.095+0000 7efeaeffd700 1 --2- 172.21.15.2:0/1216619129 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7efe9006a0e0 0x7efe9006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:11.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.095+0000 7efeaeffd700 1 --2- 172.21.15.2:0/1216619129 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7efe9006a0e0 0x7efe9006c5a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efeb015c060 tx=0x7efe98000f80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:11.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.100+0000 7efea7fff700 1 -- 172.21.15.2:0/1216619129 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7efea0053920 con 0x7efe9c004d90 2023-08-25T03:35:11.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.343+0000 7efeb60c3700 1 -- 172.21.15.2:0/1216619129 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efeb00cb090 con 0x7efe9c004d90 2023-08-25T03:35:11.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.344+0000 7efea7fff700 1 -- 172.21.15.2:0/1216619129 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4869 (secure 0 0 0) 0x7efea0056f40 con 0x7efe9c004d90 2023-08-25T03:35:11.345 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:35:11.345 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4751,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/3802947908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3802947908},{"type":"v1","addr":"172.21.15.31:6834","nonce":3802947908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1470640897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6835","nonce":1470640897},{"type":"v1","addr":"172.21.15.2:6837","nonce":1470640897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6836/3706466654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3706466654},{"type":"v1","addr":"172.21.15.2:6836","nonce":3706466654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T03:35:07.333851+0000","modified":"2023-08-25T03:35:10.931788+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5229":{"gid":5229,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1140461626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1140461626},{"type":"v1","addr":"172.21.15.31:6835","nonce":1140461626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-25T03:35:11.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.346+0000 7efeb60c3700 1 -- 172.21.15.2:0/1216619129 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7efe9006a0e0 msgr2=0x7efe9006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:11.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.346+0000 7efeb60c3700 1 --2- 172.21.15.2:0/1216619129 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7efe9006a0e0 0x7efe9006c5a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7efeb015c060 tx=0x7efe98000f80 comp rx=0 tx=0).stop 2023-08-25T03:35:11.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.346+0000 7efeb60c3700 1 -- 172.21.15.2:0/1216619129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe9c004d90 msgr2=0x7efeb0151fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:11.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.346+0000 7efeb60c3700 1 --2- 172.21.15.2:0/1216619129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe9c004d90 0x7efeb0151fb0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7efea000f7f0 tx=0x7efea000fb00 comp rx=0 tx=0).stop 2023-08-25T03:35:11.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.346+0000 7efeb60c3700 1 -- 172.21.15.2:0/1216619129 shutdown_connections 2023-08-25T03:35:11.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.346+0000 7efeb60c3700 1 --2- 172.21.15.2:0/1216619129 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7efe9006a0e0 0x7efe9006c5a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:11.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.346+0000 7efeb60c3700 1 --2- 172.21.15.2:0/1216619129 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7efe9c004d90 0x7efeb0151fb0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:11.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.346+0000 7efeb60c3700 1 --2- 172.21.15.2:0/1216619129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efeb01f23f0 0x7efeb0154f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:11.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.346+0000 7efeb60c3700 1 --2- 172.21.15.2:0/1216619129 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efeb015bc40 0x7efeb01ee430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:11.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.346+0000 7efeb60c3700 1 -- 172.21.15.2:0/1216619129 >> 172.21.15.2:0/1216619129 conn(0x7efeb00b1cf0 msgr2=0x7efeb00afb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:11.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.346+0000 7efeb60c3700 1 -- 172.21.15.2:0/1216619129 shutdown_connections 2023-08-25T03:35:11.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.346+0000 7efeb60c3700 1 -- 172.21.15.2:0/1216619129 wait complete. 2023-08-25T03:35:11.353 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2023-08-25T03:35:11.359 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:35:11.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.483+0000 7febfe3e6700 1 Processor -- start 2023-08-25T03:35:11.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.484+0000 7febfe3e6700 1 -- start start 2023-08-25T03:35:11.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.484+0000 7febfe3e6700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7febf8157850 0x7febf80ceee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:11.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.484+0000 7febfe3e6700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf815dd80 con 0x7febf80cd7b0 2023-08-25T03:35:11.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.484+0000 7febfe3e6700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febf815e060 con 0x7febf806b900 2023-08-25T03:35:11.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.484+0000 7febfe3e6700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febf80b24e0 con 0x7febf8157850 2023-08-25T03:35:11.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.484+0000 7febf77fe700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7febf80cd7b0 0x7febf806b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48076/0 (socket says 172.21.15.2:48076) 2023-08-25T03:35:11.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.485+0000 7febf77fe700 1 -- 172.21.15.2:0/2873282480 learned_addr learned my addr 172.21.15.2:0/2873282480 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:11.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.485+0000 7febf7fff700 1 --2- 172.21.15.2:0/2873282480 >> v2:172.21.15.31:3300/0 conn(0x7febf8157850 0x7febf80ceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:11.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.485+0000 7febf6ffd700 1 -- 172.21.15.2:0/2873282480 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741481680 0 0) 0x7febf815dd80 con 0x7febf80cd7b0 2023-08-25T03:35:11.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.485+0000 7febf6ffd700 1 -- 172.21.15.2:0/2873282480 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febe00035b0 con 0x7febf80cd7b0 2023-08-25T03:35:11.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.485+0000 7febf6ffd700 1 -- 172.21.15.2:0/2873282480 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2049078754 0 0) 0x7febe00035b0 con 0x7febf80cd7b0 2023-08-25T03:35:11.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.485+0000 7febf6ffd700 1 -- 172.21.15.2:0/2873282480 >> v2:172.21.15.31:3300/0 conn(0x7febf8157850 msgr2=0x7febf80ceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:11.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.485+0000 7febf6ffd700 1 --2- 172.21.15.2:0/2873282480 >> v2:172.21.15.31:3300/0 conn(0x7febf8157850 0x7febf80ceee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:11.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.485+0000 7febf6ffd700 1 -- 172.21.15.2:0/2873282480 >> v1:172.21.15.31:6789/0 conn(0x7febf806b900 legacy=0x7febf80b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:11.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.485+0000 7febf6ffd700 1 -- 172.21.15.2:0/2873282480 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf815e340 con 0x7febf80cd7b0 2023-08-25T03:35:11.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.485+0000 7febf6ffd700 1 -- 172.21.15.2:0/2873282480 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7febe8001670 con 0x7febf80cd7b0 2023-08-25T03:35:11.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.485+0000 7febf6ffd700 1 -- 172.21.15.2:0/2873282480 >> v1:172.21.15.2:6789/0 conn(0x7febf80cd7b0 legacy=0x7febf806b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:11.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.485+0000 7febf6ffd700 1 --2- 172.21.15.2:0/2873282480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febe0003a00 0x7febe0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:11.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.485+0000 7febf6ffd700 1 --2- 172.21.15.2:0/2873282480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febe0004400 0x7febe00048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:11.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.485+0000 7febf6ffd700 1 --2- 172.21.15.2:0/2873282480 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7febe00050d0 0x7febe000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:11.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.485+0000 7febf6ffd700 1 -- 172.21.15.2:0/2873282480 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf80b24e0 con 0x7febe0004400 2023-08-25T03:35:11.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.485+0000 7febf6ffd700 1 -- 172.21.15.2:0/2873282480 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe000c030 con 0x7febe0003a00 2023-08-25T03:35:11.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.485+0000 7febf6ffd700 1 -- 172.21.15.2:0/2873282480 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe000c1f0 con 0x7febe00050d0 2023-08-25T03:35:11.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.485+0000 7febf77fe700 1 --2- 172.21.15.2:0/2873282480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febe0004400 0x7febe00048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:11.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.485+0000 7febf7fff700 1 --2- 172.21.15.2:0/2873282480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febe0003a00 0x7febe0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:11.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.485+0000 7febfc983700 1 --2- 172.21.15.2:0/2873282480 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7febe00050d0 0x7febe000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:11.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.486+0000 7febf77fe700 1 -- 172.21.15.2:0/2873282480 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7febe00050d0 msgr2=0x7febe000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:11.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.486+0000 7febf77fe700 1 --2- 172.21.15.2:0/2873282480 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7febe00050d0 0x7febe000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:11.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.486+0000 7febf77fe700 1 -- 172.21.15.2:0/2873282480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febe0003a00 msgr2=0x7febe0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:11.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.486+0000 7febf77fe700 1 --2- 172.21.15.2:0/2873282480 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febe0003a00 0x7febe0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:11.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.486+0000 7febfc983700 1 -- 172.21.15.2:0/2873282480 reap_dead start 2023-08-25T03:35:11.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.486+0000 7febf77fe700 1 -- 172.21.15.2:0/2873282480 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe00035b0 con 0x7febe0004400 2023-08-25T03:35:11.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.486+0000 7febf77fe700 1 --2- 172.21.15.2:0/2873282480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febe0004400 0x7febe00048d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7febe800c2f0 tx=0x7febe0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f15b6ea78eaef1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:11.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.486+0000 7febf6ffd700 1 -- 172.21.15.2:0/2873282480 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe800cdf0 con 0x7febe0004400 2023-08-25T03:35:11.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.486+0000 7febf6ffd700 1 -- 172.21.15.2:0/2873282480 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febe8005960 con 0x7febe0004400 2023-08-25T03:35:11.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.486+0000 7febf6ffd700 1 -- 172.21.15.2:0/2873282480 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe8005d10 con 0x7febe0004400 2023-08-25T03:35:11.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.486+0000 7febfe3e6700 1 -- 172.21.15.2:0/2873282480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febe0004400 msgr2=0x7febe00048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:11.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.486+0000 7febfe3e6700 1 --2- 172.21.15.2:0/2873282480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febe0004400 0x7febe00048d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7febe800c2f0 tx=0x7febe0005880 comp rx=0 tx=0).stop 2023-08-25T03:35:11.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.487+0000 7febfe3e6700 1 -- 172.21.15.2:0/2873282480 shutdown_connections 2023-08-25T03:35:11.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.487+0000 7febfe3e6700 1 --2- 172.21.15.2:0/2873282480 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febe0004400 0x7febe00048d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:11.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.487+0000 7febfe3e6700 1 -- 172.21.15.2:0/2873282480 >> 172.21.15.2:0/2873282480 conn(0x7febf80cfd10 msgr2=0x7febf806ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:11.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.487+0000 7febfe3e6700 1 -- 172.21.15.2:0/2873282480 shutdown_connections 2023-08-25T03:35:11.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.487+0000 7febfe3e6700 1 -- 172.21.15.2:0/2873282480 wait complete. 2023-08-25T03:35:11.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.488+0000 7febfe3e6700 1 Processor -- start 2023-08-25T03:35:11.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.488+0000 7febfe3e6700 1 -- start start 2023-08-25T03:35:11.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.489+0000 7febfe3e6700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febe0004400 0x7febf8157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:11.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.489+0000 7febfe3e6700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febf80cc560 0x7febf815fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:11.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.489+0000 7febfe3e6700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7febf8152020 0x7febf81524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:11.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.489+0000 7febfe3e6700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf81585e0 con 0x7febe0004400 2023-08-25T03:35:11.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.489+0000 7febfe3e6700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febf80c8860 con 0x7febf80cc560 2023-08-25T03:35:11.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.489+0000 7febfe3e6700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febf8152a10 con 0x7febf8152020 2023-08-25T03:35:11.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.489+0000 7febf7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febe0004400 0x7febf8157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:11.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.489+0000 7febf7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febe0004400 0x7febf8157db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33190/0 (socket says 172.21.15.2:33190) 2023-08-25T03:35:11.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.489+0000 7febf7fff700 1 -- 172.21.15.2:0/1602206565 learned_addr learned my addr 172.21.15.2:0/1602206565 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:11.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.489+0000 7febf77fe700 1 --2- 172.21.15.2:0/1602206565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febf80cc560 0x7febf815fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:11.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.489+0000 7febfc983700 1 --2- 172.21.15.2:0/1602206565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7febf8152020 0x7febf81524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:11.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.489+0000 7febf7fff700 1 -- 172.21.15.2:0/1602206565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7febf8152020 msgr2=0x7febf81524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:11.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.489+0000 7febf7fff700 1 --2- 172.21.15.2:0/1602206565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7febf8152020 0x7febf81524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:11.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.489+0000 7febf7fff700 1 -- 172.21.15.2:0/1602206565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febf80cc560 msgr2=0x7febf815fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:11.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.489+0000 7febf7fff700 1 --2- 172.21.15.2:0/1602206565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febf80cc560 0x7febf815fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:11.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.489+0000 7febf7fff700 1 -- 172.21.15.2:0/1602206565 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf80c8a70 con 0x7febe0004400 2023-08-25T03:35:11.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.489+0000 7febf7fff700 1 --2- 172.21.15.2:0/1602206565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febe0004400 0x7febf8157db0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7febec011aa0 tx=0x7febec011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:11.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.489+0000 7febf4ff9700 1 -- 172.21.15.2:0/1602206565 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec00f980 con 0x7febe0004400 2023-08-25T03:35:11.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.489+0000 7febf4ff9700 1 -- 172.21.15.2:0/1602206565 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febec00fb00 con 0x7febe0004400 2023-08-25T03:35:11.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.489+0000 7febfe3e6700 1 -- 172.21.15.2:0/1602206565 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febf80c8d50 con 0x7febe0004400 2023-08-25T03:35:11.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.489+0000 7febf4ff9700 1 -- 172.21.15.2:0/1602206565 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febec00feb0 con 0x7febe0004400 2023-08-25T03:35:11.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.490+0000 7febfe3e6700 1 -- 172.21.15.2:0/1602206565 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febf80c9310 con 0x7febe0004400 2023-08-25T03:35:11.498 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.491+0000 7febfe3e6700 1 -- 172.21.15.2:0/1602206565 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febf8066650 con 0x7febe0004400 2023-08-25T03:35:11.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.491+0000 7febf4ff9700 1 -- 172.21.15.2:0/1602206565 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7febec014440 con 0x7febe0004400 2023-08-25T03:35:11.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.491+0000 7febf4ff9700 1 --2- 172.21.15.2:0/1602206565 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7febe406a0e0 0x7febe406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:11.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.491+0000 7febf4ff9700 1 -- 172.21.15.2:0/1602206565 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7febec088cf0 con 0x7febe0004400 2023-08-25T03:35:11.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.491+0000 7febf77fe700 1 --2- 172.21.15.2:0/1602206565 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7febe406a0e0 0x7febe406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:11.499 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.492+0000 7febf77fe700 1 --2- 172.21.15.2:0/1602206565 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7febe406a0e0 0x7febe406c5a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7febe8001930 tx=0x7febe8007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:11.500 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.496+0000 7febf4ff9700 1 -- 172.21.15.2:0/1602206565 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7febec08d070 con 0x7febe0004400 2023-08-25T03:35:11.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.699+0000 7febfe3e6700 1 -- 172.21.15.2:0/1602206565 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7febf806ee40 con 0x7febe0004400 2023-08-25T03:35:11.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.701+0000 7febf4ff9700 1 -- 172.21.15.2:0/1602206565 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18207 (secure 0 0 0) 0x7febec057fc0 con 0x7febe0004400 2023-08-25T03:35:11.702 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:35:11.702 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":37,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:35:08.407381+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:35:05.870790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-08-25T03:35:06.296959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:35:11.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.703+0000 7febfe3e6700 1 -- 172.21.15.2:0/1602206565 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7febe406a0e0 msgr2=0x7febe406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:11.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.703+0000 7febfe3e6700 1 --2- 172.21.15.2:0/1602206565 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7febe406a0e0 0x7febe406c5a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7febe8001930 tx=0x7febe8007000 comp rx=0 tx=0).stop 2023-08-25T03:35:11.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.703+0000 7febfe3e6700 1 -- 172.21.15.2:0/1602206565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febe0004400 msgr2=0x7febf8157db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:11.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.703+0000 7febfe3e6700 1 --2- 172.21.15.2:0/1602206565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febe0004400 0x7febf8157db0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7febec011aa0 tx=0x7febec011e60 comp rx=0 tx=0).stop 2023-08-25T03:35:11.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.703+0000 7febfe3e6700 1 -- 172.21.15.2:0/1602206565 shutdown_connections 2023-08-25T03:35:11.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.703+0000 7febfe3e6700 1 --2- 172.21.15.2:0/1602206565 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7febe406a0e0 0x7febe406c5a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:11.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.703+0000 7febfe3e6700 1 --2- 172.21.15.2:0/1602206565 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7febe0004400 0x7febf8157db0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:11.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.703+0000 7febfe3e6700 1 --2- 172.21.15.2:0/1602206565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7febf8152020 0x7febf81524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:11.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.703+0000 7febfe3e6700 1 --2- 172.21.15.2:0/1602206565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7febf80cc560 0x7febf815fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:11.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.703+0000 7febfe3e6700 1 -- 172.21.15.2:0/1602206565 >> 172.21.15.2:0/1602206565 conn(0x7febf80cfd10 msgr2=0x7febf806a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:11.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.703+0000 7febfe3e6700 1 -- 172.21.15.2:0/1602206565 shutdown_connections 2023-08-25T03:35:11.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.703+0000 7febfe3e6700 1 -- 172.21.15.2:0/1602206565 wait complete. 2023-08-25T03:35:11.717 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-08-25T03:35:11.718 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T03:35:11.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.843+0000 7fc391188700 1 Processor -- start 2023-08-25T03:35:11.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.843+0000 7fc391188700 1 -- start start 2023-08-25T03:35:11.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.844+0000 7fc391188700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fc38c15a030 0x7fc38c0cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:11.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.844+0000 7fc391188700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc38c071dc0 con 0x7fc38c15a410 2023-08-25T03:35:11.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.844+0000 7fc391188700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc38c0720a0 con 0x7fc38c0cc1f0 2023-08-25T03:35:11.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.844+0000 7fc391188700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc38c0cb4b0 con 0x7fc38c15a030 2023-08-25T03:35:11.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.844+0000 7fc38ad9d700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fc38c15a030 0x7fc38c0cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:11.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.844+0000 7fc38b59e700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fc38c15a410 0x7fc38c0ccd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48100/0 (socket says 172.21.15.2:48100) 2023-08-25T03:35:11.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.844+0000 7fc38b59e700 1 -- 172.21.15.2:0/51290733 learned_addr learned my addr 172.21.15.2:0/51290733 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:11.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.845+0000 7fc38ad9d700 1 -- 172.21.15.2:0/51290733 >> v1:172.21.15.31:6789/0 conn(0x7fc38c0cc1f0 legacy=0x7fc38c0ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:11.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.845+0000 7fc38ad9d700 1 -- 172.21.15.2:0/51290733 >> v1:172.21.15.2:6789/0 conn(0x7fc38c15a410 legacy=0x7fc38c0ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:11.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.845+0000 7fc389d9b700 1 -- 172.21.15.2:0/51290733 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1865396708 0 0) 0x7fc38c071dc0 con 0x7fc38c15a410 2023-08-25T03:35:11.849 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.845+0000 7fc38ad9d700 1 -- 172.21.15.2:0/51290733 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38c0cb790 con 0x7fc38c15a030 2023-08-25T03:35:11.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.845+0000 7fc38ad9d700 1 --2- 172.21.15.2:0/51290733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc38c15a030 0x7fc38c0cbcb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc380009840 tx=0x7fc380009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23a70d834a3498bc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:11.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.845+0000 7fc389d9b700 1 -- 172.21.15.2:0/51290733 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380004230 con 0x7fc38c15a030 2023-08-25T03:35:11.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.845+0000 7fc389d9b700 1 -- 172.21.15.2:0/51290733 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc380004390 con 0x7fc38c15a030 2023-08-25T03:35:11.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.845+0000 7fc389d9b700 1 -- 172.21.15.2:0/51290733 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3800036a0 con 0x7fc38c15a030 2023-08-25T03:35:11.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.845+0000 7fc391188700 1 -- 172.21.15.2:0/51290733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc38c15a030 msgr2=0x7fc38c0cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:11.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.845+0000 7fc391188700 1 --2- 172.21.15.2:0/51290733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc38c15a030 0x7fc38c0cbcb0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc380009840 tx=0x7fc380009b50 comp rx=0 tx=0).stop 2023-08-25T03:35:11.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.845+0000 7fc391188700 1 -- 172.21.15.2:0/51290733 shutdown_connections 2023-08-25T03:35:11.850 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.845+0000 7fc391188700 1 --2- 172.21.15.2:0/51290733 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc38c15a030 0x7fc38c0cbcb0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:11.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.845+0000 7fc391188700 1 -- 172.21.15.2:0/51290733 >> 172.21.15.2:0/51290733 conn(0x7fc38c0b1cf0 msgr2=0x7fc38c0b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:11.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.845+0000 7fc391188700 1 -- 172.21.15.2:0/51290733 shutdown_connections 2023-08-25T03:35:11.851 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.845+0000 7fc391188700 1 -- 172.21.15.2:0/51290733 wait complete. 2023-08-25T03:35:11.853 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.847+0000 7fc391188700 1 Processor -- start 2023-08-25T03:35:11.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.847+0000 7fc391188700 1 -- start start 2023-08-25T03:35:11.854 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.848+0000 7fc391188700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc38c0cc1f0 0x7fc38c0c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:11.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.848+0000 7fc391188700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc38c15a030 0x7fc38c1562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:11.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.848+0000 7fc391188700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc38c15a410 0x7fc38c0b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:11.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.848+0000 7fc391188700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c0b2c00 con 0x7fc38c0cc1f0 2023-08-25T03:35:11.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.848+0000 7fc391188700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c0b34a0 con 0x7fc38c15a030 2023-08-25T03:35:11.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.848+0000 7fc391188700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc38c0ca180 con 0x7fc38c15a410 2023-08-25T03:35:11.855 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.848+0000 7fc38ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc38c0cc1f0 0x7fc38c0c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:11.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.849+0000 7fc38ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc38c0cc1f0 0x7fc38c0c9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33204/0 (socket says 172.21.15.2:33204) 2023-08-25T03:35:11.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.849+0000 7fc38ad9d700 1 -- 172.21.15.2:0/2171568183 learned_addr learned my addr 172.21.15.2:0/2171568183 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:11.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.849+0000 7fc38a59c700 1 --2- 172.21.15.2:0/2171568183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc38c15a030 0x7fc38c1562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:11.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.849+0000 7fc38b59e700 1 --2- 172.21.15.2:0/2171568183 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc38c15a410 0x7fc38c0b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:11.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.849+0000 7fc38ad9d700 1 -- 172.21.15.2:0/2171568183 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc38c15a410 msgr2=0x7fc38c0b12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:11.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.849+0000 7fc38ad9d700 1 --2- 172.21.15.2:0/2171568183 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc38c15a410 0x7fc38c0b12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:11.856 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.849+0000 7fc38ad9d700 1 -- 172.21.15.2:0/2171568183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc38c15a030 msgr2=0x7fc38c1562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:11.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.849+0000 7fc38ad9d700 1 --2- 172.21.15.2:0/2171568183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc38c15a030 0x7fc38c1562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:11.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.849+0000 7fc38ad9d700 1 -- 172.21.15.2:0/2171568183 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3800094a0 con 0x7fc38c0cc1f0 2023-08-25T03:35:11.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.849+0000 7fc38b59e700 1 --2- 172.21.15.2:0/2171568183 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc38c15a410 0x7fc38c0b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:35:11.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.849+0000 7fc38ad9d700 1 --2- 172.21.15.2:0/2171568183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc38c0cc1f0 0x7fc38c0c9ba0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc3800090b0 tx=0x7fc380004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:11.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.849+0000 7fc3737fe700 1 -- 172.21.15.2:0/2171568183 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380011830 con 0x7fc38c0cc1f0 2023-08-25T03:35:11.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.849+0000 7fc3737fe700 1 -- 172.21.15.2:0/2171568183 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38001e070 con 0x7fc38c0cc1f0 2023-08-25T03:35:11.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.849+0000 7fc391188700 1 -- 172.21.15.2:0/2171568183 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc38c0b2e50 con 0x7fc38c0cc1f0 2023-08-25T03:35:11.857 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.849+0000 7fc3737fe700 1 -- 172.21.15.2:0/2171568183 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc380011bb0 con 0x7fc38c0cc1f0 2023-08-25T03:35:11.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.849+0000 7fc391188700 1 -- 172.21.15.2:0/2171568183 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc38c0b17f0 con 0x7fc38c0cc1f0 2023-08-25T03:35:11.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.850+0000 7fc391188700 1 -- 172.21.15.2:0/2171568183 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc38c066f70 con 0x7fc38c0cc1f0 2023-08-25T03:35:11.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.851+0000 7fc3737fe700 1 -- 172.21.15.2:0/2171568183 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc380011d10 con 0x7fc38c0cc1f0 2023-08-25T03:35:11.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.851+0000 7fc3737fe700 1 --2- 172.21.15.2:0/2171568183 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc37806a0e0 0x7fc37806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:11.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.851+0000 7fc3737fe700 1 -- 172.21.15.2:0/2171568183 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc3800898e0 con 0x7fc38c0cc1f0 2023-08-25T03:35:11.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.851+0000 7fc38a59c700 1 --2- 172.21.15.2:0/2171568183 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc37806a0e0 0x7fc37806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:11.858 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.852+0000 7fc38a59c700 1 --2- 172.21.15.2:0/2171568183 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc37806a0e0 0x7fc37806c5a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc38c0b0f50 tx=0x7fc37c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:11.859 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:11.856+0000 7fc3737fe700 1 -- 172.21.15.2:0/2171568183 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc380055060 con 0x7fc38c0cc1f0 2023-08-25T03:35:11.896 INFO:tasks.ceph.mds.d.smithi031.stderr:2023-08-25T03:35:11.896+0000 7fb4142a2700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-25T03:35:12.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.052+0000 7fc391188700 1 -- 172.21.15.2:0/2171568183 --> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc38c0ce880 con 0x7fc37806a0e0 2023-08-25T03:35:12.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.061+0000 7fc3737fe700 1 -- 172.21.15.2:0/2171568183 <== mgr.4108 v2:172.21.15.2:6832/3528779779 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311363 (secure 0 0 0) 0x7fc37c009510 con 0x7fc37806a0e0 2023-08-25T03:35:12.066 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:35:12.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.064+0000 7fc391188700 1 -- 172.21.15.2:0/2171568183 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc37806a0e0 msgr2=0x7fc37806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:12.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.064+0000 7fc391188700 1 --2- 172.21.15.2:0/2171568183 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc37806a0e0 0x7fc37806c5a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc38c0b0f50 tx=0x7fc37c008040 comp rx=0 tx=0).stop 2023-08-25T03:35:12.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.064+0000 7fc391188700 1 -- 172.21.15.2:0/2171568183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc38c0cc1f0 msgr2=0x7fc38c0c9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:12.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.064+0000 7fc391188700 1 --2- 172.21.15.2:0/2171568183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc38c0cc1f0 0x7fc38c0c9ba0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fc3800090b0 tx=0x7fc380004750 comp rx=0 tx=0).stop 2023-08-25T03:35:12.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.065+0000 7fc391188700 1 -- 172.21.15.2:0/2171568183 shutdown_connections 2023-08-25T03:35:12.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.065+0000 7fc391188700 1 --2- 172.21.15.2:0/2171568183 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc37806a0e0 0x7fc37806c5a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:12.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.065+0000 7fc391188700 1 --2- 172.21.15.2:0/2171568183 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc38c0cc1f0 0x7fc38c0c9ba0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:12.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.065+0000 7fc391188700 1 --2- 172.21.15.2:0/2171568183 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc38c15a410 0x7fc38c0b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:12.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.065+0000 7fc391188700 1 --2- 172.21.15.2:0/2171568183 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc38c15a030 0x7fc38c1562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:12.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.065+0000 7fc391188700 1 -- 172.21.15.2:0/2171568183 >> 172.21.15.2:0/2171568183 conn(0x7fc38c0b1cf0 msgr2=0x7fc38c159380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:12.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.065+0000 7fc391188700 1 -- 172.21.15.2:0/2171568183 shutdown_connections 2023-08-25T03:35:12.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.065+0000 7fc391188700 1 -- 172.21.15.2:0/2171568183 wait complete. 2023-08-25T03:35:12.070 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-08-25T03:35:12.078 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2023-08-25T03:35:11.518330+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592708,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":201,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220584,"kb_used_data":4408,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754754136,"statfs":{"total":773094113280,"available":772868235264,"internally_reserved":0,"allocated":4513792,"data_stored":2531227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2340,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.617802"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430662+0000","last_change":"2023-08-25T03:35:07.427014+0000","last_active":"2023-08-25T03:35:07.430662+0000","last_peered":"2023-08-25T03:35:07.430662+0000","last_clean":"2023-08-25T03:35:07.430662+0000","last_became_active":"2023-08-25T03:35:07.426372+0000","last_became_peered":"2023-08-25T03:35:07.426372+0000","last_unstale":"2023-08-25T03:35:07.430662+0000","last_undegraded":"2023-08-25T03:35:07.430662+0000","last_fullsized":"2023-08-25T03:35:07.430662+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:12:41.463967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.426648+0000","last_change":"2023-08-25T03:35:07.425122+0000","last_active":"2023-08-25T03:35:07.426648+0000","last_peered":"2023-08-25T03:35:07.426648+0000","last_clean":"2023-08-25T03:35:07.426648+0000","last_became_active":"2023-08-25T03:35:07.424562+0000","last_became_peered":"2023-08-25T03:35:07.424562+0000","last_unstale":"2023-08-25T03:35:07.426648+0000","last_undegraded":"2023-08-25T03:35:07.426648+0000","last_fullsized":"2023-08-25T03:35:07.426648+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:37:27.118343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430322+0000","last_change":"2023-08-25T03:35:07.427639+0000","last_active":"2023-08-25T03:35:07.430322+0000","last_peered":"2023-08-25T03:35:07.430322+0000","last_clean":"2023-08-25T03:35:07.430322+0000","last_became_active":"2023-08-25T03:35:07.427398+0000","last_became_peered":"2023-08-25T03:35:07.427398+0000","last_unstale":"2023-08-25T03:35:07.430322+0000","last_undegraded":"2023-08-25T03:35:07.430322+0000","last_fullsized":"2023-08-25T03:35:07.430322+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:41:58.994138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430603+0000","last_change":"2023-08-25T03:35:07.428463+0000","last_active":"2023-08-25T03:35:07.430603+0000","last_peered":"2023-08-25T03:35:07.430603+0000","last_clean":"2023-08-25T03:35:07.430603+0000","last_became_active":"2023-08-25T03:35:07.428033+0000","last_became_peered":"2023-08-25T03:35:07.428033+0000","last_unstale":"2023-08-25T03:35:07.430603+0000","last_undegraded":"2023-08-25T03:35:07.430603+0000","last_fullsized":"2023-08-25T03:35:07.430603+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:39:17.858453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.422489+0000","last_change":"2023-08-25T03:35:07.421403+0000","last_active":"2023-08-25T03:35:07.422489+0000","last_peered":"2023-08-25T03:35:07.422489+0000","last_clean":"2023-08-25T03:35:07.422489+0000","last_became_active":"2023-08-25T03:35:07.421108+0000","last_became_peered":"2023-08-25T03:35:07.421108+0000","last_unstale":"2023-08-25T03:35:07.422489+0000","last_undegraded":"2023-08-25T03:35:07.422489+0000","last_fullsized":"2023-08-25T03:35:07.422489+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T03:54:02.006558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430320+0000","last_change":"2023-08-25T03:35:07.425989+0000","last_active":"2023-08-25T03:35:07.430320+0000","last_peered":"2023-08-25T03:35:07.430320+0000","last_clean":"2023-08-25T03:35:07.430320+0000","last_became_active":"2023-08-25T03:35:07.425684+0000","last_became_peered":"2023-08-25T03:35:07.425684+0000","last_unstale":"2023-08-25T03:35:07.430320+0000","last_undegraded":"2023-08-25T03:35:07.430320+0000","last_fullsized":"2023-08-25T03:35:07.430320+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:58:52.102133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.416357+0000","last_change":"2023-08-25T03:35:07.429613+0000","last_active":"2023-08-25T03:35:08.416357+0000","last_peered":"2023-08-25T03:35:08.416357+0000","last_clean":"2023-08-25T03:35:08.416357+0000","last_became_active":"2023-08-25T03:35:07.429299+0000","last_became_peered":"2023-08-25T03:35:07.429299+0000","last_unstale":"2023-08-25T03:35:08.416357+0000","last_undegraded":"2023-08-25T03:35:08.416357+0000","last_fullsized":"2023-08-25T03:35:08.416357+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:56:34.245129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430474+0000","last_change":"2023-08-25T03:35:07.425856+0000","last_active":"2023-08-25T03:35:07.430474+0000","last_peered":"2023-08-25T03:35:07.430474+0000","last_clean":"2023-08-25T03:35:07.430474+0000","last_became_active":"2023-08-25T03:35:07.425510+0000","last_became_peered":"2023-08-25T03:35:07.425510+0000","last_unstale":"2023-08-25T03:35:07.430474+0000","last_undegraded":"2023-08-25T03:35:07.430474+0000","last_fullsized":"2023-08-25T03:35:07.430474+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:40:02.227089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430267+0000","last_change":"2023-08-25T03:35:07.427338+0000","last_active":"2023-08-25T03:35:07.430267+0000","last_peered":"2023-08-25T03:35:07.430267+0000","last_clean":"2023-08-25T03:35:07.430267+0000","last_became_active":"2023-08-25T03:35:07.427025+0000","last_became_peered":"2023-08-25T03:35:07.427025+0000","last_unstale":"2023-08-25T03:35:07.430267+0000","last_undegraded":"2023-08-25T03:35:07.430267+0000","last_fullsized":"2023-08-25T03:35:07.430267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:35:53.452778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417787+0000","last_change":"2023-08-25T03:35:07.429618+0000","last_active":"2023-08-25T03:35:08.417787+0000","last_peered":"2023-08-25T03:35:08.417787+0000","last_clean":"2023-08-25T03:35:08.417787+0000","last_became_active":"2023-08-25T03:35:07.429255+0000","last_became_peered":"2023-08-25T03:35:07.429255+0000","last_unstale":"2023-08-25T03:35:08.417787+0000","last_undegraded":"2023-08-25T03:35:08.417787+0000","last_fullsized":"2023-08-25T03:35:08.417787+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:32:48.908546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417769+0000","last_change":"2023-08-25T03:35:07.422337+0000","last_active":"2023-08-25T03:35:08.417769+0000","last_peered":"2023-08-25T03:35:08.417769+0000","last_clean":"2023-08-25T03:35:08.417769+0000","last_became_active":"2023-08-25T03:35:07.421937+0000","last_became_peered":"2023-08-25T03:35:07.421937+0000","last_unstale":"2023-08-25T03:35:08.417769+0000","last_undegraded":"2023-08-25T03:35:08.417769+0000","last_fullsized":"2023-08-25T03:35:08.417769+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:58:47.877253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.419362+0000","last_change":"2023-08-25T03:35:07.417162+0000","last_active":"2023-08-25T03:35:07.419362+0000","last_peered":"2023-08-25T03:35:07.419362+0000","last_clean":"2023-08-25T03:35:07.419362+0000","last_became_active":"2023-08-25T03:35:07.416924+0000","last_became_peered":"2023-08-25T03:35:07.416924+0000","last_unstale":"2023-08-25T03:35:07.419362+0000","last_undegraded":"2023-08-25T03:35:07.419362+0000","last_fullsized":"2023-08-25T03:35:07.419362+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:08:24.760786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.416315+0000","last_change":"2023-08-25T03:35:07.426563+0000","last_active":"2023-08-25T03:35:08.416315+0000","last_peered":"2023-08-25T03:35:08.416315+0000","last_clean":"2023-08-25T03:35:08.416315+0000","last_became_active":"2023-08-25T03:35:07.426176+0000","last_became_peered":"2023-08-25T03:35:07.426176+0000","last_unstale":"2023-08-25T03:35:08.416315+0000","last_undegraded":"2023-08-25T03:35:08.416315+0000","last_fullsized":"2023-08-25T03:35:08.416315+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:23:41.853641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.876115+0000","last_change":"2023-08-25T03:35:07.428844+0000","last_active":"2023-08-25T03:35:08.876115+0000","last_peered":"2023-08-25T03:35:08.876115+0000","last_clean":"2023-08-25T03:35:08.876115+0000","last_became_active":"2023-08-25T03:35:07.428140+0000","last_became_peered":"2023-08-25T03:35:07.428140+0000","last_unstale":"2023-08-25T03:35:08.876115+0000","last_undegraded":"2023-08-25T03:35:08.876115+0000","last_fullsized":"2023-08-25T03:35:08.876115+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:25:51.954228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.416549+0000","last_change":"2023-08-25T03:35:07.417780+0000","last_active":"2023-08-25T03:35:08.416549+0000","last_peered":"2023-08-25T03:35:08.416549+0000","last_clean":"2023-08-25T03:35:08.416549+0000","last_became_active":"2023-08-25T03:35:07.417133+0000","last_became_peered":"2023-08-25T03:35:07.417133+0000","last_unstale":"2023-08-25T03:35:08.416549+0000","last_undegraded":"2023-08-25T03:35:08.416549+0000","last_fullsized":"2023-08-25T03:35:08.416549+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:25:51.954228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.437799+0000","last_change":"2023-08-25T03:35:07.433438+0000","last_active":"2023-08-25T03:35:07.437799+0000","last_peered":"2023-08-25T03:35:07.437799+0000","last_clean":"2023-08-25T03:35:07.437799+0000","last_became_active":"2023-08-25T03:35:07.433155+0000","last_became_peered":"2023-08-25T03:35:07.433155+0000","last_unstale":"2023-08-25T03:35:07.437799+0000","last_undegraded":"2023-08-25T03:35:07.437799+0000","last_fullsized":"2023-08-25T03:35:07.437799+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:54:18.712932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.440381+0000","last_change":"2023-08-25T03:35:07.435773+0000","last_active":"2023-08-25T03:35:07.440381+0000","last_peered":"2023-08-25T03:35:07.440381+0000","last_clean":"2023-08-25T03:35:07.440381+0000","last_became_active":"2023-08-25T03:35:07.435473+0000","last_became_peered":"2023-08-25T03:35:07.435473+0000","last_unstale":"2023-08-25T03:35:07.440381+0000","last_undegraded":"2023-08-25T03:35:07.440381+0000","last_fullsized":"2023-08-25T03:35:07.440381+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:38:14.586034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417469+0000","last_change":"2023-08-25T03:35:07.426305+0000","last_active":"2023-08-25T03:35:08.417469+0000","last_peered":"2023-08-25T03:35:08.417469+0000","last_clean":"2023-08-25T03:35:08.417469+0000","last_became_active":"2023-08-25T03:35:07.425265+0000","last_became_peered":"2023-08-25T03:35:07.425265+0000","last_unstale":"2023-08-25T03:35:08.417469+0000","last_undegraded":"2023-08-25T03:35:08.417469+0000","last_fullsized":"2023-08-25T03:35:08.417469+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:24:45.526607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.418554+0000","last_change":"2023-08-25T03:35:07.416345+0000","last_active":"2023-08-25T03:35:07.418554+0000","last_peered":"2023-08-25T03:35:07.418554+0000","last_clean":"2023-08-25T03:35:07.418554+0000","last_became_active":"2023-08-25T03:35:07.416085+0000","last_became_peered":"2023-08-25T03:35:07.416085+0000","last_unstale":"2023-08-25T03:35:07.418554+0000","last_undegraded":"2023-08-25T03:35:07.418554+0000","last_fullsized":"2023-08-25T03:35:07.418554+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:52:49.107239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.416708+0000","last_change":"2023-08-25T03:35:07.428310+0000","last_active":"2023-08-25T03:35:08.416708+0000","last_peered":"2023-08-25T03:35:08.416708+0000","last_clean":"2023-08-25T03:35:08.416708+0000","last_became_active":"2023-08-25T03:35:07.427915+0000","last_became_peered":"2023-08-25T03:35:07.427915+0000","last_unstale":"2023-08-25T03:35:08.416708+0000","last_undegraded":"2023-08-25T03:35:08.416708+0000","last_fullsized":"2023-08-25T03:35:08.416708+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:15:17.522640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417177+0000","last_change":"2023-08-25T03:35:07.418389+0000","last_active":"2023-08-25T03:35:08.417177+0000","last_peered":"2023-08-25T03:35:08.417177+0000","last_clean":"2023-08-25T03:35:08.417177+0000","last_became_active":"2023-08-25T03:35:07.416998+0000","last_became_peered":"2023-08-25T03:35:07.416998+0000","last_unstale":"2023-08-25T03:35:08.417177+0000","last_undegraded":"2023-08-25T03:35:08.417177+0000","last_fullsized":"2023-08-25T03:35:08.417177+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:48:29.838778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.876410+0000","last_change":"2023-08-25T03:35:07.422028+0000","last_active":"2023-08-25T03:35:08.876410+0000","last_peered":"2023-08-25T03:35:08.876410+0000","last_clean":"2023-08-25T03:35:08.876410+0000","last_became_active":"2023-08-25T03:35:07.421536+0000","last_became_peered":"2023-08-25T03:35:07.421536+0000","last_unstale":"2023-08-25T03:35:08.876410+0000","last_undegraded":"2023-08-25T03:35:08.876410+0000","last_fullsized":"2023-08-25T03:35:08.876410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:47:23.984917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.435316+0000","last_change":"2023-08-25T03:35:07.430813+0000","last_active":"2023-08-25T03:35:07.435316+0000","last_peered":"2023-08-25T03:35:07.435316+0000","last_clean":"2023-08-25T03:35:07.435316+0000","last_became_active":"2023-08-25T03:35:07.430512+0000","last_became_peered":"2023-08-25T03:35:07.430512+0000","last_unstale":"2023-08-25T03:35:07.435316+0000","last_undegraded":"2023-08-25T03:35:07.435316+0000","last_fullsized":"2023-08-25T03:35:07.435316+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:10:43.180614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.418527+0000","last_change":"2023-08-25T03:35:07.416291+0000","last_active":"2023-08-25T03:35:07.418527+0000","last_peered":"2023-08-25T03:35:07.418527+0000","last_clean":"2023-08-25T03:35:07.418527+0000","last_became_active":"2023-08-25T03:35:07.416004+0000","last_became_peered":"2023-08-25T03:35:07.416004+0000","last_unstale":"2023-08-25T03:35:07.418527+0000","last_undegraded":"2023-08-25T03:35:07.418527+0000","last_fullsized":"2023-08-25T03:35:07.418527+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:10:41.985896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417635+0000","last_change":"2023-08-25T03:35:07.420875+0000","last_active":"2023-08-25T03:35:08.417635+0000","last_peered":"2023-08-25T03:35:08.417635+0000","last_clean":"2023-08-25T03:35:08.417635+0000","last_became_active":"2023-08-25T03:35:07.420415+0000","last_became_peered":"2023-08-25T03:35:07.420415+0000","last_unstale":"2023-08-25T03:35:08.417635+0000","last_undegraded":"2023-08-25T03:35:08.417635+0000","last_fullsized":"2023-08-25T03:35:08.417635+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:04:02.903147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.420453+0000","last_change":"2023-08-25T03:35:07.419448+0000","last_active":"2023-08-25T03:35:07.420453+0000","last_peered":"2023-08-25T03:35:07.420453+0000","last_clean":"2023-08-25T03:35:07.420453+0000","last_became_active":"2023-08-25T03:35:07.419122+0000","last_became_peered":"2023-08-25T03:35:07.419122+0000","last_unstale":"2023-08-25T03:35:07.420453+0000","last_undegraded":"2023-08-25T03:35:07.420453+0000","last_fullsized":"2023-08-25T03:35:07.420453+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:35:53.452778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417200+0000","last_change":"2023-08-25T03:35:07.421488+0000","last_active":"2023-08-25T03:35:08.417200+0000","last_peered":"2023-08-25T03:35:08.417200+0000","last_clean":"2023-08-25T03:35:08.417200+0000","last_became_active":"2023-08-25T03:35:07.419932+0000","last_became_peered":"2023-08-25T03:35:07.419932+0000","last_unstale":"2023-08-25T03:35:08.417200+0000","last_undegraded":"2023-08-25T03:35:08.417200+0000","last_fullsized":"2023-08-25T03:35:08.417200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:18:15.305693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417142+0000","last_change":"2023-08-25T03:35:07.426011+0000","last_active":"2023-08-25T03:35:08.417142+0000","last_peered":"2023-08-25T03:35:08.417142+0000","last_clean":"2023-08-25T03:35:08.417142+0000","last_became_active":"2023-08-25T03:35:07.425681+0000","last_became_peered":"2023-08-25T03:35:07.425681+0000","last_unstale":"2023-08-25T03:35:08.417142+0000","last_undegraded":"2023-08-25T03:35:08.417142+0000","last_fullsized":"2023-08-25T03:35:08.417142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:47:50.765716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417043+0000","last_change":"2023-08-25T03:35:07.420451+0000","last_active":"2023-08-25T03:35:08.417043+0000","last_peered":"2023-08-25T03:35:08.417043+0000","last_clean":"2023-08-25T03:35:08.417043+0000","last_became_active":"2023-08-25T03:35:07.419829+0000","last_became_peered":"2023-08-25T03:35:07.419829+0000","last_unstale":"2023-08-25T03:35:08.417043+0000","last_undegraded":"2023-08-25T03:35:08.417043+0000","last_fullsized":"2023-08-25T03:35:08.417043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:11:43.194516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.416841+0000","last_change":"2023-08-25T03:35:07.426030+0000","last_active":"2023-08-25T03:35:08.416841+0000","last_peered":"2023-08-25T03:35:08.416841+0000","last_clean":"2023-08-25T03:35:08.416841+0000","last_became_active":"2023-08-25T03:35:07.425704+0000","last_became_peered":"2023-08-25T03:35:07.425704+0000","last_unstale":"2023-08-25T03:35:08.416841+0000","last_undegraded":"2023-08-25T03:35:08.416841+0000","last_fullsized":"2023-08-25T03:35:08.416841+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:53:41.033308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430167+0000","last_change":"2023-08-25T03:35:07.428547+0000","last_active":"2023-08-25T03:35:07.430167+0000","last_peered":"2023-08-25T03:35:07.430167+0000","last_clean":"2023-08-25T03:35:07.430167+0000","last_became_active":"2023-08-25T03:35:07.428140+0000","last_became_peered":"2023-08-25T03:35:07.428140+0000","last_unstale":"2023-08-25T03:35:07.430167+0000","last_undegraded":"2023-08-25T03:35:07.430167+0000","last_fullsized":"2023-08-25T03:35:07.430167+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:31:11.022575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.419854+0000","last_change":"2023-08-25T03:35:07.417690+0000","last_active":"2023-08-25T03:35:07.419854+0000","last_peered":"2023-08-25T03:35:07.419854+0000","last_clean":"2023-08-25T03:35:07.419854+0000","last_became_active":"2023-08-25T03:35:07.417416+0000","last_became_peered":"2023-08-25T03:35:07.417416+0000","last_unstale":"2023-08-25T03:35:07.419854+0000","last_undegraded":"2023-08-25T03:35:07.419854+0000","last_fullsized":"2023-08-25T03:35:07.419854+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:26:39.801388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.437822+0000","last_change":"2023-08-25T03:35:07.433922+0000","last_active":"2023-08-25T03:35:07.437822+0000","last_peered":"2023-08-25T03:35:07.437822+0000","last_clean":"2023-08-25T03:35:07.437822+0000","last_became_active":"2023-08-25T03:35:07.433609+0000","last_became_peered":"2023-08-25T03:35:07.433609+0000","last_unstale":"2023-08-25T03:35:07.437822+0000","last_undegraded":"2023-08-25T03:35:07.437822+0000","last_fullsized":"2023-08-25T03:35:07.437822+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:40:02.227089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"36'4","reported_seq":17,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417744+0000","last_change":"2023-08-25T03:35:06.918831+0000","last_active":"2023-08-25T03:35:08.417744+0000","last_peered":"2023-08-25T03:35:08.417744+0000","last_clean":"2023-08-25T03:35:08.417744+0000","last_became_active":"2023-08-25T03:35:06.917482+0000","last_became_peered":"2023-08-25T03:35:06.917482+0000","last_unstale":"2023-08-25T03:35:08.417744+0000","last_undegraded":"2023-08-25T03:35:08.417744+0000","last_fullsized":"2023-08-25T03:35:08.417744+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:52:26.271207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.423101+0000","last_change":"2023-08-25T03:35:07.422242+0000","last_active":"2023-08-25T03:35:07.423101+0000","last_peered":"2023-08-25T03:35:07.423101+0000","last_clean":"2023-08-25T03:35:07.423101+0000","last_became_active":"2023-08-25T03:35:07.421935+0000","last_became_peered":"2023-08-25T03:35:07.421935+0000","last_unstale":"2023-08-25T03:35:07.423101+0000","last_undegraded":"2023-08-25T03:35:07.423101+0000","last_fullsized":"2023-08-25T03:35:07.423101+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:14:36.942982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.930319+0000","last_change":"2023-08-25T03:35:06.925380+0000","last_active":"2023-08-25T03:35:07.930319+0000","last_peered":"2023-08-25T03:35:07.930319+0000","last_clean":"2023-08-25T03:35:07.930319+0000","last_became_active":"2023-08-25T03:35:06.925001+0000","last_became_peered":"2023-08-25T03:35:06.925001+0000","last_unstale":"2023-08-25T03:35:07.930319+0000","last_undegraded":"2023-08-25T03:35:07.930319+0000","last_fullsized":"2023-08-25T03:35:07.930319+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:04:52.463770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.876512+0000","last_change":"2023-08-25T03:35:07.427990+0000","last_active":"2023-08-25T03:35:08.876512+0000","last_peered":"2023-08-25T03:35:08.876512+0000","last_clean":"2023-08-25T03:35:08.876512+0000","last_became_active":"2023-08-25T03:35:07.427701+0000","last_became_peered":"2023-08-25T03:35:07.427701+0000","last_unstale":"2023-08-25T03:35:08.876512+0000","last_undegraded":"2023-08-25T03:35:08.876512+0000","last_fullsized":"2023-08-25T03:35:08.876512+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:42:24.718848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.926845+0000","last_change":"2023-08-25T03:35:06.916315+0000","last_active":"2023-08-25T03:35:07.926845+0000","last_peered":"2023-08-25T03:35:07.926845+0000","last_clean":"2023-08-25T03:35:07.926845+0000","last_became_active":"2023-08-25T03:35:06.916002+0000","last_became_peered":"2023-08-25T03:35:06.916002+0000","last_unstale":"2023-08-25T03:35:07.926845+0000","last_undegraded":"2023-08-25T03:35:07.926845+0000","last_fullsized":"2023-08-25T03:35:07.926845+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:21:21.889508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430685+0000","last_change":"2023-08-25T03:35:07.427178+0000","last_active":"2023-08-25T03:35:07.430685+0000","last_peered":"2023-08-25T03:35:07.430685+0000","last_clean":"2023-08-25T03:35:07.430685+0000","last_became_active":"2023-08-25T03:35:07.426727+0000","last_became_peered":"2023-08-25T03:35:07.426727+0000","last_unstale":"2023-08-25T03:35:07.430685+0000","last_undegraded":"2023-08-25T03:35:07.430685+0000","last_fullsized":"2023-08-25T03:35:07.430685+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:43:45.461082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"36'2","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.876284+0000","last_change":"2023-08-25T03:35:06.922388+0000","last_active":"2023-08-25T03:35:08.876284+0000","last_peered":"2023-08-25T03:35:08.876284+0000","last_clean":"2023-08-25T03:35:08.876284+0000","last_became_active":"2023-08-25T03:35:06.922063+0000","last_became_peered":"2023-08-25T03:35:06.922063+0000","last_unstale":"2023-08-25T03:35:08.876284+0000","last_undegraded":"2023-08-25T03:35:08.876284+0000","last_fullsized":"2023-08-25T03:35:08.876284+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:39:50.273892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.422124+0000","last_change":"2023-08-25T03:35:07.420543+0000","last_active":"2023-08-25T03:35:07.422124+0000","last_peered":"2023-08-25T03:35:07.422124+0000","last_clean":"2023-08-25T03:35:07.422124+0000","last_became_active":"2023-08-25T03:35:07.420308+0000","last_became_peered":"2023-08-25T03:35:07.420308+0000","last_unstale":"2023-08-25T03:35:07.422124+0000","last_undegraded":"2023-08-25T03:35:07.422124+0000","last_fullsized":"2023-08-25T03:35:07.422124+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:21:28.094063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"36'1","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417237+0000","last_change":"2023-08-25T03:35:06.920456+0000","last_active":"2023-08-25T03:35:08.417237+0000","last_peered":"2023-08-25T03:35:08.417237+0000","last_clean":"2023-08-25T03:35:08.417237+0000","last_became_active":"2023-08-25T03:35:06.920158+0000","last_became_peered":"2023-08-25T03:35:06.920158+0000","last_unstale":"2023-08-25T03:35:08.417237+0000","last_undegraded":"2023-08-25T03:35:08.417237+0000","last_fullsized":"2023-08-25T03:35:08.417237+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:47:22.980811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.416483+0000","last_change":"2023-08-25T03:35:07.418017+0000","last_active":"2023-08-25T03:35:08.416483+0000","last_peered":"2023-08-25T03:35:08.416483+0000","last_clean":"2023-08-25T03:35:08.416483+0000","last_became_active":"2023-08-25T03:35:07.417494+0000","last_became_peered":"2023-08-25T03:35:07.417494+0000","last_unstale":"2023-08-25T03:35:08.416483+0000","last_undegraded":"2023-08-25T03:35:08.416483+0000","last_fullsized":"2023-08-25T03:35:08.416483+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:13:28.418703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.925494+0000","last_change":"2023-08-25T03:35:06.924347+0000","last_active":"2023-08-25T03:35:07.925494+0000","last_peered":"2023-08-25T03:35:07.925494+0000","last_clean":"2023-08-25T03:35:07.925494+0000","last_became_active":"2023-08-25T03:35:06.923920+0000","last_became_peered":"2023-08-25T03:35:06.923920+0000","last_unstale":"2023-08-25T03:35:07.925494+0000","last_undegraded":"2023-08-25T03:35:07.925494+0000","last_fullsized":"2023-08-25T03:35:07.925494+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:22:22.400143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.876317+0000","last_change":"2023-08-25T03:35:07.433164+0000","last_active":"2023-08-25T03:35:08.876317+0000","last_peered":"2023-08-25T03:35:08.876317+0000","last_clean":"2023-08-25T03:35:08.876317+0000","last_became_active":"2023-08-25T03:35:07.432832+0000","last_became_peered":"2023-08-25T03:35:07.432832+0000","last_unstale":"2023-08-25T03:35:08.876317+0000","last_undegraded":"2023-08-25T03:35:08.876317+0000","last_fullsized":"2023-08-25T03:35:08.876317+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:45:03.766857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.412909+0000","last_change":"2023-08-25T03:35:06.922664+0000","last_active":"2023-08-25T03:35:07.412909+0000","last_peered":"2023-08-25T03:35:07.412909+0000","last_clean":"2023-08-25T03:35:07.412909+0000","last_became_active":"2023-08-25T03:35:06.922349+0000","last_became_peered":"2023-08-25T03:35:06.922349+0000","last_unstale":"2023-08-25T03:35:07.412909+0000","last_undegraded":"2023-08-25T03:35:07.412909+0000","last_fullsized":"2023-08-25T03:35:07.412909+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:41:57.990032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.426780+0000","last_change":"2023-08-25T03:35:07.424107+0000","last_active":"2023-08-25T03:35:07.426780+0000","last_peered":"2023-08-25T03:35:07.426780+0000","last_clean":"2023-08-25T03:35:07.426780+0000","last_became_active":"2023-08-25T03:35:07.423882+0000","last_became_peered":"2023-08-25T03:35:07.423882+0000","last_unstale":"2023-08-25T03:35:07.426780+0000","last_undegraded":"2023-08-25T03:35:07.426780+0000","last_fullsized":"2023-08-25T03:35:07.426780+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:41:17.471725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.875946+0000","last_change":"2023-08-25T03:35:06.920173+0000","last_active":"2023-08-25T03:35:08.875946+0000","last_peered":"2023-08-25T03:35:08.875946+0000","last_clean":"2023-08-25T03:35:08.875946+0000","last_became_active":"2023-08-25T03:35:06.919830+0000","last_became_peered":"2023-08-25T03:35:06.919830+0000","last_unstale":"2023-08-25T03:35:08.875946+0000","last_undegraded":"2023-08-25T03:35:08.875946+0000","last_fullsized":"2023-08-25T03:35:08.875946+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:56:33.241024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430584+0000","last_change":"2023-08-25T03:35:07.428243+0000","last_active":"2023-08-25T03:35:07.430584+0000","last_peered":"2023-08-25T03:35:07.430584+0000","last_clean":"2023-08-25T03:35:07.430584+0000","last_became_active":"2023-08-25T03:35:07.427749+0000","last_became_peered":"2023-08-25T03:35:07.427749+0000","last_unstale":"2023-08-25T03:35:07.430584+0000","last_undegraded":"2023-08-25T03:35:07.430584+0000","last_fullsized":"2023-08-25T03:35:07.430584+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:38:58.557458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.421286+0000","last_change":"2023-08-25T03:35:06.922296+0000","last_active":"2023-08-25T03:35:07.421286+0000","last_peered":"2023-08-25T03:35:07.421286+0000","last_clean":"2023-08-25T03:35:07.421286+0000","last_became_active":"2023-08-25T03:35:06.921995+0000","last_became_peered":"2023-08-25T03:35:06.921995+0000","last_unstale":"2023-08-25T03:35:07.421286+0000","last_undegraded":"2023-08-25T03:35:07.421286+0000","last_fullsized":"2023-08-25T03:35:07.421286+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:54:46.489267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.435965+0000","last_change":"2023-08-25T03:35:07.431869+0000","last_active":"2023-08-25T03:35:07.435965+0000","last_peered":"2023-08-25T03:35:07.435965+0000","last_clean":"2023-08-25T03:35:07.435965+0000","last_became_active":"2023-08-25T03:35:07.431535+0000","last_became_peered":"2023-08-25T03:35:07.431535+0000","last_unstale":"2023-08-25T03:35:07.435965+0000","last_undegraded":"2023-08-25T03:35:07.435965+0000","last_fullsized":"2023-08-25T03:35:07.435965+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:12:41.463967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417521+0000","last_change":"2023-08-25T03:35:06.921964+0000","last_active":"2023-08-25T03:35:08.417521+0000","last_peered":"2023-08-25T03:35:08.417521+0000","last_clean":"2023-08-25T03:35:08.417521+0000","last_became_active":"2023-08-25T03:35:06.921702+0000","last_became_peered":"2023-08-25T03:35:06.921702+0000","last_unstale":"2023-08-25T03:35:08.417521+0000","last_undegraded":"2023-08-25T03:35:08.417521+0000","last_fullsized":"2023-08-25T03:35:08.417521+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:07:03.573902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430342+0000","last_change":"2023-08-25T03:35:07.428479+0000","last_active":"2023-08-25T03:35:07.430342+0000","last_peered":"2023-08-25T03:35:07.430342+0000","last_clean":"2023-08-25T03:35:07.430342+0000","last_became_active":"2023-08-25T03:35:07.428223+0000","last_became_peered":"2023-08-25T03:35:07.428223+0000","last_unstale":"2023-08-25T03:35:07.430342+0000","last_undegraded":"2023-08-25T03:35:07.430342+0000","last_fullsized":"2023-08-25T03:35:07.430342+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:14:36.942982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"36'3","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417024+0000","last_change":"2023-08-25T03:35:06.918555+0000","last_active":"2023-08-25T03:35:08.417024+0000","last_peered":"2023-08-25T03:35:08.417024+0000","last_clean":"2023-08-25T03:35:08.417024+0000","last_became_active":"2023-08-25T03:35:06.918229+0000","last_became_peered":"2023-08-25T03:35:06.918229+0000","last_unstale":"2023-08-25T03:35:08.417024+0000","last_undegraded":"2023-08-25T03:35:08.417024+0000","last_fullsized":"2023-08-25T03:35:08.417024+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:45:02.762751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.876576+0000","last_change":"2023-08-25T03:35:07.421607+0000","last_active":"2023-08-25T03:35:08.876576+0000","last_peered":"2023-08-25T03:35:08.876576+0000","last_clean":"2023-08-25T03:35:08.876576+0000","last_became_active":"2023-08-25T03:35:07.421279+0000","last_became_peered":"2023-08-25T03:35:07.421279+0000","last_unstale":"2023-08-25T03:35:08.876576+0000","last_undegraded":"2023-08-25T03:35:08.876576+0000","last_fullsized":"2023-08-25T03:35:08.876576+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:26:04.402442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"36'6","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.932756+0000","last_change":"2023-08-25T03:35:06.925767+0000","last_active":"2023-08-25T03:35:07.932756+0000","last_peered":"2023-08-25T03:35:07.932756+0000","last_clean":"2023-08-25T03:35:07.932756+0000","last_became_active":"2023-08-25T03:35:06.925490+0000","last_became_peered":"2023-08-25T03:35:06.925490+0000","last_unstale":"2023-08-25T03:35:07.932756+0000","last_undegraded":"2023-08-25T03:35:07.932756+0000","last_fullsized":"2023-08-25T03:35:07.932756+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:58:28.709654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430209+0000","last_change":"2023-08-25T03:35:07.425893+0000","last_active":"2023-08-25T03:35:07.430209+0000","last_peered":"2023-08-25T03:35:07.430209+0000","last_clean":"2023-08-25T03:35:07.430209+0000","last_became_active":"2023-08-25T03:35:07.425300+0000","last_became_peered":"2023-08-25T03:35:07.425300+0000","last_unstale":"2023-08-25T03:35:07.430209+0000","last_undegraded":"2023-08-25T03:35:07.430209+0000","last_fullsized":"2023-08-25T03:35:07.430209+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:24:53.465344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.922754+0000","last_change":"2023-08-25T03:35:06.924446+0000","last_active":"2023-08-25T03:35:07.922754+0000","last_peered":"2023-08-25T03:35:07.922754+0000","last_clean":"2023-08-25T03:35:07.922754+0000","last_became_active":"2023-08-25T03:35:06.924012+0000","last_became_peered":"2023-08-25T03:35:06.924012+0000","last_unstale":"2023-08-25T03:35:07.922754+0000","last_undegraded":"2023-08-25T03:35:07.922754+0000","last_fullsized":"2023-08-25T03:35:07.922754+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:38:13.581929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.429266+0000","last_change":"2023-08-25T03:35:07.425215+0000","last_active":"2023-08-25T03:35:07.429266+0000","last_peered":"2023-08-25T03:35:07.429266+0000","last_clean":"2023-08-25T03:35:07.429266+0000","last_became_active":"2023-08-25T03:35:07.424627+0000","last_became_peered":"2023-08-25T03:35:07.424627+0000","last_unstale":"2023-08-25T03:35:07.429266+0000","last_undegraded":"2023-08-25T03:35:07.429266+0000","last_fullsized":"2023-08-25T03:35:07.429266+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:54:18.712932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.413379+0000","last_change":"2023-08-25T03:35:06.921196+0000","last_active":"2023-08-25T03:35:07.413379+0000","last_peered":"2023-08-25T03:35:07.413379+0000","last_clean":"2023-08-25T03:35:07.413379+0000","last_became_active":"2023-08-25T03:35:06.920899+0000","last_became_peered":"2023-08-25T03:35:06.920899+0000","last_unstale":"2023-08-25T03:35:07.413379+0000","last_undegraded":"2023-08-25T03:35:07.413379+0000","last_fullsized":"2023-08-25T03:35:07.413379+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:41:16.467620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.433621+0000","last_change":"2023-08-25T03:35:07.429964+0000","last_active":"2023-08-25T03:35:07.433621+0000","last_peered":"2023-08-25T03:35:07.433621+0000","last_clean":"2023-08-25T03:35:07.433621+0000","last_became_active":"2023-08-25T03:35:07.429680+0000","last_became_peered":"2023-08-25T03:35:07.429680+0000","last_unstale":"2023-08-25T03:35:07.433621+0000","last_undegraded":"2023-08-25T03:35:07.433621+0000","last_fullsized":"2023-08-25T03:35:07.433621+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:58:29.713760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.416779+0000","last_change":"2023-08-25T03:35:06.917702+0000","last_active":"2023-08-25T03:35:08.416779+0000","last_peered":"2023-08-25T03:35:08.416779+0000","last_clean":"2023-08-25T03:35:08.416779+0000","last_became_active":"2023-08-25T03:35:06.917384+0000","last_became_peered":"2023-08-25T03:35:06.917384+0000","last_unstale":"2023-08-25T03:35:08.416779+0000","last_undegraded":"2023-08-25T03:35:08.416779+0000","last_fullsized":"2023-08-25T03:35:08.416779+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:08:01.715495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.429294+0000","last_change":"2023-08-25T03:35:07.425308+0000","last_active":"2023-08-25T03:35:07.429294+0000","last_peered":"2023-08-25T03:35:07.429294+0000","last_clean":"2023-08-25T03:35:07.429294+0000","last_became_active":"2023-08-25T03:35:07.424532+0000","last_became_peered":"2023-08-25T03:35:07.424532+0000","last_unstale":"2023-08-25T03:35:07.429294+0000","last_undegraded":"2023-08-25T03:35:07.429294+0000","last_fullsized":"2023-08-25T03:35:07.429294+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:01:29.759578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"36'8","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.931047+0000","last_change":"2023-08-25T03:35:06.925794+0000","last_active":"2023-08-25T03:35:07.931047+0000","last_peered":"2023-08-25T03:35:07.931047+0000","last_clean":"2023-08-25T03:35:07.931047+0000","last_became_active":"2023-08-25T03:35:06.925517+0000","last_became_peered":"2023-08-25T03:35:06.925517+0000","last_unstale":"2023-08-25T03:35:07.931047+0000","last_undegraded":"2023-08-25T03:35:07.931047+0000","last_fullsized":"2023-08-25T03:35:07.931047+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:43:44.456976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430595+0000","last_change":"2023-08-25T03:35:07.426813+0000","last_active":"2023-08-25T03:35:07.430595+0000","last_peered":"2023-08-25T03:35:07.430595+0000","last_clean":"2023-08-25T03:35:07.430595+0000","last_became_active":"2023-08-25T03:35:07.426491+0000","last_became_peered":"2023-08-25T03:35:07.426491+0000","last_unstale":"2023-08-25T03:35:07.430595+0000","last_undegraded":"2023-08-25T03:35:07.430595+0000","last_fullsized":"2023-08-25T03:35:07.430595+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:24:53.465344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"36'1","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417556+0000","last_change":"2023-08-25T03:35:06.919030+0000","last_active":"2023-08-25T03:35:08.417556+0000","last_peered":"2023-08-25T03:35:08.417556+0000","last_clean":"2023-08-25T03:35:08.417556+0000","last_became_active":"2023-08-25T03:35:06.918711+0000","last_became_peered":"2023-08-25T03:35:06.918711+0000","last_unstale":"2023-08-25T03:35:08.417556+0000","last_undegraded":"2023-08-25T03:35:08.417556+0000","last_fullsized":"2023-08-25T03:35:08.417556+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:45:31.400474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417457+0000","last_change":"2023-08-25T03:35:07.420169+0000","last_active":"2023-08-25T03:35:08.417457+0000","last_peered":"2023-08-25T03:35:08.417457+0000","last_clean":"2023-08-25T03:35:08.417457+0000","last_became_active":"2023-08-25T03:35:07.419856+0000","last_became_peered":"2023-08-25T03:35:07.419856+0000","last_unstale":"2023-08-25T03:35:08.417457+0000","last_undegraded":"2023-08-25T03:35:08.417457+0000","last_fullsized":"2023-08-25T03:35:08.417457+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:42:24.718848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"36'2","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.876400+0000","last_change":"2023-08-25T03:35:06.919217+0000","last_active":"2023-08-25T03:35:08.876400+0000","last_peered":"2023-08-25T03:35:08.876400+0000","last_clean":"2023-08-25T03:35:08.876400+0000","last_became_active":"2023-08-25T03:35:06.918932+0000","last_became_peered":"2023-08-25T03:35:06.918932+0000","last_unstale":"2023-08-25T03:35:08.876400+0000","last_undegraded":"2023-08-25T03:35:08.876400+0000","last_fullsized":"2023-08-25T03:35:08.876400+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:48:28.834672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.876354+0000","last_change":"2023-08-25T03:35:07.422467+0000","last_active":"2023-08-25T03:35:08.876354+0000","last_peered":"2023-08-25T03:35:08.876354+0000","last_clean":"2023-08-25T03:35:08.876354+0000","last_became_active":"2023-08-25T03:35:07.421892+0000","last_became_peered":"2023-08-25T03:35:07.421892+0000","last_unstale":"2023-08-25T03:35:08.876354+0000","last_undegraded":"2023-08-25T03:35:08.876354+0000","last_fullsized":"2023-08-25T03:35:08.876354+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:47:50.765716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.410579+0000","last_change":"2023-08-25T03:35:06.925867+0000","last_active":"2023-08-25T03:35:07.410579+0000","last_peered":"2023-08-25T03:35:07.410579+0000","last_clean":"2023-08-25T03:35:07.410579+0000","last_became_active":"2023-08-25T03:35:06.925536+0000","last_became_peered":"2023-08-25T03:35:06.925536+0000","last_unstale":"2023-08-25T03:35:07.410579+0000","last_undegraded":"2023-08-25T03:35:07.410579+0000","last_fullsized":"2023-08-25T03:35:07.410579+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:21:58.955199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430830+0000","last_change":"2023-08-25T03:35:07.428493+0000","last_active":"2023-08-25T03:35:07.430830+0000","last_peered":"2023-08-25T03:35:07.430830+0000","last_clean":"2023-08-25T03:35:07.430830+0000","last_became_active":"2023-08-25T03:35:07.428211+0000","last_became_peered":"2023-08-25T03:35:07.428211+0000","last_unstale":"2023-08-25T03:35:07.430830+0000","last_undegraded":"2023-08-25T03:35:07.430830+0000","last_fullsized":"2023-08-25T03:35:07.430830+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:51:59.147211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.413214+0000","last_change":"2023-08-25T03:35:06.919498+0000","last_active":"2023-08-25T03:35:07.413214+0000","last_peered":"2023-08-25T03:35:07.413214+0000","last_clean":"2023-08-25T03:35:07.413214+0000","last_became_active":"2023-08-25T03:35:06.919235+0000","last_became_peered":"2023-08-25T03:35:06.919235+0000","last_unstale":"2023-08-25T03:35:07.413214+0000","last_undegraded":"2023-08-25T03:35:07.413214+0000","last_fullsized":"2023-08-25T03:35:07.413214+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:15:18.138334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.435349+0000","last_change":"2023-08-25T03:35:07.430947+0000","last_active":"2023-08-25T03:35:07.435349+0000","last_peered":"2023-08-25T03:35:07.435349+0000","last_clean":"2023-08-25T03:35:07.435349+0000","last_became_active":"2023-08-25T03:35:07.430634+0000","last_became_peered":"2023-08-25T03:35:07.430634+0000","last_unstale":"2023-08-25T03:35:07.435349+0000","last_undegraded":"2023-08-25T03:35:07.435349+0000","last_fullsized":"2023-08-25T03:35:07.435349+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:38:37.183111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.410741+0000","last_change":"2023-08-25T03:35:06.923471+0000","last_active":"2023-08-25T03:35:07.410741+0000","last_peered":"2023-08-25T03:35:07.410741+0000","last_clean":"2023-08-25T03:35:07.410741+0000","last_became_active":"2023-08-25T03:35:06.923065+0000","last_became_peered":"2023-08-25T03:35:06.923065+0000","last_unstale":"2023-08-25T03:35:07.410741+0000","last_undegraded":"2023-08-25T03:35:07.410741+0000","last_fullsized":"2023-08-25T03:35:07.410741+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:56:24.758139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.422961+0000","last_change":"2023-08-25T03:35:07.422059+0000","last_active":"2023-08-25T03:35:07.422961+0000","last_peered":"2023-08-25T03:35:07.422961+0000","last_clean":"2023-08-25T03:35:07.422961+0000","last_became_active":"2023-08-25T03:35:07.421768+0000","last_became_peered":"2023-08-25T03:35:07.421768+0000","last_unstale":"2023-08-25T03:35:07.422961+0000","last_undegraded":"2023-08-25T03:35:07.422961+0000","last_fullsized":"2023-08-25T03:35:07.422961+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:41:17.471725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.410526+0000","last_change":"2023-08-25T03:35:06.918197+0000","last_active":"2023-08-25T03:35:07.410526+0000","last_peered":"2023-08-25T03:35:07.410526+0000","last_clean":"2023-08-25T03:35:07.410526+0000","last_became_active":"2023-08-25T03:35:06.917871+0000","last_became_peered":"2023-08-25T03:35:06.917871+0000","last_unstale":"2023-08-25T03:35:07.410526+0000","last_undegraded":"2023-08-25T03:35:07.410526+0000","last_fullsized":"2023-08-25T03:35:07.410526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:49:48.227321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.416335+0000","last_change":"2023-08-25T03:35:07.422923+0000","last_active":"2023-08-25T03:35:08.416335+0000","last_peered":"2023-08-25T03:35:08.416335+0000","last_clean":"2023-08-25T03:35:08.416335+0000","last_became_active":"2023-08-25T03:35:07.422608+0000","last_became_peered":"2023-08-25T03:35:07.422608+0000","last_unstale":"2023-08-25T03:35:08.416335+0000","last_undegraded":"2023-08-25T03:35:08.416335+0000","last_fullsized":"2023-08-25T03:35:08.416335+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:39:35.960897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.423940+0000","last_change":"2023-08-25T03:35:06.926686+0000","last_active":"2023-08-25T03:35:07.423940+0000","last_peered":"2023-08-25T03:35:07.423940+0000","last_clean":"2023-08-25T03:35:07.423940+0000","last_became_active":"2023-08-25T03:35:06.926400+0000","last_became_peered":"2023-08-25T03:35:06.926400+0000","last_unstale":"2023-08-25T03:35:07.423940+0000","last_undegraded":"2023-08-25T03:35:07.423940+0000","last_fullsized":"2023-08-25T03:35:07.423940+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:25:10.189227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430550+0000","last_change":"2023-08-25T03:35:07.426694+0000","last_active":"2023-08-25T03:35:07.430550+0000","last_peered":"2023-08-25T03:35:07.430550+0000","last_clean":"2023-08-25T03:35:07.430550+0000","last_became_active":"2023-08-25T03:35:07.426416+0000","last_became_peered":"2023-08-25T03:35:07.426416+0000","last_unstale":"2023-08-25T03:35:07.430550+0000","last_undegraded":"2023-08-25T03:35:07.430550+0000","last_fullsized":"2023-08-25T03:35:07.430550+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:21:28.094063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.410396+0000","last_change":"2023-08-25T03:35:06.923379+0000","last_active":"2023-08-25T03:35:07.410396+0000","last_peered":"2023-08-25T03:35:07.410396+0000","last_clean":"2023-08-25T03:35:07.410396+0000","last_became_active":"2023-08-25T03:35:06.923087+0000","last_became_peered":"2023-08-25T03:35:06.923087+0000","last_unstale":"2023-08-25T03:35:07.410396+0000","last_undegraded":"2023-08-25T03:35:07.410396+0000","last_fullsized":"2023-08-25T03:35:07.410396+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:09:17.865796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430258+0000","last_change":"2023-08-25T03:35:07.426052+0000","last_active":"2023-08-25T03:35:07.430258+0000","last_peered":"2023-08-25T03:35:07.430258+0000","last_clean":"2023-08-25T03:35:07.430258+0000","last_became_active":"2023-08-25T03:35:07.425640+0000","last_became_peered":"2023-08-25T03:35:07.425640+0000","last_unstale":"2023-08-25T03:35:07.430258+0000","last_undegraded":"2023-08-25T03:35:07.430258+0000","last_fullsized":"2023-08-25T03:35:07.430258+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:21:22.893614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.412175+0000","last_change":"2023-08-25T03:35:06.924905+0000","last_active":"2023-08-25T03:35:07.412175+0000","last_peered":"2023-08-25T03:35:07.412175+0000","last_clean":"2023-08-25T03:35:07.412175+0000","last_became_active":"2023-08-25T03:35:06.924536+0000","last_became_peered":"2023-08-25T03:35:06.924536+0000","last_unstale":"2023-08-25T03:35:07.412175+0000","last_undegraded":"2023-08-25T03:35:07.412175+0000","last_fullsized":"2023-08-25T03:35:07.412175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:58:51.098028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.413296+0000","last_change":"2023-08-25T03:33:57.570267+0000","last_active":"2023-08-25T03:35:07.413296+0000","last_peered":"2023-08-25T03:35:07.413296+0000","last_clean":"2023-08-25T03:35:07.413296+0000","last_became_active":"2023-08-25T03:33:57.569882+0000","last_became_peered":"2023-08-25T03:33:57.569882+0000","last_unstale":"2023-08-25T03:35:07.413296+0000","last_undegraded":"2023-08-25T03:35:07.413296+0000","last_fullsized":"2023-08-25T03:35:07.413296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:03:43.121931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430915+0000","last_change":"2023-08-25T03:35:07.429830+0000","last_active":"2023-08-25T03:35:07.430915+0000","last_peered":"2023-08-25T03:35:07.430915+0000","last_clean":"2023-08-25T03:35:07.430915+0000","last_became_active":"2023-08-25T03:35:07.429577+0000","last_became_peered":"2023-08-25T03:35:07.429577+0000","last_unstale":"2023-08-25T03:35:07.430915+0000","last_undegraded":"2023-08-25T03:35:07.430915+0000","last_fullsized":"2023-08-25T03:35:07.430915+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:21:22.893614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.412355+0000","last_change":"2023-08-25T03:35:06.924926+0000","last_active":"2023-08-25T03:35:07.412355+0000","last_peered":"2023-08-25T03:35:07.412355+0000","last_clean":"2023-08-25T03:35:07.412355+0000","last_became_active":"2023-08-25T03:35:06.924351+0000","last_became_peered":"2023-08-25T03:35:06.924351+0000","last_unstale":"2023-08-25T03:35:07.412355+0000","last_undegraded":"2023-08-25T03:35:07.412355+0000","last_fullsized":"2023-08-25T03:35:07.412355+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:15:18.138334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":45,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.416632+0000","last_change":"2023-08-25T03:33:57.569595+0000","last_active":"2023-08-25T03:35:08.416632+0000","last_peered":"2023-08-25T03:35:08.416632+0000","last_clean":"2023-08-25T03:35:08.416632+0000","last_became_active":"2023-08-25T03:33:57.569307+0000","last_became_peered":"2023-08-25T03:33:57.569307+0000","last_unstale":"2023-08-25T03:35:08.416632+0000","last_undegraded":"2023-08-25T03:35:08.416632+0000","last_fullsized":"2023-08-25T03:35:08.416632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:41:45.092985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.437868+0000","last_change":"2023-08-25T03:35:07.434675+0000","last_active":"2023-08-25T03:35:07.437868+0000","last_peered":"2023-08-25T03:35:07.437868+0000","last_clean":"2023-08-25T03:35:07.437868+0000","last_became_active":"2023-08-25T03:35:07.434369+0000","last_became_peered":"2023-08-25T03:35:07.434369+0000","last_unstale":"2023-08-25T03:35:07.437868+0000","last_undegraded":"2023-08-25T03:35:07.437868+0000","last_fullsized":"2023-08-25T03:35:07.437868+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:29:40.853414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.410690+0000","last_change":"2023-08-25T03:35:06.919122+0000","last_active":"2023-08-25T03:35:07.410690+0000","last_peered":"2023-08-25T03:35:07.410690+0000","last_clean":"2023-08-25T03:35:07.410690+0000","last_became_active":"2023-08-25T03:35:06.918837+0000","last_became_peered":"2023-08-25T03:35:06.918837+0000","last_unstale":"2023-08-25T03:35:07.410690+0000","last_undegraded":"2023-08-25T03:35:07.410690+0000","last_fullsized":"2023-08-25T03:35:07.410690+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:07:00.853225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":43,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.410847+0000","last_change":"2023-08-25T03:33:57.886704+0000","last_active":"2023-08-25T03:35:07.410847+0000","last_peered":"2023-08-25T03:35:07.410847+0000","last_clean":"2023-08-25T03:35:07.410847+0000","last_became_active":"2023-08-25T03:33:57.886294+0000","last_became_peered":"2023-08-25T03:33:57.886294+0000","last_unstale":"2023-08-25T03:35:07.410847+0000","last_undegraded":"2023-08-25T03:35:07.410847+0000","last_fullsized":"2023-08-25T03:35:07.410847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:27:20.869204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417460+0000","last_change":"2023-08-25T03:35:07.426139+0000","last_active":"2023-08-25T03:35:08.417460+0000","last_peered":"2023-08-25T03:35:08.417460+0000","last_clean":"2023-08-25T03:35:08.417460+0000","last_became_active":"2023-08-25T03:35:07.425802+0000","last_became_peered":"2023-08-25T03:35:07.425802+0000","last_unstale":"2023-08-25T03:35:08.417460+0000","last_undegraded":"2023-08-25T03:35:08.417460+0000","last_fullsized":"2023-08-25T03:35:08.417460+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:26:04.402442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.423050+0000","last_change":"2023-08-25T03:35:06.926727+0000","last_active":"2023-08-25T03:35:07.423050+0000","last_peered":"2023-08-25T03:35:07.423050+0000","last_clean":"2023-08-25T03:35:07.423050+0000","last_became_active":"2023-08-25T03:35:06.926447+0000","last_became_peered":"2023-08-25T03:35:06.926447+0000","last_unstale":"2023-08-25T03:35:07.423050+0000","last_undegraded":"2023-08-25T03:35:07.423050+0000","last_fullsized":"2023-08-25T03:35:07.423050+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:37:26.114237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":45,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.416899+0000","last_change":"2023-08-25T03:33:57.570657+0000","last_active":"2023-08-25T03:35:08.416899+0000","last_peered":"2023-08-25T03:35:08.416899+0000","last_clean":"2023-08-25T03:35:08.416899+0000","last_became_active":"2023-08-25T03:33:57.570425+0000","last_became_peered":"2023-08-25T03:33:57.570425+0000","last_unstale":"2023-08-25T03:35:08.416899+0000","last_undegraded":"2023-08-25T03:35:08.416899+0000","last_fullsized":"2023-08-25T03:35:08.416899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:38:40.932053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":193,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.423115+0000","last_change":"2023-08-25T03:33:51.259898+0000","last_active":"2023-08-25T03:35:07.423115+0000","last_peered":"2023-08-25T03:35:07.423115+0000","last_clean":"2023-08-25T03:35:07.423115+0000","last_became_active":"2023-08-25T03:33:51.258780+0000","last_became_peered":"2023-08-25T03:33:51.258780+0000","last_unstale":"2023-08-25T03:35:07.423115+0000","last_undegraded":"2023-08-25T03:35:07.423115+0000","last_fullsized":"2023-08-25T03:35:07.423115+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:49.666858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:49.666858+0000","last_clean_scrub_stamp":"2023-08-25T03:33:49.666858+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:53:30.259949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.420489+0000","last_change":"2023-08-25T03:35:07.419689+0000","last_active":"2023-08-25T03:35:07.420489+0000","last_peered":"2023-08-25T03:35:07.420489+0000","last_clean":"2023-08-25T03:35:07.420489+0000","last_became_active":"2023-08-25T03:35:07.419419+0000","last_became_peered":"2023-08-25T03:35:07.419419+0000","last_unstale":"2023-08-25T03:35:07.420489+0000","last_undegraded":"2023-08-25T03:35:07.420489+0000","last_fullsized":"2023-08-25T03:35:07.420489+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:08:24.760786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"36'2","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417632+0000","last_change":"2023-08-25T03:35:06.921662+0000","last_active":"2023-08-25T03:35:08.417632+0000","last_peered":"2023-08-25T03:35:08.417632+0000","last_clean":"2023-08-25T03:35:08.417632+0000","last_became_active":"2023-08-25T03:35:06.921352+0000","last_became_peered":"2023-08-25T03:35:06.921352+0000","last_unstale":"2023-08-25T03:35:08.417632+0000","last_undegraded":"2023-08-25T03:35:08.417632+0000","last_fullsized":"2023-08-25T03:35:08.417632+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:49:26.211793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.423268+0000","last_change":"2023-08-25T03:33:57.571821+0000","last_active":"2023-08-25T03:35:07.423268+0000","last_peered":"2023-08-25T03:35:07.423268+0000","last_clean":"2023-08-25T03:35:07.423268+0000","last_became_active":"2023-08-25T03:33:57.571528+0000","last_became_peered":"2023-08-25T03:33:57.571528+0000","last_unstale":"2023-08-25T03:35:07.423268+0000","last_undegraded":"2023-08-25T03:35:07.423268+0000","last_fullsized":"2023-08-25T03:35:07.423268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:25:39.853603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430540+0000","last_change":"2023-08-25T03:35:07.426463+0000","last_active":"2023-08-25T03:35:07.430540+0000","last_peered":"2023-08-25T03:35:07.430540+0000","last_clean":"2023-08-25T03:35:07.430540+0000","last_became_active":"2023-08-25T03:35:07.426213+0000","last_became_peered":"2023-08-25T03:35:07.426213+0000","last_unstale":"2023-08-25T03:35:07.430540+0000","last_undegraded":"2023-08-25T03:35:07.430540+0000","last_fullsized":"2023-08-25T03:35:07.430540+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:46:49.278016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.416988+0000","last_change":"2023-08-25T03:35:06.922775+0000","last_active":"2023-08-25T03:35:08.416988+0000","last_peered":"2023-08-25T03:35:08.416988+0000","last_clean":"2023-08-25T03:35:08.416988+0000","last_became_active":"2023-08-25T03:35:06.921468+0000","last_became_peered":"2023-08-25T03:35:06.921468+0000","last_unstale":"2023-08-25T03:35:08.416988+0000","last_undegraded":"2023-08-25T03:35:08.416988+0000","last_fullsized":"2023-08-25T03:35:08.416988+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:17:21.597681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.423599+0000","last_change":"2023-08-25T03:33:57.571730+0000","last_active":"2023-08-25T03:35:07.423599+0000","last_peered":"2023-08-25T03:35:07.423599+0000","last_clean":"2023-08-25T03:35:07.423599+0000","last_became_active":"2023-08-25T03:33:57.571442+0000","last_became_peered":"2023-08-25T03:33:57.571442+0000","last_unstale":"2023-08-25T03:35:07.423599+0000","last_undegraded":"2023-08-25T03:35:07.423599+0000","last_fullsized":"2023-08-25T03:35:07.423599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:29:37.516806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.876191+0000","last_change":"2023-08-25T03:35:07.431421+0000","last_active":"2023-08-25T03:35:08.876191+0000","last_peered":"2023-08-25T03:35:08.876191+0000","last_clean":"2023-08-25T03:35:08.876191+0000","last_became_active":"2023-08-25T03:35:07.431107+0000","last_became_peered":"2023-08-25T03:35:07.431107+0000","last_unstale":"2023-08-25T03:35:08.876191+0000","last_undegraded":"2023-08-25T03:35:08.876191+0000","last_fullsized":"2023-08-25T03:35:08.876191+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:42:55.438930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.875994+0000","last_change":"2023-08-25T03:35:06.922373+0000","last_active":"2023-08-25T03:35:08.875994+0000","last_peered":"2023-08-25T03:35:08.875994+0000","last_clean":"2023-08-25T03:35:08.875994+0000","last_became_active":"2023-08-25T03:35:06.922049+0000","last_became_peered":"2023-08-25T03:35:06.922049+0000","last_unstale":"2023-08-25T03:35:08.875994+0000","last_undegraded":"2023-08-25T03:35:08.875994+0000","last_fullsized":"2023-08-25T03:35:08.875994+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:13:27.414597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.413546+0000","last_change":"2023-08-25T03:33:57.570506+0000","last_active":"2023-08-25T03:35:07.413546+0000","last_peered":"2023-08-25T03:35:07.413546+0000","last_clean":"2023-08-25T03:35:07.413546+0000","last_became_active":"2023-08-25T03:33:57.570171+0000","last_became_peered":"2023-08-25T03:33:57.570171+0000","last_unstale":"2023-08-25T03:35:07.413546+0000","last_undegraded":"2023-08-25T03:35:07.413546+0000","last_fullsized":"2023-08-25T03:35:07.413546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:23:43.119399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.437861+0000","last_change":"2023-08-25T03:35:07.434619+0000","last_active":"2023-08-25T03:35:07.437861+0000","last_peered":"2023-08-25T03:35:07.437861+0000","last_clean":"2023-08-25T03:35:07.437861+0000","last_became_active":"2023-08-25T03:35:07.434295+0000","last_became_peered":"2023-08-25T03:35:07.434295+0000","last_unstale":"2023-08-25T03:35:07.437861+0000","last_undegraded":"2023-08-25T03:35:07.437861+0000","last_fullsized":"2023-08-25T03:35:07.437861+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:52:27.271866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.417992+0000","last_change":"2023-08-25T03:35:06.922104+0000","last_active":"2023-08-25T03:35:07.417992+0000","last_peered":"2023-08-25T03:35:07.417992+0000","last_clean":"2023-08-25T03:35:07.417992+0000","last_became_active":"2023-08-25T03:35:06.921783+0000","last_became_peered":"2023-08-25T03:35:06.921783+0000","last_unstale":"2023-08-25T03:35:07.417992+0000","last_undegraded":"2023-08-25T03:35:07.417992+0000","last_fullsized":"2023-08-25T03:35:07.417992+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:29:39.849308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":45,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417823+0000","last_change":"2023-08-25T03:33:57.570598+0000","last_active":"2023-08-25T03:35:08.417823+0000","last_peered":"2023-08-25T03:35:08.417823+0000","last_clean":"2023-08-25T03:35:08.417823+0000","last_became_active":"2023-08-25T03:33:57.570215+0000","last_became_peered":"2023-08-25T03:33:57.570215+0000","last_unstale":"2023-08-25T03:35:08.417823+0000","last_undegraded":"2023-08-25T03:35:08.417823+0000","last_fullsized":"2023-08-25T03:35:08.417823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:42:30.286958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2340,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542165,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1015808,"data_stored":758668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.70799999999999996}]},{"osd":2,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.54700000000000004}]},{"osd":3,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.65800000000000003}]},{"osd":4,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.748},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.77400000000000002}]},{"osd":6,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.67900000000000005}]}]},{"osd":6,"up_from":14,"seq":60129542165,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":421888,"data_stored":173853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.58999999999999997}]},{"osd":1,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.621},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":4.5419999999999998,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":0.68799999999999994}]},{"osd":2,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":5.5179999999999998,"5min":5.5179999999999998,"15min":5.5179999999999998},"last":0.745},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.77400000000000002}]},{"osd":3,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.81100000000000005}]},{"osd":4,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":6.4050000000000002,"5min":6.4050000000000002,"15min":6.4050000000000002},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.89600000000000002}]},{"osd":5,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":7.4029999999999996,"5min":7.4029999999999996,"15min":7.4029999999999996},"last":0.84299999999999997}]},{"osd":7,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.97899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":166708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":6.6120000000000001,"5min":6.6120000000000001,"15min":6.6120000000000001},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.3530000000000002,"5min":3.3530000000000002,"15min":3.3530000000000002},"last":0.27600000000000002}]},{"osd":2,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":9.0950000000000006,"5min":9.0950000000000006,"15min":9.0950000000000006},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.35799999999999998}]},{"osd":3,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":3.4820000000000002,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":6.1970000000000001,"5min":6.1970000000000001,"15min":6.1970000000000001},"last":0.376}]},{"osd":4,"last update":"Fri Aug 25 03:34:49 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":6.4400000000000004,"5min":6.4400000000000004,"15min":6.4400000000000004},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":6.5860000000000003,"5min":6.5860000000000003,"15min":6.5860000000000003},"last":0.46100000000000002}]},{"osd":5,"last update":"Fri Aug 25 03:34:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":3.3900000000000001,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.436},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":6.399,"5min":6.399,"15min":6.399},"last":0.47499999999999998}]},{"osd":6,"last update":"Fri Aug 25 03:34:49 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":6.5229999999999997,"5min":6.5229999999999997,"15min":6.5229999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":3.4529999999999998,"5min":3.4529999999999998,"15min":3.4529999999999998},"last":0.498}]},{"osd":7,"last update":"Fri Aug 25 03:34:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":6.5590000000000002,"5min":6.5590000000000002,"15min":6.5590000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":3.4220000000000002,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.54300000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":999424,"data_stored":756986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"last":0.52900000000000003}]},{"osd":2,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.436},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.50900000000000001}]},{"osd":3,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.48399999999999999}]},{"osd":4,"last update":"Fri Aug 25 03:34:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.69799999999999995}]},{"osd":5,"last update":"Fri Aug 25 03:34:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.755},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.69099999999999995}]},{"osd":6,"last update":"Fri Aug 25 03:34:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.73299999999999998}]},{"osd":7,"last update":"Fri Aug 25 03:34:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.81399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":159490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.26300000000000001}]},{"osd":1,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.32300000000000001}]},{"osd":3,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.373},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.38100000000000001}]},{"osd":4,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.58299999999999996}]},{"osd":5,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.55600000000000005}]},{"osd":6,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.69799999999999995}]},{"osd":7,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.59199999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":421888,"data_stored":168264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.23200000000000001},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":12.923999999999999,"5min":12.923999999999999,"15min":12.923999999999999},"last":0.36499999999999999}]},{"osd":1,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":2.6680000000000001,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.26700000000000002}]},{"osd":2,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.311}]},{"osd":4,"last update":"Fri Aug 25 03:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.443},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.60099999999999998}]},{"osd":5,"last update":"Fri Aug 25 03:34:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":6.3520000000000003,"5min":6.3520000000000003,"15min":6.3520000000000003},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.497}]},{"osd":6,"last update":"Fri Aug 25 03:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.9849999999999999,"5min":2.9849999999999999,"15min":2.9849999999999999},"last":0.58399999999999996}]},{"osd":7,"last update":"Fri Aug 25 03:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.51500000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542165,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":173875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.41999999999999998}]},{"osd":1,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":8.327,"5min":8.327,"15min":8.327},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.58499999999999996}]},{"osd":2,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.70799999999999996}]},{"osd":3,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.63100000000000001}]},{"osd":5,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.81999999999999995}]},{"osd":6,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.91900000000000004}]},{"osd":7,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.79300000000000004}]}]},{"osd":5,"up_from":14,"seq":60129542164,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":173383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.628}]},{"osd":1,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.497},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.45000000000000001}]},{"osd":2,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.54900000000000004}]},{"osd":3,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.754}]},{"osd":4,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.58199999999999996}]},{"osd":6,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.60199999999999998}]},{"osd":7,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.77900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-25T03:35:12.088 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T03:35:12.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.242+0000 7fda6c162700 1 Processor -- start 2023-08-25T03:35:12.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.242+0000 7fda6c162700 1 -- start start 2023-08-25T03:35:12.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.243+0000 7fda6c162700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fda6415a030 0x7fda640cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:12.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.243+0000 7fda6c162700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda64071dc0 con 0x7fda640cc1f0 2023-08-25T03:35:12.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.243+0000 7fda6c162700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda640720a0 con 0x7fda6415a410 2023-08-25T03:35:12.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.243+0000 7fda6c162700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda640cb4b0 con 0x7fda6415a030 2023-08-25T03:35:12.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.243+0000 7fda696fd700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fda640cc1f0 0x7fda640ce170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48110/0 (socket says 172.21.15.2:48110) 2023-08-25T03:35:12.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.243+0000 7fda696fd700 1 -- 172.21.15.2:0/226012451 learned_addr learned my addr 172.21.15.2:0/226012451 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:12.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.243+0000 7fda68efc700 1 -- 172.21.15.2:0/226012451 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3037454578 0 0) 0x7fda64071dc0 con 0x7fda640cc1f0 2023-08-25T03:35:12.246 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.243+0000 7fda69efe700 1 --2- 172.21.15.2:0/226012451 >> v2:172.21.15.31:3300/0 conn(0x7fda6415a030 0x7fda640cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:12.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.243+0000 7fda68efc700 1 -- 172.21.15.2:0/226012451 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda500035b0 con 0x7fda640cc1f0 2023-08-25T03:35:12.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.243+0000 7fda68efc700 1 -- 172.21.15.2:0/226012451 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3951787831 0 0) 0x7fda500035b0 con 0x7fda640cc1f0 2023-08-25T03:35:12.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.243+0000 7fda68efc700 1 -- 172.21.15.2:0/226012451 >> v2:172.21.15.31:3300/0 conn(0x7fda6415a030 msgr2=0x7fda640cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:12.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.243+0000 7fda68efc700 1 --2- 172.21.15.2:0/226012451 >> v2:172.21.15.31:3300/0 conn(0x7fda6415a030 0x7fda640cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:12.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.243+0000 7fda68efc700 1 -- 172.21.15.2:0/226012451 >> v1:172.21.15.31:6789/0 conn(0x7fda6415a410 legacy=0x7fda640ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:12.247 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.243+0000 7fda68efc700 1 -- 172.21.15.2:0/226012451 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda640cb790 con 0x7fda640cc1f0 2023-08-25T03:35:12.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.243+0000 7fda68efc700 1 -- 172.21.15.2:0/226012451 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7fda58001670 con 0x7fda640cc1f0 2023-08-25T03:35:12.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.243+0000 7fda68efc700 1 -- 172.21.15.2:0/226012451 >> v1:172.21.15.2:6789/0 conn(0x7fda640cc1f0 legacy=0x7fda640ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:12.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.243+0000 7fda68efc700 1 --2- 172.21.15.2:0/226012451 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fda50003a00 0x7fda50003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:12.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.243+0000 7fda68efc700 1 --2- 172.21.15.2:0/226012451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda50004850 0x7fda50004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:12.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.243+0000 7fda68efc700 1 --2- 172.21.15.2:0/226012451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda50005210 0x7fda5000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:12.248 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.243+0000 7fda68efc700 1 -- 172.21.15.2:0/226012451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda640cb4b0 con 0x7fda50004850 2023-08-25T03:35:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.243+0000 7fda68efc700 1 -- 172.21.15.2:0/226012451 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5000c0b0 con 0x7fda50005210 2023-08-25T03:35:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.243+0000 7fda68efc700 1 -- 172.21.15.2:0/226012451 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5000e480 con 0x7fda50003a00 2023-08-25T03:35:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.244+0000 7fda6a6ff700 1 --2- 172.21.15.2:0/226012451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda50005210 0x7fda5000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.244+0000 7fda696fd700 1 --2- 172.21.15.2:0/226012451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda50004850 0x7fda50004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.244+0000 7fda69efe700 1 --2- 172.21.15.2:0/226012451 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fda50003a00 0x7fda50003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.244+0000 7fda696fd700 1 -- 172.21.15.2:0/226012451 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fda50003a00 msgr2=0x7fda50003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.244+0000 7fda696fd700 1 --2- 172.21.15.2:0/226012451 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fda50003a00 0x7fda50003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:12.249 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.244+0000 7fda696fd700 1 -- 172.21.15.2:0/226012451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda50005210 msgr2=0x7fda5000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:12.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.244+0000 7fda696fd700 1 --2- 172.21.15.2:0/226012451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda50005210 0x7fda5000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:12.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.244+0000 7fda696fd700 1 -- 172.21.15.2:0/226012451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda500035b0 con 0x7fda50004850 2023-08-25T03:35:12.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.244+0000 7fda6a6ff700 1 --2- 172.21.15.2:0/226012451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda50005210 0x7fda5000ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:35:12.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.244+0000 7fda6a6ff700 1 -- 172.21.15.2:0/226012451 reap_dead start 2023-08-25T03:35:12.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.244+0000 7fda696fd700 1 --2- 172.21.15.2:0/226012451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda50004850 0x7fda50004cd0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fda5800c240 tx=0x7fda50005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac626ef8333fb9b3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:12.250 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.244+0000 7fda68efc700 1 -- 172.21.15.2:0/226012451 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda58007c10 con 0x7fda50004850 2023-08-25T03:35:12.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.244+0000 7fda68efc700 1 -- 172.21.15.2:0/226012451 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5800cde0 con 0x7fda50004850 2023-08-25T03:35:12.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.244+0000 7fda68efc700 1 -- 172.21.15.2:0/226012451 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda58005b80 con 0x7fda50004850 2023-08-25T03:35:12.251 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.245+0000 7fda6c162700 1 -- 172.21.15.2:0/226012451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda50004850 msgr2=0x7fda50004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:12.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.245+0000 7fda6c162700 1 --2- 172.21.15.2:0/226012451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda50004850 0x7fda50004cd0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fda5800c240 tx=0x7fda50005880 comp rx=0 tx=0).stop 2023-08-25T03:35:12.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.245+0000 7fda6c162700 1 -- 172.21.15.2:0/226012451 shutdown_connections 2023-08-25T03:35:12.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.245+0000 7fda6c162700 1 --2- 172.21.15.2:0/226012451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda50004850 0x7fda50004cd0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:12.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.245+0000 7fda6c162700 1 -- 172.21.15.2:0/226012451 >> 172.21.15.2:0/226012451 conn(0x7fda640b1cf0 msgr2=0x7fda640b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:12.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.245+0000 7fda6c162700 1 -- 172.21.15.2:0/226012451 shutdown_connections 2023-08-25T03:35:12.253 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.245+0000 7fda6c162700 1 -- 172.21.15.2:0/226012451 wait complete. 2023-08-25T03:35:12.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.246+0000 7fda6c162700 1 Processor -- start 2023-08-25T03:35:12.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.246+0000 7fda6c162700 1 -- start start 2023-08-25T03:35:12.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.247+0000 7fda6c162700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fda50004850 0x7fda641565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:12.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.247+0000 7fda6c162700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda640ae270 0x7fda641591d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:12.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.247+0000 7fda6c162700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda640ae6f0 0x7fda64159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:12.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.247+0000 7fda6c162700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda64156ae0 con 0x7fda640ae6f0 2023-08-25T03:35:12.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.247+0000 7fda6c162700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda64158800 con 0x7fda640ae270 2023-08-25T03:35:12.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.247+0000 7fda6c162700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda641f60d0 con 0x7fda50004850 2023-08-25T03:35:12.258 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.247+0000 7fda6a6ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda640ae6f0 0x7fda64159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:12.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.247+0000 7fda6a6ff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda640ae6f0 0x7fda64159710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33228/0 (socket says 172.21.15.2:33228) 2023-08-25T03:35:12.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.247+0000 7fda6a6ff700 1 -- 172.21.15.2:0/2843938865 learned_addr learned my addr 172.21.15.2:0/2843938865 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:12.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.247+0000 7fda69efe700 1 --2- 172.21.15.2:0/2843938865 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fda50004850 0x7fda641565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:12.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.247+0000 7fda696fd700 1 --2- 172.21.15.2:0/2843938865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda640ae270 0x7fda641591d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:12.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.247+0000 7fda6a6ff700 1 -- 172.21.15.2:0/2843938865 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fda50004850 msgr2=0x7fda641565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:12.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.247+0000 7fda6a6ff700 1 --2- 172.21.15.2:0/2843938865 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fda50004850 0x7fda641565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:12.259 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.247+0000 7fda6a6ff700 1 -- 172.21.15.2:0/2843938865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda640ae270 msgr2=0x7fda641591d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:12.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.247+0000 7fda6a6ff700 1 --2- 172.21.15.2:0/2843938865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda640ae270 0x7fda641591d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:12.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.247+0000 7fda6a6ff700 1 -- 172.21.15.2:0/2843938865 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda641f30e0 con 0x7fda640ae6f0 2023-08-25T03:35:12.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.247+0000 7fda6a6ff700 1 --2- 172.21.15.2:0/2843938865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda640ae6f0 0x7fda64159710 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fda640cc940 tx=0x7fda5c00d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:12.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.248+0000 7fda567fc700 1 -- 172.21.15.2:0/2843938865 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c0020b0 con 0x7fda640ae6f0 2023-08-25T03:35:12.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.248+0000 7fda567fc700 1 -- 172.21.15.2:0/2843938865 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5c003c10 con 0x7fda640ae6f0 2023-08-25T03:35:12.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.248+0000 7fda6c162700 1 -- 172.21.15.2:0/2843938865 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda641f33c0 con 0x7fda640ae6f0 2023-08-25T03:35:12.260 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.248+0000 7fda567fc700 1 -- 172.21.15.2:0/2843938865 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c00d990 con 0x7fda640ae6f0 2023-08-25T03:35:12.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.248+0000 7fda6c162700 1 -- 172.21.15.2:0/2843938865 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda641f3980 con 0x7fda640ae6f0 2023-08-25T03:35:12.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.249+0000 7fda6c162700 1 -- 172.21.15.2:0/2843938865 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda64066f70 con 0x7fda640ae6f0 2023-08-25T03:35:12.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.249+0000 7fda567fc700 1 -- 172.21.15.2:0/2843938865 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fda5c00daf0 con 0x7fda640ae6f0 2023-08-25T03:35:12.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.249+0000 7fda567fc700 1 --2- 172.21.15.2:0/2843938865 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fda4806a0e0 0x7fda4806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:12.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.249+0000 7fda567fc700 1 -- 172.21.15.2:0/2843938865 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fda5c088500 con 0x7fda640ae6f0 2023-08-25T03:35:12.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.249+0000 7fda69efe700 1 --2- 172.21.15.2:0/2843938865 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fda4806a0e0 0x7fda4806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:12.261 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.250+0000 7fda69efe700 1 --2- 172.21.15.2:0/2843938865 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fda4806a0e0 0x7fda4806c5a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fda60009ce0 tx=0x7fda60009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:12.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.254+0000 7fda567fc700 1 -- 172.21.15.2:0/2843938865 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fda5c053c80 con 0x7fda640ae6f0 2023-08-25T03:35:12.450 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.450+0000 7fda6c162700 1 -- 172.21.15.2:0/2843938865 --> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fda641f3b10 con 0x7fda4806a0e0 2023-08-25T03:35:12.459 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.458+0000 7fda567fc700 1 -- 172.21.15.2:0/2843938865 <== mgr.4108 v2:172.21.15.2:6832/3528779779 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311363 (secure 0 0 0) 0x7fda641f3b10 con 0x7fda4806a0e0 2023-08-25T03:35:12.465 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:35:12.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.462+0000 7fda6c162700 1 -- 172.21.15.2:0/2843938865 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fda4806a0e0 msgr2=0x7fda4806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:12.467 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.462+0000 7fda6c162700 1 --2- 172.21.15.2:0/2843938865 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fda4806a0e0 0x7fda4806c5a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fda60009ce0 tx=0x7fda60009210 comp rx=0 tx=0).stop 2023-08-25T03:35:12.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.462+0000 7fda6c162700 1 -- 172.21.15.2:0/2843938865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda640ae6f0 msgr2=0x7fda64159710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:12.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.462+0000 7fda6c162700 1 --2- 172.21.15.2:0/2843938865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda640ae6f0 0x7fda64159710 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fda640cc940 tx=0x7fda5c00d430 comp rx=0 tx=0).stop 2023-08-25T03:35:12.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.462+0000 7fda6c162700 1 -- 172.21.15.2:0/2843938865 shutdown_connections 2023-08-25T03:35:12.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.462+0000 7fda6c162700 1 --2- 172.21.15.2:0/2843938865 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fda4806a0e0 0x7fda4806c5a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:12.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.462+0000 7fda6c162700 1 --2- 172.21.15.2:0/2843938865 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fda640ae6f0 0x7fda64159710 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:12.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.462+0000 7fda6c162700 1 --2- 172.21.15.2:0/2843938865 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fda50004850 0x7fda641565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:12.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.462+0000 7fda6c162700 1 --2- 172.21.15.2:0/2843938865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda640ae270 0x7fda641591d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:12.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.462+0000 7fda6c162700 1 -- 172.21.15.2:0/2843938865 >> 172.21.15.2:0/2843938865 conn(0x7fda640b1cf0 msgr2=0x7fda640afa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:12.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.462+0000 7fda6c162700 1 -- 172.21.15.2:0/2843938865 shutdown_connections 2023-08-25T03:35:12.469 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.462+0000 7fda6c162700 1 -- 172.21.15.2:0/2843938865 wait complete. 2023-08-25T03:35:12.469 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-08-25T03:35:12.475 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":72,"stamp":"2023-08-25T03:35:11.518330+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592708,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":201,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220584,"kb_used_data":4408,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754754136,"statfs":{"total":773094113280,"available":772868235264,"internally_reserved":0,"allocated":4513792,"data_stored":2531227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2340,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.617802"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430662+0000","last_change":"2023-08-25T03:35:07.427014+0000","last_active":"2023-08-25T03:35:07.430662+0000","last_peered":"2023-08-25T03:35:07.430662+0000","last_clean":"2023-08-25T03:35:07.430662+0000","last_became_active":"2023-08-25T03:35:07.426372+0000","last_became_peered":"2023-08-25T03:35:07.426372+0000","last_unstale":"2023-08-25T03:35:07.430662+0000","last_undegraded":"2023-08-25T03:35:07.430662+0000","last_fullsized":"2023-08-25T03:35:07.430662+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:12:41.463967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.426648+0000","last_change":"2023-08-25T03:35:07.425122+0000","last_active":"2023-08-25T03:35:07.426648+0000","last_peered":"2023-08-25T03:35:07.426648+0000","last_clean":"2023-08-25T03:35:07.426648+0000","last_became_active":"2023-08-25T03:35:07.424562+0000","last_became_peered":"2023-08-25T03:35:07.424562+0000","last_unstale":"2023-08-25T03:35:07.426648+0000","last_undegraded":"2023-08-25T03:35:07.426648+0000","last_fullsized":"2023-08-25T03:35:07.426648+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:37:27.118343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430322+0000","last_change":"2023-08-25T03:35:07.427639+0000","last_active":"2023-08-25T03:35:07.430322+0000","last_peered":"2023-08-25T03:35:07.430322+0000","last_clean":"2023-08-25T03:35:07.430322+0000","last_became_active":"2023-08-25T03:35:07.427398+0000","last_became_peered":"2023-08-25T03:35:07.427398+0000","last_unstale":"2023-08-25T03:35:07.430322+0000","last_undegraded":"2023-08-25T03:35:07.430322+0000","last_fullsized":"2023-08-25T03:35:07.430322+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:41:58.994138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430603+0000","last_change":"2023-08-25T03:35:07.428463+0000","last_active":"2023-08-25T03:35:07.430603+0000","last_peered":"2023-08-25T03:35:07.430603+0000","last_clean":"2023-08-25T03:35:07.430603+0000","last_became_active":"2023-08-25T03:35:07.428033+0000","last_became_peered":"2023-08-25T03:35:07.428033+0000","last_unstale":"2023-08-25T03:35:07.430603+0000","last_undegraded":"2023-08-25T03:35:07.430603+0000","last_fullsized":"2023-08-25T03:35:07.430603+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:39:17.858453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.422489+0000","last_change":"2023-08-25T03:35:07.421403+0000","last_active":"2023-08-25T03:35:07.422489+0000","last_peered":"2023-08-25T03:35:07.422489+0000","last_clean":"2023-08-25T03:35:07.422489+0000","last_became_active":"2023-08-25T03:35:07.421108+0000","last_became_peered":"2023-08-25T03:35:07.421108+0000","last_unstale":"2023-08-25T03:35:07.422489+0000","last_undegraded":"2023-08-25T03:35:07.422489+0000","last_fullsized":"2023-08-25T03:35:07.422489+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T03:54:02.006558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430320+0000","last_change":"2023-08-25T03:35:07.425989+0000","last_active":"2023-08-25T03:35:07.430320+0000","last_peered":"2023-08-25T03:35:07.430320+0000","last_clean":"2023-08-25T03:35:07.430320+0000","last_became_active":"2023-08-25T03:35:07.425684+0000","last_became_peered":"2023-08-25T03:35:07.425684+0000","last_unstale":"2023-08-25T03:35:07.430320+0000","last_undegraded":"2023-08-25T03:35:07.430320+0000","last_fullsized":"2023-08-25T03:35:07.430320+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:58:52.102133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.416357+0000","last_change":"2023-08-25T03:35:07.429613+0000","last_active":"2023-08-25T03:35:08.416357+0000","last_peered":"2023-08-25T03:35:08.416357+0000","last_clean":"2023-08-25T03:35:08.416357+0000","last_became_active":"2023-08-25T03:35:07.429299+0000","last_became_peered":"2023-08-25T03:35:07.429299+0000","last_unstale":"2023-08-25T03:35:08.416357+0000","last_undegraded":"2023-08-25T03:35:08.416357+0000","last_fullsized":"2023-08-25T03:35:08.416357+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:56:34.245129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430474+0000","last_change":"2023-08-25T03:35:07.425856+0000","last_active":"2023-08-25T03:35:07.430474+0000","last_peered":"2023-08-25T03:35:07.430474+0000","last_clean":"2023-08-25T03:35:07.430474+0000","last_became_active":"2023-08-25T03:35:07.425510+0000","last_became_peered":"2023-08-25T03:35:07.425510+0000","last_unstale":"2023-08-25T03:35:07.430474+0000","last_undegraded":"2023-08-25T03:35:07.430474+0000","last_fullsized":"2023-08-25T03:35:07.430474+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:40:02.227089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430267+0000","last_change":"2023-08-25T03:35:07.427338+0000","last_active":"2023-08-25T03:35:07.430267+0000","last_peered":"2023-08-25T03:35:07.430267+0000","last_clean":"2023-08-25T03:35:07.430267+0000","last_became_active":"2023-08-25T03:35:07.427025+0000","last_became_peered":"2023-08-25T03:35:07.427025+0000","last_unstale":"2023-08-25T03:35:07.430267+0000","last_undegraded":"2023-08-25T03:35:07.430267+0000","last_fullsized":"2023-08-25T03:35:07.430267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:35:53.452778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417787+0000","last_change":"2023-08-25T03:35:07.429618+0000","last_active":"2023-08-25T03:35:08.417787+0000","last_peered":"2023-08-25T03:35:08.417787+0000","last_clean":"2023-08-25T03:35:08.417787+0000","last_became_active":"2023-08-25T03:35:07.429255+0000","last_became_peered":"2023-08-25T03:35:07.429255+0000","last_unstale":"2023-08-25T03:35:08.417787+0000","last_undegraded":"2023-08-25T03:35:08.417787+0000","last_fullsized":"2023-08-25T03:35:08.417787+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:32:48.908546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417769+0000","last_change":"2023-08-25T03:35:07.422337+0000","last_active":"2023-08-25T03:35:08.417769+0000","last_peered":"2023-08-25T03:35:08.417769+0000","last_clean":"2023-08-25T03:35:08.417769+0000","last_became_active":"2023-08-25T03:35:07.421937+0000","last_became_peered":"2023-08-25T03:35:07.421937+0000","last_unstale":"2023-08-25T03:35:08.417769+0000","last_undegraded":"2023-08-25T03:35:08.417769+0000","last_fullsized":"2023-08-25T03:35:08.417769+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:58:47.877253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.419362+0000","last_change":"2023-08-25T03:35:07.417162+0000","last_active":"2023-08-25T03:35:07.419362+0000","last_peered":"2023-08-25T03:35:07.419362+0000","last_clean":"2023-08-25T03:35:07.419362+0000","last_became_active":"2023-08-25T03:35:07.416924+0000","last_became_peered":"2023-08-25T03:35:07.416924+0000","last_unstale":"2023-08-25T03:35:07.419362+0000","last_undegraded":"2023-08-25T03:35:07.419362+0000","last_fullsized":"2023-08-25T03:35:07.419362+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:08:24.760786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.416315+0000","last_change":"2023-08-25T03:35:07.426563+0000","last_active":"2023-08-25T03:35:08.416315+0000","last_peered":"2023-08-25T03:35:08.416315+0000","last_clean":"2023-08-25T03:35:08.416315+0000","last_became_active":"2023-08-25T03:35:07.426176+0000","last_became_peered":"2023-08-25T03:35:07.426176+0000","last_unstale":"2023-08-25T03:35:08.416315+0000","last_undegraded":"2023-08-25T03:35:08.416315+0000","last_fullsized":"2023-08-25T03:35:08.416315+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:23:41.853641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.876115+0000","last_change":"2023-08-25T03:35:07.428844+0000","last_active":"2023-08-25T03:35:08.876115+0000","last_peered":"2023-08-25T03:35:08.876115+0000","last_clean":"2023-08-25T03:35:08.876115+0000","last_became_active":"2023-08-25T03:35:07.428140+0000","last_became_peered":"2023-08-25T03:35:07.428140+0000","last_unstale":"2023-08-25T03:35:08.876115+0000","last_undegraded":"2023-08-25T03:35:08.876115+0000","last_fullsized":"2023-08-25T03:35:08.876115+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:25:51.954228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.416549+0000","last_change":"2023-08-25T03:35:07.417780+0000","last_active":"2023-08-25T03:35:08.416549+0000","last_peered":"2023-08-25T03:35:08.416549+0000","last_clean":"2023-08-25T03:35:08.416549+0000","last_became_active":"2023-08-25T03:35:07.417133+0000","last_became_peered":"2023-08-25T03:35:07.417133+0000","last_unstale":"2023-08-25T03:35:08.416549+0000","last_undegraded":"2023-08-25T03:35:08.416549+0000","last_fullsized":"2023-08-25T03:35:08.416549+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:25:51.954228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.437799+0000","last_change":"2023-08-25T03:35:07.433438+0000","last_active":"2023-08-25T03:35:07.437799+0000","last_peered":"2023-08-25T03:35:07.437799+0000","last_clean":"2023-08-25T03:35:07.437799+0000","last_became_active":"2023-08-25T03:35:07.433155+0000","last_became_peered":"2023-08-25T03:35:07.433155+0000","last_unstale":"2023-08-25T03:35:07.437799+0000","last_undegraded":"2023-08-25T03:35:07.437799+0000","last_fullsized":"2023-08-25T03:35:07.437799+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:54:18.712932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.440381+0000","last_change":"2023-08-25T03:35:07.435773+0000","last_active":"2023-08-25T03:35:07.440381+0000","last_peered":"2023-08-25T03:35:07.440381+0000","last_clean":"2023-08-25T03:35:07.440381+0000","last_became_active":"2023-08-25T03:35:07.435473+0000","last_became_peered":"2023-08-25T03:35:07.435473+0000","last_unstale":"2023-08-25T03:35:07.440381+0000","last_undegraded":"2023-08-25T03:35:07.440381+0000","last_fullsized":"2023-08-25T03:35:07.440381+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:38:14.586034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417469+0000","last_change":"2023-08-25T03:35:07.426305+0000","last_active":"2023-08-25T03:35:08.417469+0000","last_peered":"2023-08-25T03:35:08.417469+0000","last_clean":"2023-08-25T03:35:08.417469+0000","last_became_active":"2023-08-25T03:35:07.425265+0000","last_became_peered":"2023-08-25T03:35:07.425265+0000","last_unstale":"2023-08-25T03:35:08.417469+0000","last_undegraded":"2023-08-25T03:35:08.417469+0000","last_fullsized":"2023-08-25T03:35:08.417469+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:24:45.526607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.418554+0000","last_change":"2023-08-25T03:35:07.416345+0000","last_active":"2023-08-25T03:35:07.418554+0000","last_peered":"2023-08-25T03:35:07.418554+0000","last_clean":"2023-08-25T03:35:07.418554+0000","last_became_active":"2023-08-25T03:35:07.416085+0000","last_became_peered":"2023-08-25T03:35:07.416085+0000","last_unstale":"2023-08-25T03:35:07.418554+0000","last_undegraded":"2023-08-25T03:35:07.418554+0000","last_fullsized":"2023-08-25T03:35:07.418554+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:52:49.107239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.416708+0000","last_change":"2023-08-25T03:35:07.428310+0000","last_active":"2023-08-25T03:35:08.416708+0000","last_peered":"2023-08-25T03:35:08.416708+0000","last_clean":"2023-08-25T03:35:08.416708+0000","last_became_active":"2023-08-25T03:35:07.427915+0000","last_became_peered":"2023-08-25T03:35:07.427915+0000","last_unstale":"2023-08-25T03:35:08.416708+0000","last_undegraded":"2023-08-25T03:35:08.416708+0000","last_fullsized":"2023-08-25T03:35:08.416708+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:15:17.522640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417177+0000","last_change":"2023-08-25T03:35:07.418389+0000","last_active":"2023-08-25T03:35:08.417177+0000","last_peered":"2023-08-25T03:35:08.417177+0000","last_clean":"2023-08-25T03:35:08.417177+0000","last_became_active":"2023-08-25T03:35:07.416998+0000","last_became_peered":"2023-08-25T03:35:07.416998+0000","last_unstale":"2023-08-25T03:35:08.417177+0000","last_undegraded":"2023-08-25T03:35:08.417177+0000","last_fullsized":"2023-08-25T03:35:08.417177+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:48:29.838778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.876410+0000","last_change":"2023-08-25T03:35:07.422028+0000","last_active":"2023-08-25T03:35:08.876410+0000","last_peered":"2023-08-25T03:35:08.876410+0000","last_clean":"2023-08-25T03:35:08.876410+0000","last_became_active":"2023-08-25T03:35:07.421536+0000","last_became_peered":"2023-08-25T03:35:07.421536+0000","last_unstale":"2023-08-25T03:35:08.876410+0000","last_undegraded":"2023-08-25T03:35:08.876410+0000","last_fullsized":"2023-08-25T03:35:08.876410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:47:23.984917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.435316+0000","last_change":"2023-08-25T03:35:07.430813+0000","last_active":"2023-08-25T03:35:07.435316+0000","last_peered":"2023-08-25T03:35:07.435316+0000","last_clean":"2023-08-25T03:35:07.435316+0000","last_became_active":"2023-08-25T03:35:07.430512+0000","last_became_peered":"2023-08-25T03:35:07.430512+0000","last_unstale":"2023-08-25T03:35:07.435316+0000","last_undegraded":"2023-08-25T03:35:07.435316+0000","last_fullsized":"2023-08-25T03:35:07.435316+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:10:43.180614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.418527+0000","last_change":"2023-08-25T03:35:07.416291+0000","last_active":"2023-08-25T03:35:07.418527+0000","last_peered":"2023-08-25T03:35:07.418527+0000","last_clean":"2023-08-25T03:35:07.418527+0000","last_became_active":"2023-08-25T03:35:07.416004+0000","last_became_peered":"2023-08-25T03:35:07.416004+0000","last_unstale":"2023-08-25T03:35:07.418527+0000","last_undegraded":"2023-08-25T03:35:07.418527+0000","last_fullsized":"2023-08-25T03:35:07.418527+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:10:41.985896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417635+0000","last_change":"2023-08-25T03:35:07.420875+0000","last_active":"2023-08-25T03:35:08.417635+0000","last_peered":"2023-08-25T03:35:08.417635+0000","last_clean":"2023-08-25T03:35:08.417635+0000","last_became_active":"2023-08-25T03:35:07.420415+0000","last_became_peered":"2023-08-25T03:35:07.420415+0000","last_unstale":"2023-08-25T03:35:08.417635+0000","last_undegraded":"2023-08-25T03:35:08.417635+0000","last_fullsized":"2023-08-25T03:35:08.417635+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:04:02.903147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.420453+0000","last_change":"2023-08-25T03:35:07.419448+0000","last_active":"2023-08-25T03:35:07.420453+0000","last_peered":"2023-08-25T03:35:07.420453+0000","last_clean":"2023-08-25T03:35:07.420453+0000","last_became_active":"2023-08-25T03:35:07.419122+0000","last_became_peered":"2023-08-25T03:35:07.419122+0000","last_unstale":"2023-08-25T03:35:07.420453+0000","last_undegraded":"2023-08-25T03:35:07.420453+0000","last_fullsized":"2023-08-25T03:35:07.420453+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:35:53.452778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417200+0000","last_change":"2023-08-25T03:35:07.421488+0000","last_active":"2023-08-25T03:35:08.417200+0000","last_peered":"2023-08-25T03:35:08.417200+0000","last_clean":"2023-08-25T03:35:08.417200+0000","last_became_active":"2023-08-25T03:35:07.419932+0000","last_became_peered":"2023-08-25T03:35:07.419932+0000","last_unstale":"2023-08-25T03:35:08.417200+0000","last_undegraded":"2023-08-25T03:35:08.417200+0000","last_fullsized":"2023-08-25T03:35:08.417200+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:18:15.305693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417142+0000","last_change":"2023-08-25T03:35:07.426011+0000","last_active":"2023-08-25T03:35:08.417142+0000","last_peered":"2023-08-25T03:35:08.417142+0000","last_clean":"2023-08-25T03:35:08.417142+0000","last_became_active":"2023-08-25T03:35:07.425681+0000","last_became_peered":"2023-08-25T03:35:07.425681+0000","last_unstale":"2023-08-25T03:35:08.417142+0000","last_undegraded":"2023-08-25T03:35:08.417142+0000","last_fullsized":"2023-08-25T03:35:08.417142+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:47:50.765716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417043+0000","last_change":"2023-08-25T03:35:07.420451+0000","last_active":"2023-08-25T03:35:08.417043+0000","last_peered":"2023-08-25T03:35:08.417043+0000","last_clean":"2023-08-25T03:35:08.417043+0000","last_became_active":"2023-08-25T03:35:07.419829+0000","last_became_peered":"2023-08-25T03:35:07.419829+0000","last_unstale":"2023-08-25T03:35:08.417043+0000","last_undegraded":"2023-08-25T03:35:08.417043+0000","last_fullsized":"2023-08-25T03:35:08.417043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:11:43.194516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.416841+0000","last_change":"2023-08-25T03:35:07.426030+0000","last_active":"2023-08-25T03:35:08.416841+0000","last_peered":"2023-08-25T03:35:08.416841+0000","last_clean":"2023-08-25T03:35:08.416841+0000","last_became_active":"2023-08-25T03:35:07.425704+0000","last_became_peered":"2023-08-25T03:35:07.425704+0000","last_unstale":"2023-08-25T03:35:08.416841+0000","last_undegraded":"2023-08-25T03:35:08.416841+0000","last_fullsized":"2023-08-25T03:35:08.416841+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:53:41.033308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430167+0000","last_change":"2023-08-25T03:35:07.428547+0000","last_active":"2023-08-25T03:35:07.430167+0000","last_peered":"2023-08-25T03:35:07.430167+0000","last_clean":"2023-08-25T03:35:07.430167+0000","last_became_active":"2023-08-25T03:35:07.428140+0000","last_became_peered":"2023-08-25T03:35:07.428140+0000","last_unstale":"2023-08-25T03:35:07.430167+0000","last_undegraded":"2023-08-25T03:35:07.430167+0000","last_fullsized":"2023-08-25T03:35:07.430167+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:31:11.022575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.419854+0000","last_change":"2023-08-25T03:35:07.417690+0000","last_active":"2023-08-25T03:35:07.419854+0000","last_peered":"2023-08-25T03:35:07.419854+0000","last_clean":"2023-08-25T03:35:07.419854+0000","last_became_active":"2023-08-25T03:35:07.417416+0000","last_became_peered":"2023-08-25T03:35:07.417416+0000","last_unstale":"2023-08-25T03:35:07.419854+0000","last_undegraded":"2023-08-25T03:35:07.419854+0000","last_fullsized":"2023-08-25T03:35:07.419854+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:26:39.801388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.437822+0000","last_change":"2023-08-25T03:35:07.433922+0000","last_active":"2023-08-25T03:35:07.437822+0000","last_peered":"2023-08-25T03:35:07.437822+0000","last_clean":"2023-08-25T03:35:07.437822+0000","last_became_active":"2023-08-25T03:35:07.433609+0000","last_became_peered":"2023-08-25T03:35:07.433609+0000","last_unstale":"2023-08-25T03:35:07.437822+0000","last_undegraded":"2023-08-25T03:35:07.437822+0000","last_fullsized":"2023-08-25T03:35:07.437822+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:40:02.227089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"36'4","reported_seq":17,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417744+0000","last_change":"2023-08-25T03:35:06.918831+0000","last_active":"2023-08-25T03:35:08.417744+0000","last_peered":"2023-08-25T03:35:08.417744+0000","last_clean":"2023-08-25T03:35:08.417744+0000","last_became_active":"2023-08-25T03:35:06.917482+0000","last_became_peered":"2023-08-25T03:35:06.917482+0000","last_unstale":"2023-08-25T03:35:08.417744+0000","last_undegraded":"2023-08-25T03:35:08.417744+0000","last_fullsized":"2023-08-25T03:35:08.417744+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:52:26.271207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.423101+0000","last_change":"2023-08-25T03:35:07.422242+0000","last_active":"2023-08-25T03:35:07.423101+0000","last_peered":"2023-08-25T03:35:07.423101+0000","last_clean":"2023-08-25T03:35:07.423101+0000","last_became_active":"2023-08-25T03:35:07.421935+0000","last_became_peered":"2023-08-25T03:35:07.421935+0000","last_unstale":"2023-08-25T03:35:07.423101+0000","last_undegraded":"2023-08-25T03:35:07.423101+0000","last_fullsized":"2023-08-25T03:35:07.423101+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:14:36.942982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.930319+0000","last_change":"2023-08-25T03:35:06.925380+0000","last_active":"2023-08-25T03:35:07.930319+0000","last_peered":"2023-08-25T03:35:07.930319+0000","last_clean":"2023-08-25T03:35:07.930319+0000","last_became_active":"2023-08-25T03:35:06.925001+0000","last_became_peered":"2023-08-25T03:35:06.925001+0000","last_unstale":"2023-08-25T03:35:07.930319+0000","last_undegraded":"2023-08-25T03:35:07.930319+0000","last_fullsized":"2023-08-25T03:35:07.930319+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:04:52.463770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.876512+0000","last_change":"2023-08-25T03:35:07.427990+0000","last_active":"2023-08-25T03:35:08.876512+0000","last_peered":"2023-08-25T03:35:08.876512+0000","last_clean":"2023-08-25T03:35:08.876512+0000","last_became_active":"2023-08-25T03:35:07.427701+0000","last_became_peered":"2023-08-25T03:35:07.427701+0000","last_unstale":"2023-08-25T03:35:08.876512+0000","last_undegraded":"2023-08-25T03:35:08.876512+0000","last_fullsized":"2023-08-25T03:35:08.876512+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:42:24.718848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.926845+0000","last_change":"2023-08-25T03:35:06.916315+0000","last_active":"2023-08-25T03:35:07.926845+0000","last_peered":"2023-08-25T03:35:07.926845+0000","last_clean":"2023-08-25T03:35:07.926845+0000","last_became_active":"2023-08-25T03:35:06.916002+0000","last_became_peered":"2023-08-25T03:35:06.916002+0000","last_unstale":"2023-08-25T03:35:07.926845+0000","last_undegraded":"2023-08-25T03:35:07.926845+0000","last_fullsized":"2023-08-25T03:35:07.926845+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:21:21.889508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430685+0000","last_change":"2023-08-25T03:35:07.427178+0000","last_active":"2023-08-25T03:35:07.430685+0000","last_peered":"2023-08-25T03:35:07.430685+0000","last_clean":"2023-08-25T03:35:07.430685+0000","last_became_active":"2023-08-25T03:35:07.426727+0000","last_became_peered":"2023-08-25T03:35:07.426727+0000","last_unstale":"2023-08-25T03:35:07.430685+0000","last_undegraded":"2023-08-25T03:35:07.430685+0000","last_fullsized":"2023-08-25T03:35:07.430685+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:43:45.461082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"36'2","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.876284+0000","last_change":"2023-08-25T03:35:06.922388+0000","last_active":"2023-08-25T03:35:08.876284+0000","last_peered":"2023-08-25T03:35:08.876284+0000","last_clean":"2023-08-25T03:35:08.876284+0000","last_became_active":"2023-08-25T03:35:06.922063+0000","last_became_peered":"2023-08-25T03:35:06.922063+0000","last_unstale":"2023-08-25T03:35:08.876284+0000","last_undegraded":"2023-08-25T03:35:08.876284+0000","last_fullsized":"2023-08-25T03:35:08.876284+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:39:50.273892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.422124+0000","last_change":"2023-08-25T03:35:07.420543+0000","last_active":"2023-08-25T03:35:07.422124+0000","last_peered":"2023-08-25T03:35:07.422124+0000","last_clean":"2023-08-25T03:35:07.422124+0000","last_became_active":"2023-08-25T03:35:07.420308+0000","last_became_peered":"2023-08-25T03:35:07.420308+0000","last_unstale":"2023-08-25T03:35:07.422124+0000","last_undegraded":"2023-08-25T03:35:07.422124+0000","last_fullsized":"2023-08-25T03:35:07.422124+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:21:28.094063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"36'1","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417237+0000","last_change":"2023-08-25T03:35:06.920456+0000","last_active":"2023-08-25T03:35:08.417237+0000","last_peered":"2023-08-25T03:35:08.417237+0000","last_clean":"2023-08-25T03:35:08.417237+0000","last_became_active":"2023-08-25T03:35:06.920158+0000","last_became_peered":"2023-08-25T03:35:06.920158+0000","last_unstale":"2023-08-25T03:35:08.417237+0000","last_undegraded":"2023-08-25T03:35:08.417237+0000","last_fullsized":"2023-08-25T03:35:08.417237+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:47:22.980811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.416483+0000","last_change":"2023-08-25T03:35:07.418017+0000","last_active":"2023-08-25T03:35:08.416483+0000","last_peered":"2023-08-25T03:35:08.416483+0000","last_clean":"2023-08-25T03:35:08.416483+0000","last_became_active":"2023-08-25T03:35:07.417494+0000","last_became_peered":"2023-08-25T03:35:07.417494+0000","last_unstale":"2023-08-25T03:35:08.416483+0000","last_undegraded":"2023-08-25T03:35:08.416483+0000","last_fullsized":"2023-08-25T03:35:08.416483+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:13:28.418703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"36'1","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.925494+0000","last_change":"2023-08-25T03:35:06.924347+0000","last_active":"2023-08-25T03:35:07.925494+0000","last_peered":"2023-08-25T03:35:07.925494+0000","last_clean":"2023-08-25T03:35:07.925494+0000","last_became_active":"2023-08-25T03:35:06.923920+0000","last_became_peered":"2023-08-25T03:35:06.923920+0000","last_unstale":"2023-08-25T03:35:07.925494+0000","last_undegraded":"2023-08-25T03:35:07.925494+0000","last_fullsized":"2023-08-25T03:35:07.925494+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:22:22.400143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.876317+0000","last_change":"2023-08-25T03:35:07.433164+0000","last_active":"2023-08-25T03:35:08.876317+0000","last_peered":"2023-08-25T03:35:08.876317+0000","last_clean":"2023-08-25T03:35:08.876317+0000","last_became_active":"2023-08-25T03:35:07.432832+0000","last_became_peered":"2023-08-25T03:35:07.432832+0000","last_unstale":"2023-08-25T03:35:08.876317+0000","last_undegraded":"2023-08-25T03:35:08.876317+0000","last_fullsized":"2023-08-25T03:35:08.876317+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:45:03.766857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.412909+0000","last_change":"2023-08-25T03:35:06.922664+0000","last_active":"2023-08-25T03:35:07.412909+0000","last_peered":"2023-08-25T03:35:07.412909+0000","last_clean":"2023-08-25T03:35:07.412909+0000","last_became_active":"2023-08-25T03:35:06.922349+0000","last_became_peered":"2023-08-25T03:35:06.922349+0000","last_unstale":"2023-08-25T03:35:07.412909+0000","last_undegraded":"2023-08-25T03:35:07.412909+0000","last_fullsized":"2023-08-25T03:35:07.412909+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:41:57.990032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.426780+0000","last_change":"2023-08-25T03:35:07.424107+0000","last_active":"2023-08-25T03:35:07.426780+0000","last_peered":"2023-08-25T03:35:07.426780+0000","last_clean":"2023-08-25T03:35:07.426780+0000","last_became_active":"2023-08-25T03:35:07.423882+0000","last_became_peered":"2023-08-25T03:35:07.423882+0000","last_unstale":"2023-08-25T03:35:07.426780+0000","last_undegraded":"2023-08-25T03:35:07.426780+0000","last_fullsized":"2023-08-25T03:35:07.426780+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:41:17.471725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.875946+0000","last_change":"2023-08-25T03:35:06.920173+0000","last_active":"2023-08-25T03:35:08.875946+0000","last_peered":"2023-08-25T03:35:08.875946+0000","last_clean":"2023-08-25T03:35:08.875946+0000","last_became_active":"2023-08-25T03:35:06.919830+0000","last_became_peered":"2023-08-25T03:35:06.919830+0000","last_unstale":"2023-08-25T03:35:08.875946+0000","last_undegraded":"2023-08-25T03:35:08.875946+0000","last_fullsized":"2023-08-25T03:35:08.875946+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:56:33.241024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430584+0000","last_change":"2023-08-25T03:35:07.428243+0000","last_active":"2023-08-25T03:35:07.430584+0000","last_peered":"2023-08-25T03:35:07.430584+0000","last_clean":"2023-08-25T03:35:07.430584+0000","last_became_active":"2023-08-25T03:35:07.427749+0000","last_became_peered":"2023-08-25T03:35:07.427749+0000","last_unstale":"2023-08-25T03:35:07.430584+0000","last_undegraded":"2023-08-25T03:35:07.430584+0000","last_fullsized":"2023-08-25T03:35:07.430584+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:38:58.557458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.421286+0000","last_change":"2023-08-25T03:35:06.922296+0000","last_active":"2023-08-25T03:35:07.421286+0000","last_peered":"2023-08-25T03:35:07.421286+0000","last_clean":"2023-08-25T03:35:07.421286+0000","last_became_active":"2023-08-25T03:35:06.921995+0000","last_became_peered":"2023-08-25T03:35:06.921995+0000","last_unstale":"2023-08-25T03:35:07.421286+0000","last_undegraded":"2023-08-25T03:35:07.421286+0000","last_fullsized":"2023-08-25T03:35:07.421286+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:54:46.489267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.435965+0000","last_change":"2023-08-25T03:35:07.431869+0000","last_active":"2023-08-25T03:35:07.435965+0000","last_peered":"2023-08-25T03:35:07.435965+0000","last_clean":"2023-08-25T03:35:07.435965+0000","last_became_active":"2023-08-25T03:35:07.431535+0000","last_became_peered":"2023-08-25T03:35:07.431535+0000","last_unstale":"2023-08-25T03:35:07.435965+0000","last_undegraded":"2023-08-25T03:35:07.435965+0000","last_fullsized":"2023-08-25T03:35:07.435965+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:12:41.463967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417521+0000","last_change":"2023-08-25T03:35:06.921964+0000","last_active":"2023-08-25T03:35:08.417521+0000","last_peered":"2023-08-25T03:35:08.417521+0000","last_clean":"2023-08-25T03:35:08.417521+0000","last_became_active":"2023-08-25T03:35:06.921702+0000","last_became_peered":"2023-08-25T03:35:06.921702+0000","last_unstale":"2023-08-25T03:35:08.417521+0000","last_undegraded":"2023-08-25T03:35:08.417521+0000","last_fullsized":"2023-08-25T03:35:08.417521+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:07:03.573902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430342+0000","last_change":"2023-08-25T03:35:07.428479+0000","last_active":"2023-08-25T03:35:07.430342+0000","last_peered":"2023-08-25T03:35:07.430342+0000","last_clean":"2023-08-25T03:35:07.430342+0000","last_became_active":"2023-08-25T03:35:07.428223+0000","last_became_peered":"2023-08-25T03:35:07.428223+0000","last_unstale":"2023-08-25T03:35:07.430342+0000","last_undegraded":"2023-08-25T03:35:07.430342+0000","last_fullsized":"2023-08-25T03:35:07.430342+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:14:36.942982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"36'3","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417024+0000","last_change":"2023-08-25T03:35:06.918555+0000","last_active":"2023-08-25T03:35:08.417024+0000","last_peered":"2023-08-25T03:35:08.417024+0000","last_clean":"2023-08-25T03:35:08.417024+0000","last_became_active":"2023-08-25T03:35:06.918229+0000","last_became_peered":"2023-08-25T03:35:06.918229+0000","last_unstale":"2023-08-25T03:35:08.417024+0000","last_undegraded":"2023-08-25T03:35:08.417024+0000","last_fullsized":"2023-08-25T03:35:08.417024+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:45:02.762751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.876576+0000","last_change":"2023-08-25T03:35:07.421607+0000","last_active":"2023-08-25T03:35:08.876576+0000","last_peered":"2023-08-25T03:35:08.876576+0000","last_clean":"2023-08-25T03:35:08.876576+0000","last_became_active":"2023-08-25T03:35:07.421279+0000","last_became_peered":"2023-08-25T03:35:07.421279+0000","last_unstale":"2023-08-25T03:35:08.876576+0000","last_undegraded":"2023-08-25T03:35:08.876576+0000","last_fullsized":"2023-08-25T03:35:08.876576+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:26:04.402442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"36'6","reported_seq":17,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.932756+0000","last_change":"2023-08-25T03:35:06.925767+0000","last_active":"2023-08-25T03:35:07.932756+0000","last_peered":"2023-08-25T03:35:07.932756+0000","last_clean":"2023-08-25T03:35:07.932756+0000","last_became_active":"2023-08-25T03:35:06.925490+0000","last_became_peered":"2023-08-25T03:35:06.925490+0000","last_unstale":"2023-08-25T03:35:07.932756+0000","last_undegraded":"2023-08-25T03:35:07.932756+0000","last_fullsized":"2023-08-25T03:35:07.932756+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:58:28.709654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430209+0000","last_change":"2023-08-25T03:35:07.425893+0000","last_active":"2023-08-25T03:35:07.430209+0000","last_peered":"2023-08-25T03:35:07.430209+0000","last_clean":"2023-08-25T03:35:07.430209+0000","last_became_active":"2023-08-25T03:35:07.425300+0000","last_became_peered":"2023-08-25T03:35:07.425300+0000","last_unstale":"2023-08-25T03:35:07.430209+0000","last_undegraded":"2023-08-25T03:35:07.430209+0000","last_fullsized":"2023-08-25T03:35:07.430209+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:24:53.465344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"36'2","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.922754+0000","last_change":"2023-08-25T03:35:06.924446+0000","last_active":"2023-08-25T03:35:07.922754+0000","last_peered":"2023-08-25T03:35:07.922754+0000","last_clean":"2023-08-25T03:35:07.922754+0000","last_became_active":"2023-08-25T03:35:06.924012+0000","last_became_peered":"2023-08-25T03:35:06.924012+0000","last_unstale":"2023-08-25T03:35:07.922754+0000","last_undegraded":"2023-08-25T03:35:07.922754+0000","last_fullsized":"2023-08-25T03:35:07.922754+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:38:13.581929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.429266+0000","last_change":"2023-08-25T03:35:07.425215+0000","last_active":"2023-08-25T03:35:07.429266+0000","last_peered":"2023-08-25T03:35:07.429266+0000","last_clean":"2023-08-25T03:35:07.429266+0000","last_became_active":"2023-08-25T03:35:07.424627+0000","last_became_peered":"2023-08-25T03:35:07.424627+0000","last_unstale":"2023-08-25T03:35:07.429266+0000","last_undegraded":"2023-08-25T03:35:07.429266+0000","last_fullsized":"2023-08-25T03:35:07.429266+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:54:18.712932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.413379+0000","last_change":"2023-08-25T03:35:06.921196+0000","last_active":"2023-08-25T03:35:07.413379+0000","last_peered":"2023-08-25T03:35:07.413379+0000","last_clean":"2023-08-25T03:35:07.413379+0000","last_became_active":"2023-08-25T03:35:06.920899+0000","last_became_peered":"2023-08-25T03:35:06.920899+0000","last_unstale":"2023-08-25T03:35:07.413379+0000","last_undegraded":"2023-08-25T03:35:07.413379+0000","last_fullsized":"2023-08-25T03:35:07.413379+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:41:16.467620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.433621+0000","last_change":"2023-08-25T03:35:07.429964+0000","last_active":"2023-08-25T03:35:07.433621+0000","last_peered":"2023-08-25T03:35:07.433621+0000","last_clean":"2023-08-25T03:35:07.433621+0000","last_became_active":"2023-08-25T03:35:07.429680+0000","last_became_peered":"2023-08-25T03:35:07.429680+0000","last_unstale":"2023-08-25T03:35:07.433621+0000","last_undegraded":"2023-08-25T03:35:07.433621+0000","last_fullsized":"2023-08-25T03:35:07.433621+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:58:29.713760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.416779+0000","last_change":"2023-08-25T03:35:06.917702+0000","last_active":"2023-08-25T03:35:08.416779+0000","last_peered":"2023-08-25T03:35:08.416779+0000","last_clean":"2023-08-25T03:35:08.416779+0000","last_became_active":"2023-08-25T03:35:06.917384+0000","last_became_peered":"2023-08-25T03:35:06.917384+0000","last_unstale":"2023-08-25T03:35:08.416779+0000","last_undegraded":"2023-08-25T03:35:08.416779+0000","last_fullsized":"2023-08-25T03:35:08.416779+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:08:01.715495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.429294+0000","last_change":"2023-08-25T03:35:07.425308+0000","last_active":"2023-08-25T03:35:07.429294+0000","last_peered":"2023-08-25T03:35:07.429294+0000","last_clean":"2023-08-25T03:35:07.429294+0000","last_became_active":"2023-08-25T03:35:07.424532+0000","last_became_peered":"2023-08-25T03:35:07.424532+0000","last_unstale":"2023-08-25T03:35:07.429294+0000","last_undegraded":"2023-08-25T03:35:07.429294+0000","last_fullsized":"2023-08-25T03:35:07.429294+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:01:29.759578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"36'8","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.931047+0000","last_change":"2023-08-25T03:35:06.925794+0000","last_active":"2023-08-25T03:35:07.931047+0000","last_peered":"2023-08-25T03:35:07.931047+0000","last_clean":"2023-08-25T03:35:07.931047+0000","last_became_active":"2023-08-25T03:35:06.925517+0000","last_became_peered":"2023-08-25T03:35:06.925517+0000","last_unstale":"2023-08-25T03:35:07.931047+0000","last_undegraded":"2023-08-25T03:35:07.931047+0000","last_fullsized":"2023-08-25T03:35:07.931047+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:43:44.456976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430595+0000","last_change":"2023-08-25T03:35:07.426813+0000","last_active":"2023-08-25T03:35:07.430595+0000","last_peered":"2023-08-25T03:35:07.430595+0000","last_clean":"2023-08-25T03:35:07.430595+0000","last_became_active":"2023-08-25T03:35:07.426491+0000","last_became_peered":"2023-08-25T03:35:07.426491+0000","last_unstale":"2023-08-25T03:35:07.430595+0000","last_undegraded":"2023-08-25T03:35:07.430595+0000","last_fullsized":"2023-08-25T03:35:07.430595+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:24:53.465344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"36'1","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417556+0000","last_change":"2023-08-25T03:35:06.919030+0000","last_active":"2023-08-25T03:35:08.417556+0000","last_peered":"2023-08-25T03:35:08.417556+0000","last_clean":"2023-08-25T03:35:08.417556+0000","last_became_active":"2023-08-25T03:35:06.918711+0000","last_became_peered":"2023-08-25T03:35:06.918711+0000","last_unstale":"2023-08-25T03:35:08.417556+0000","last_undegraded":"2023-08-25T03:35:08.417556+0000","last_fullsized":"2023-08-25T03:35:08.417556+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:45:31.400474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417457+0000","last_change":"2023-08-25T03:35:07.420169+0000","last_active":"2023-08-25T03:35:08.417457+0000","last_peered":"2023-08-25T03:35:08.417457+0000","last_clean":"2023-08-25T03:35:08.417457+0000","last_became_active":"2023-08-25T03:35:07.419856+0000","last_became_peered":"2023-08-25T03:35:07.419856+0000","last_unstale":"2023-08-25T03:35:08.417457+0000","last_undegraded":"2023-08-25T03:35:08.417457+0000","last_fullsized":"2023-08-25T03:35:08.417457+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:42:24.718848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"36'2","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.876400+0000","last_change":"2023-08-25T03:35:06.919217+0000","last_active":"2023-08-25T03:35:08.876400+0000","last_peered":"2023-08-25T03:35:08.876400+0000","last_clean":"2023-08-25T03:35:08.876400+0000","last_became_active":"2023-08-25T03:35:06.918932+0000","last_became_peered":"2023-08-25T03:35:06.918932+0000","last_unstale":"2023-08-25T03:35:08.876400+0000","last_undegraded":"2023-08-25T03:35:08.876400+0000","last_fullsized":"2023-08-25T03:35:08.876400+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:48:28.834672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.876354+0000","last_change":"2023-08-25T03:35:07.422467+0000","last_active":"2023-08-25T03:35:08.876354+0000","last_peered":"2023-08-25T03:35:08.876354+0000","last_clean":"2023-08-25T03:35:08.876354+0000","last_became_active":"2023-08-25T03:35:07.421892+0000","last_became_peered":"2023-08-25T03:35:07.421892+0000","last_unstale":"2023-08-25T03:35:08.876354+0000","last_undegraded":"2023-08-25T03:35:08.876354+0000","last_fullsized":"2023-08-25T03:35:08.876354+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:47:50.765716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.410579+0000","last_change":"2023-08-25T03:35:06.925867+0000","last_active":"2023-08-25T03:35:07.410579+0000","last_peered":"2023-08-25T03:35:07.410579+0000","last_clean":"2023-08-25T03:35:07.410579+0000","last_became_active":"2023-08-25T03:35:06.925536+0000","last_became_peered":"2023-08-25T03:35:06.925536+0000","last_unstale":"2023-08-25T03:35:07.410579+0000","last_undegraded":"2023-08-25T03:35:07.410579+0000","last_fullsized":"2023-08-25T03:35:07.410579+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:21:58.955199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430830+0000","last_change":"2023-08-25T03:35:07.428493+0000","last_active":"2023-08-25T03:35:07.430830+0000","last_peered":"2023-08-25T03:35:07.430830+0000","last_clean":"2023-08-25T03:35:07.430830+0000","last_became_active":"2023-08-25T03:35:07.428211+0000","last_became_peered":"2023-08-25T03:35:07.428211+0000","last_unstale":"2023-08-25T03:35:07.430830+0000","last_undegraded":"2023-08-25T03:35:07.430830+0000","last_fullsized":"2023-08-25T03:35:07.430830+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:51:59.147211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.413214+0000","last_change":"2023-08-25T03:35:06.919498+0000","last_active":"2023-08-25T03:35:07.413214+0000","last_peered":"2023-08-25T03:35:07.413214+0000","last_clean":"2023-08-25T03:35:07.413214+0000","last_became_active":"2023-08-25T03:35:06.919235+0000","last_became_peered":"2023-08-25T03:35:06.919235+0000","last_unstale":"2023-08-25T03:35:07.413214+0000","last_undegraded":"2023-08-25T03:35:07.413214+0000","last_fullsized":"2023-08-25T03:35:07.413214+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:15:18.138334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.435349+0000","last_change":"2023-08-25T03:35:07.430947+0000","last_active":"2023-08-25T03:35:07.435349+0000","last_peered":"2023-08-25T03:35:07.435349+0000","last_clean":"2023-08-25T03:35:07.435349+0000","last_became_active":"2023-08-25T03:35:07.430634+0000","last_became_peered":"2023-08-25T03:35:07.430634+0000","last_unstale":"2023-08-25T03:35:07.435349+0000","last_undegraded":"2023-08-25T03:35:07.435349+0000","last_fullsized":"2023-08-25T03:35:07.435349+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:38:37.183111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.410741+0000","last_change":"2023-08-25T03:35:06.923471+0000","last_active":"2023-08-25T03:35:07.410741+0000","last_peered":"2023-08-25T03:35:07.410741+0000","last_clean":"2023-08-25T03:35:07.410741+0000","last_became_active":"2023-08-25T03:35:06.923065+0000","last_became_peered":"2023-08-25T03:35:06.923065+0000","last_unstale":"2023-08-25T03:35:07.410741+0000","last_undegraded":"2023-08-25T03:35:07.410741+0000","last_fullsized":"2023-08-25T03:35:07.410741+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:56:24.758139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.422961+0000","last_change":"2023-08-25T03:35:07.422059+0000","last_active":"2023-08-25T03:35:07.422961+0000","last_peered":"2023-08-25T03:35:07.422961+0000","last_clean":"2023-08-25T03:35:07.422961+0000","last_became_active":"2023-08-25T03:35:07.421768+0000","last_became_peered":"2023-08-25T03:35:07.421768+0000","last_unstale":"2023-08-25T03:35:07.422961+0000","last_undegraded":"2023-08-25T03:35:07.422961+0000","last_fullsized":"2023-08-25T03:35:07.422961+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:41:17.471725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.410526+0000","last_change":"2023-08-25T03:35:06.918197+0000","last_active":"2023-08-25T03:35:07.410526+0000","last_peered":"2023-08-25T03:35:07.410526+0000","last_clean":"2023-08-25T03:35:07.410526+0000","last_became_active":"2023-08-25T03:35:06.917871+0000","last_became_peered":"2023-08-25T03:35:06.917871+0000","last_unstale":"2023-08-25T03:35:07.410526+0000","last_undegraded":"2023-08-25T03:35:07.410526+0000","last_fullsized":"2023-08-25T03:35:07.410526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:49:48.227321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.416335+0000","last_change":"2023-08-25T03:35:07.422923+0000","last_active":"2023-08-25T03:35:08.416335+0000","last_peered":"2023-08-25T03:35:08.416335+0000","last_clean":"2023-08-25T03:35:08.416335+0000","last_became_active":"2023-08-25T03:35:07.422608+0000","last_became_peered":"2023-08-25T03:35:07.422608+0000","last_unstale":"2023-08-25T03:35:08.416335+0000","last_undegraded":"2023-08-25T03:35:08.416335+0000","last_fullsized":"2023-08-25T03:35:08.416335+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:39:35.960897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.423940+0000","last_change":"2023-08-25T03:35:06.926686+0000","last_active":"2023-08-25T03:35:07.423940+0000","last_peered":"2023-08-25T03:35:07.423940+0000","last_clean":"2023-08-25T03:35:07.423940+0000","last_became_active":"2023-08-25T03:35:06.926400+0000","last_became_peered":"2023-08-25T03:35:06.926400+0000","last_unstale":"2023-08-25T03:35:07.423940+0000","last_undegraded":"2023-08-25T03:35:07.423940+0000","last_fullsized":"2023-08-25T03:35:07.423940+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:25:10.189227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430550+0000","last_change":"2023-08-25T03:35:07.426694+0000","last_active":"2023-08-25T03:35:07.430550+0000","last_peered":"2023-08-25T03:35:07.430550+0000","last_clean":"2023-08-25T03:35:07.430550+0000","last_became_active":"2023-08-25T03:35:07.426416+0000","last_became_peered":"2023-08-25T03:35:07.426416+0000","last_unstale":"2023-08-25T03:35:07.430550+0000","last_undegraded":"2023-08-25T03:35:07.430550+0000","last_fullsized":"2023-08-25T03:35:07.430550+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:21:28.094063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.410396+0000","last_change":"2023-08-25T03:35:06.923379+0000","last_active":"2023-08-25T03:35:07.410396+0000","last_peered":"2023-08-25T03:35:07.410396+0000","last_clean":"2023-08-25T03:35:07.410396+0000","last_became_active":"2023-08-25T03:35:06.923087+0000","last_became_peered":"2023-08-25T03:35:06.923087+0000","last_unstale":"2023-08-25T03:35:07.410396+0000","last_undegraded":"2023-08-25T03:35:07.410396+0000","last_fullsized":"2023-08-25T03:35:07.410396+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:09:17.865796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430258+0000","last_change":"2023-08-25T03:35:07.426052+0000","last_active":"2023-08-25T03:35:07.430258+0000","last_peered":"2023-08-25T03:35:07.430258+0000","last_clean":"2023-08-25T03:35:07.430258+0000","last_became_active":"2023-08-25T03:35:07.425640+0000","last_became_peered":"2023-08-25T03:35:07.425640+0000","last_unstale":"2023-08-25T03:35:07.430258+0000","last_undegraded":"2023-08-25T03:35:07.430258+0000","last_fullsized":"2023-08-25T03:35:07.430258+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:21:22.893614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.412175+0000","last_change":"2023-08-25T03:35:06.924905+0000","last_active":"2023-08-25T03:35:07.412175+0000","last_peered":"2023-08-25T03:35:07.412175+0000","last_clean":"2023-08-25T03:35:07.412175+0000","last_became_active":"2023-08-25T03:35:06.924536+0000","last_became_peered":"2023-08-25T03:35:06.924536+0000","last_unstale":"2023-08-25T03:35:07.412175+0000","last_undegraded":"2023-08-25T03:35:07.412175+0000","last_fullsized":"2023-08-25T03:35:07.412175+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:58:51.098028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":37,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.413296+0000","last_change":"2023-08-25T03:33:57.570267+0000","last_active":"2023-08-25T03:35:07.413296+0000","last_peered":"2023-08-25T03:35:07.413296+0000","last_clean":"2023-08-25T03:35:07.413296+0000","last_became_active":"2023-08-25T03:33:57.569882+0000","last_became_peered":"2023-08-25T03:33:57.569882+0000","last_unstale":"2023-08-25T03:35:07.413296+0000","last_undegraded":"2023-08-25T03:35:07.413296+0000","last_fullsized":"2023-08-25T03:35:07.413296+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:03:43.121931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430915+0000","last_change":"2023-08-25T03:35:07.429830+0000","last_active":"2023-08-25T03:35:07.430915+0000","last_peered":"2023-08-25T03:35:07.430915+0000","last_clean":"2023-08-25T03:35:07.430915+0000","last_became_active":"2023-08-25T03:35:07.429577+0000","last_became_peered":"2023-08-25T03:35:07.429577+0000","last_unstale":"2023-08-25T03:35:07.430915+0000","last_undegraded":"2023-08-25T03:35:07.430915+0000","last_fullsized":"2023-08-25T03:35:07.430915+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:21:22.893614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.412355+0000","last_change":"2023-08-25T03:35:06.924926+0000","last_active":"2023-08-25T03:35:07.412355+0000","last_peered":"2023-08-25T03:35:07.412355+0000","last_clean":"2023-08-25T03:35:07.412355+0000","last_became_active":"2023-08-25T03:35:06.924351+0000","last_became_peered":"2023-08-25T03:35:06.924351+0000","last_unstale":"2023-08-25T03:35:07.412355+0000","last_undegraded":"2023-08-25T03:35:07.412355+0000","last_fullsized":"2023-08-25T03:35:07.412355+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T04:15:18.138334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":45,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.416632+0000","last_change":"2023-08-25T03:33:57.569595+0000","last_active":"2023-08-25T03:35:08.416632+0000","last_peered":"2023-08-25T03:35:08.416632+0000","last_clean":"2023-08-25T03:35:08.416632+0000","last_became_active":"2023-08-25T03:33:57.569307+0000","last_became_peered":"2023-08-25T03:33:57.569307+0000","last_unstale":"2023-08-25T03:35:08.416632+0000","last_undegraded":"2023-08-25T03:35:08.416632+0000","last_fullsized":"2023-08-25T03:35:08.416632+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:41:45.092985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.437868+0000","last_change":"2023-08-25T03:35:07.434675+0000","last_active":"2023-08-25T03:35:07.437868+0000","last_peered":"2023-08-25T03:35:07.437868+0000","last_clean":"2023-08-25T03:35:07.437868+0000","last_became_active":"2023-08-25T03:35:07.434369+0000","last_became_peered":"2023-08-25T03:35:07.434369+0000","last_unstale":"2023-08-25T03:35:07.437868+0000","last_undegraded":"2023-08-25T03:35:07.437868+0000","last_fullsized":"2023-08-25T03:35:07.437868+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:29:40.853414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.410690+0000","last_change":"2023-08-25T03:35:06.919122+0000","last_active":"2023-08-25T03:35:07.410690+0000","last_peered":"2023-08-25T03:35:07.410690+0000","last_clean":"2023-08-25T03:35:07.410690+0000","last_became_active":"2023-08-25T03:35:06.918837+0000","last_became_peered":"2023-08-25T03:35:06.918837+0000","last_unstale":"2023-08-25T03:35:07.410690+0000","last_undegraded":"2023-08-25T03:35:07.410690+0000","last_fullsized":"2023-08-25T03:35:07.410690+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:07:00.853225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":43,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.410847+0000","last_change":"2023-08-25T03:33:57.886704+0000","last_active":"2023-08-25T03:35:07.410847+0000","last_peered":"2023-08-25T03:35:07.410847+0000","last_clean":"2023-08-25T03:35:07.410847+0000","last_became_active":"2023-08-25T03:33:57.886294+0000","last_became_peered":"2023-08-25T03:33:57.886294+0000","last_unstale":"2023-08-25T03:35:07.410847+0000","last_undegraded":"2023-08-25T03:35:07.410847+0000","last_fullsized":"2023-08-25T03:35:07.410847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:27:20.869204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417460+0000","last_change":"2023-08-25T03:35:07.426139+0000","last_active":"2023-08-25T03:35:08.417460+0000","last_peered":"2023-08-25T03:35:08.417460+0000","last_clean":"2023-08-25T03:35:08.417460+0000","last_became_active":"2023-08-25T03:35:07.425802+0000","last_became_peered":"2023-08-25T03:35:07.425802+0000","last_unstale":"2023-08-25T03:35:08.417460+0000","last_undegraded":"2023-08-25T03:35:08.417460+0000","last_fullsized":"2023-08-25T03:35:08.417460+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:26:04.402442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.423050+0000","last_change":"2023-08-25T03:35:06.926727+0000","last_active":"2023-08-25T03:35:07.423050+0000","last_peered":"2023-08-25T03:35:07.423050+0000","last_clean":"2023-08-25T03:35:07.423050+0000","last_became_active":"2023-08-25T03:35:06.926447+0000","last_became_peered":"2023-08-25T03:35:06.926447+0000","last_unstale":"2023-08-25T03:35:07.423050+0000","last_undegraded":"2023-08-25T03:35:07.423050+0000","last_fullsized":"2023-08-25T03:35:07.423050+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:37:26.114237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":45,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.416899+0000","last_change":"2023-08-25T03:33:57.570657+0000","last_active":"2023-08-25T03:35:08.416899+0000","last_peered":"2023-08-25T03:35:08.416899+0000","last_clean":"2023-08-25T03:35:08.416899+0000","last_became_active":"2023-08-25T03:33:57.570425+0000","last_became_peered":"2023-08-25T03:33:57.570425+0000","last_unstale":"2023-08-25T03:35:08.416899+0000","last_undegraded":"2023-08-25T03:35:08.416899+0000","last_fullsized":"2023-08-25T03:35:08.416899+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:38:40.932053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":193,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.423115+0000","last_change":"2023-08-25T03:33:51.259898+0000","last_active":"2023-08-25T03:35:07.423115+0000","last_peered":"2023-08-25T03:35:07.423115+0000","last_clean":"2023-08-25T03:35:07.423115+0000","last_became_active":"2023-08-25T03:33:51.258780+0000","last_became_peered":"2023-08-25T03:33:51.258780+0000","last_unstale":"2023-08-25T03:35:07.423115+0000","last_undegraded":"2023-08-25T03:35:07.423115+0000","last_fullsized":"2023-08-25T03:35:07.423115+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:49.666858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:49.666858+0000","last_clean_scrub_stamp":"2023-08-25T03:33:49.666858+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:53:30.259949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.420489+0000","last_change":"2023-08-25T03:35:07.419689+0000","last_active":"2023-08-25T03:35:07.420489+0000","last_peered":"2023-08-25T03:35:07.420489+0000","last_clean":"2023-08-25T03:35:07.420489+0000","last_became_active":"2023-08-25T03:35:07.419419+0000","last_became_peered":"2023-08-25T03:35:07.419419+0000","last_unstale":"2023-08-25T03:35:07.420489+0000","last_undegraded":"2023-08-25T03:35:07.420489+0000","last_fullsized":"2023-08-25T03:35:07.420489+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:08:24.760786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"36'2","reported_seq":15,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417632+0000","last_change":"2023-08-25T03:35:06.921662+0000","last_active":"2023-08-25T03:35:08.417632+0000","last_peered":"2023-08-25T03:35:08.417632+0000","last_clean":"2023-08-25T03:35:08.417632+0000","last_became_active":"2023-08-25T03:35:06.921352+0000","last_became_peered":"2023-08-25T03:35:06.921352+0000","last_unstale":"2023-08-25T03:35:08.417632+0000","last_undegraded":"2023-08-25T03:35:08.417632+0000","last_fullsized":"2023-08-25T03:35:08.417632+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:49:26.211793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":37,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.423268+0000","last_change":"2023-08-25T03:33:57.571821+0000","last_active":"2023-08-25T03:35:07.423268+0000","last_peered":"2023-08-25T03:35:07.423268+0000","last_clean":"2023-08-25T03:35:07.423268+0000","last_became_active":"2023-08-25T03:33:57.571528+0000","last_became_peered":"2023-08-25T03:33:57.571528+0000","last_unstale":"2023-08-25T03:35:07.423268+0000","last_undegraded":"2023-08-25T03:35:07.423268+0000","last_fullsized":"2023-08-25T03:35:07.423268+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:25:39.853603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.430540+0000","last_change":"2023-08-25T03:35:07.426463+0000","last_active":"2023-08-25T03:35:07.430540+0000","last_peered":"2023-08-25T03:35:07.430540+0000","last_clean":"2023-08-25T03:35:07.430540+0000","last_became_active":"2023-08-25T03:35:07.426213+0000","last_became_peered":"2023-08-25T03:35:07.426213+0000","last_unstale":"2023-08-25T03:35:07.430540+0000","last_undegraded":"2023-08-25T03:35:07.430540+0000","last_fullsized":"2023-08-25T03:35:07.430540+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:46:49.278016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.416988+0000","last_change":"2023-08-25T03:35:06.922775+0000","last_active":"2023-08-25T03:35:08.416988+0000","last_peered":"2023-08-25T03:35:08.416988+0000","last_clean":"2023-08-25T03:35:08.416988+0000","last_became_active":"2023-08-25T03:35:06.921468+0000","last_became_peered":"2023-08-25T03:35:06.921468+0000","last_unstale":"2023-08-25T03:35:08.416988+0000","last_undegraded":"2023-08-25T03:35:08.416988+0000","last_fullsized":"2023-08-25T03:35:08.416988+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:17:21.597681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.423599+0000","last_change":"2023-08-25T03:33:57.571730+0000","last_active":"2023-08-25T03:35:07.423599+0000","last_peered":"2023-08-25T03:35:07.423599+0000","last_clean":"2023-08-25T03:35:07.423599+0000","last_became_active":"2023-08-25T03:33:57.571442+0000","last_became_peered":"2023-08-25T03:33:57.571442+0000","last_unstale":"2023-08-25T03:35:07.423599+0000","last_undegraded":"2023-08-25T03:35:07.423599+0000","last_fullsized":"2023-08-25T03:35:07.423599+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:29:37.516806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.876191+0000","last_change":"2023-08-25T03:35:07.431421+0000","last_active":"2023-08-25T03:35:08.876191+0000","last_peered":"2023-08-25T03:35:08.876191+0000","last_clean":"2023-08-25T03:35:08.876191+0000","last_became_active":"2023-08-25T03:35:07.431107+0000","last_became_peered":"2023-08-25T03:35:07.431107+0000","last_unstale":"2023-08-25T03:35:08.876191+0000","last_undegraded":"2023-08-25T03:35:08.876191+0000","last_fullsized":"2023-08-25T03:35:08.876191+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:42:55.438930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.875994+0000","last_change":"2023-08-25T03:35:06.922373+0000","last_active":"2023-08-25T03:35:08.875994+0000","last_peered":"2023-08-25T03:35:08.875994+0000","last_clean":"2023-08-25T03:35:08.875994+0000","last_became_active":"2023-08-25T03:35:06.922049+0000","last_became_peered":"2023-08-25T03:35:06.922049+0000","last_unstale":"2023-08-25T03:35:08.875994+0000","last_undegraded":"2023-08-25T03:35:08.875994+0000","last_fullsized":"2023-08-25T03:35:08.875994+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:13:27.414597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":37,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.413546+0000","last_change":"2023-08-25T03:33:57.570506+0000","last_active":"2023-08-25T03:35:07.413546+0000","last_peered":"2023-08-25T03:35:07.413546+0000","last_clean":"2023-08-25T03:35:07.413546+0000","last_became_active":"2023-08-25T03:33:57.570171+0000","last_became_peered":"2023-08-25T03:33:57.570171+0000","last_unstale":"2023-08-25T03:35:07.413546+0000","last_undegraded":"2023-08-25T03:35:07.413546+0000","last_fullsized":"2023-08-25T03:35:07.413546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T05:23:43.119399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.437861+0000","last_change":"2023-08-25T03:35:07.434619+0000","last_active":"2023-08-25T03:35:07.437861+0000","last_peered":"2023-08-25T03:35:07.437861+0000","last_clean":"2023-08-25T03:35:07.437861+0000","last_became_active":"2023-08-25T03:35:07.434295+0000","last_became_peered":"2023-08-25T03:35:07.434295+0000","last_unstale":"2023-08-25T03:35:07.437861+0000","last_undegraded":"2023-08-25T03:35:07.437861+0000","last_fullsized":"2023-08-25T03:35:07.437861+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":35,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:06.900282+0000","last_clean_scrub_stamp":"2023-08-25T03:35:06.900282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:52:27.271866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-08-25T03:35:07.417992+0000","last_change":"2023-08-25T03:35:06.922104+0000","last_active":"2023-08-25T03:35:07.417992+0000","last_peered":"2023-08-25T03:35:07.417992+0000","last_clean":"2023-08-25T03:35:07.417992+0000","last_became_active":"2023-08-25T03:35:06.921783+0000","last_became_peered":"2023-08-25T03:35:06.921783+0000","last_unstale":"2023-08-25T03:35:07.417992+0000","last_undegraded":"2023-08-25T03:35:07.417992+0000","last_fullsized":"2023-08-25T03:35:07.417992+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:05.896177+0000","last_clean_scrub_stamp":"2023-08-25T03:35:05.896177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:29:39.849308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":45,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-08-25T03:35:08.417823+0000","last_change":"2023-08-25T03:33:57.570598+0000","last_active":"2023-08-25T03:35:08.417823+0000","last_peered":"2023-08-25T03:35:08.417823+0000","last_clean":"2023-08-25T03:35:08.417823+0000","last_became_active":"2023-08-25T03:33:57.570215+0000","last_became_peered":"2023-08-25T03:33:57.570215+0000","last_unstale":"2023-08-25T03:35:08.417823+0000","last_undegraded":"2023-08-25T03:35:08.417823+0000","last_fullsized":"2023-08-25T03:35:08.417823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:42:30.286958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2340,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":42,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":4750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542165,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1015808,"data_stored":758668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.45000000000000001}]},{"osd":1,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.70799999999999996}]},{"osd":2,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.54700000000000004}]},{"osd":3,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.65800000000000003}]},{"osd":4,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.748},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.63500000000000001}]},{"osd":5,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.77400000000000002}]},{"osd":6,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.67900000000000005}]}]},{"osd":6,"up_from":14,"seq":60129542165,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27432,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344408,"statfs":{"total":96636764160,"available":96608673792,"internally_reserved":0,"allocated":421888,"data_stored":173853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.58999999999999997}]},{"osd":1,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.621},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":4.5419999999999998,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":0.68799999999999994}]},{"osd":2,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":5.5179999999999998,"5min":5.5179999999999998,"15min":5.5179999999999998},"last":0.745},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.77400000000000002}]},{"osd":3,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.81100000000000005}]},{"osd":4,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":6.4050000000000002,"5min":6.4050000000000002,"15min":6.4050000000000002},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.89600000000000002}]},{"osd":5,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":7.4029999999999996,"5min":7.4029999999999996,"15min":7.4029999999999996},"last":0.84299999999999997}]},{"osd":7,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.94599999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.97899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":409600,"data_stored":166708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":6.6120000000000001,"5min":6.6120000000000001,"15min":6.6120000000000001},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.3530000000000002,"5min":3.3530000000000002,"15min":3.3530000000000002},"last":0.27600000000000002}]},{"osd":2,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":9.0950000000000006,"5min":9.0950000000000006,"15min":9.0950000000000006},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.35799999999999998}]},{"osd":3,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":3.4820000000000002,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":6.1970000000000001,"5min":6.1970000000000001,"15min":6.1970000000000001},"last":0.376}]},{"osd":4,"last update":"Fri Aug 25 03:34:49 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":6.4400000000000004,"5min":6.4400000000000004,"15min":6.4400000000000004},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":6.5860000000000003,"5min":6.5860000000000003,"15min":6.5860000000000003},"last":0.46100000000000002}]},{"osd":5,"last update":"Fri Aug 25 03:34:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":3.3900000000000001,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.436},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":6.399,"5min":6.399,"15min":6.399},"last":0.47499999999999998}]},{"osd":6,"last update":"Fri Aug 25 03:34:49 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":6.5229999999999997,"5min":6.5229999999999997,"15min":6.5229999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":3.4529999999999998,"5min":3.4529999999999998,"15min":3.4529999999999998},"last":0.498}]},{"osd":7,"last update":"Fri Aug 25 03:34:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":6.5590000000000002,"5min":6.5590000000000002,"15min":6.5590000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":3.4220000000000002,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.54300000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":999424,"data_stored":756986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"last":0.52900000000000003}]},{"osd":2,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.436},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.50900000000000001}]},{"osd":3,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.48399999999999999}]},{"osd":4,"last update":"Fri Aug 25 03:34:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.69799999999999995}]},{"osd":5,"last update":"Fri Aug 25 03:34:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.755},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.69099999999999995}]},{"osd":6,"last update":"Fri Aug 25 03:34:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.73299999999999998}]},{"osd":7,"last update":"Fri Aug 25 03:34:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.81399999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":393216,"data_stored":159490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.26300000000000001}]},{"osd":1,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.32300000000000001}]},{"osd":3,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.373},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.38100000000000001}]},{"osd":4,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.58299999999999996}]},{"osd":5,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.55600000000000005}]},{"osd":6,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.69799999999999995}]},{"osd":7,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.59199999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":421888,"data_stored":168264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.23200000000000001},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":12.923999999999999,"5min":12.923999999999999,"15min":12.923999999999999},"last":0.36499999999999999}]},{"osd":1,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":2.6680000000000001,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.26700000000000002}]},{"osd":2,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.311}]},{"osd":4,"last update":"Fri Aug 25 03:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.443},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.60099999999999998}]},{"osd":5,"last update":"Fri Aug 25 03:34:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":6.3520000000000003,"5min":6.3520000000000003,"15min":6.3520000000000003},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.497}]},{"osd":6,"last update":"Fri Aug 25 03:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.9849999999999999,"5min":2.9849999999999999,"15min":2.9849999999999999},"last":0.58399999999999996}]},{"osd":7,"last update":"Fri Aug 25 03:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.51500000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542165,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":173875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.41999999999999998}]},{"osd":1,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":8.327,"5min":8.327,"15min":8.327},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.58499999999999996}]},{"osd":2,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.70799999999999996}]},{"osd":3,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.63100000000000001}]},{"osd":5,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.81999999999999995}]},{"osd":6,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.91900000000000004}]},{"osd":7,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.79300000000000004}]}]},{"osd":5,"up_from":14,"seq":60129542164,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":416,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":425984,"data_stored":173383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.628}]},{"osd":1,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.497},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.45000000000000001}]},{"osd":2,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.54900000000000004}]},{"osd":3,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.754}]},{"osd":4,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.58199999999999996}]},{"osd":6,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.60199999999999998}]},{"osd":7,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.77900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-25T03:35:12.483 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-08-25T03:35:12.484 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-08-25T03:35:12.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.648+0000 7f6eaa041700 1 Processor -- start 2023-08-25T03:35:12.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.648+0000 7f6eaa041700 1 -- start start 2023-08-25T03:35:12.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.648+0000 7f6eaa041700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6ea406a450 0x7f6ea40ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:12.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.648+0000 7f6eaa041700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea40cc6f0 con 0x7f6ea406a830 2023-08-25T03:35:12.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.648+0000 7f6eaa041700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea4072010 con 0x7f6ea415ee40 2023-08-25T03:35:12.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.648+0000 7f6eaa041700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea40cc930 con 0x7f6ea406a450 2023-08-25T03:35:12.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.649+0000 7f6ea2ffd700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f6ea406a830 0x7f6ea40cbfe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48132/0 (socket says 172.21.15.2:48132) 2023-08-25T03:35:12.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.649+0000 7f6ea2ffd700 1 -- 172.21.15.2:0/3102431049 learned_addr learned my addr 172.21.15.2:0/3102431049 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:12.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.649+0000 7f6ea37fe700 1 --2- 172.21.15.2:0/3102431049 >> v2:172.21.15.31:3300/0 conn(0x7f6ea406a450 0x7f6ea40ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:12.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.649+0000 7f6ea27fc700 1 -- 172.21.15.2:0/3102431049 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1285231182 0 0) 0x7f6e8c001110 con 0x7f6ea406a830 2023-08-25T03:35:12.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.649+0000 7f6ea27fc700 1 -- 172.21.15.2:0/3102431049 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e900035b0 con 0x7f6ea406a830 2023-08-25T03:35:12.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.649+0000 7f6ea27fc700 1 -- 172.21.15.2:0/3102431049 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1320848616 0 0) 0x7f6e900035b0 con 0x7f6ea406a830 2023-08-25T03:35:12.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.649+0000 7f6ea27fc700 1 -- 172.21.15.2:0/3102431049 >> v2:172.21.15.31:3300/0 conn(0x7f6ea406a450 msgr2=0x7f6ea40ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:12.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.649+0000 7f6ea27fc700 1 --2- 172.21.15.2:0/3102431049 >> v2:172.21.15.31:3300/0 conn(0x7f6ea406a450 0x7f6ea40ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:12.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.649+0000 7f6ea27fc700 1 -- 172.21.15.2:0/3102431049 >> v1:172.21.15.31:6789/0 conn(0x7f6ea415ee40 legacy=0x7f6ea415f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:12.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.649+0000 7f6ea27fc700 1 -- 172.21.15.2:0/3102431049 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea415c820 con 0x7f6ea406a830 2023-08-25T03:35:12.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.649+0000 7f6ea27fc700 1 -- 172.21.15.2:0/3102431049 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f6e8c001800 con 0x7f6ea406a830 2023-08-25T03:35:12.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.649+0000 7f6ea27fc700 1 -- 172.21.15.2:0/3102431049 >> v1:172.21.15.2:6789/0 conn(0x7f6ea406a830 legacy=0x7f6ea40cbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:12.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.649+0000 7f6ea27fc700 1 --2- 172.21.15.2:0/3102431049 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6e90003a00 0x7f6e90003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:12.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.649+0000 7f6ea27fc700 1 --2- 172.21.15.2:0/3102431049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e90004850 0x7f6e90004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:12.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.649+0000 7f6ea27fc700 1 --2- 172.21.15.2:0/3102431049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e90005210 0x7f6e9000fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:12.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.649+0000 7f6ea27fc700 1 -- 172.21.15.2:0/3102431049 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e90010490 con 0x7f6e90004850 2023-08-25T03:35:12.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.649+0000 7f6ea27fc700 1 -- 172.21.15.2:0/3102431049 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e90010600 con 0x7f6e90005210 2023-08-25T03:35:12.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.649+0000 7f6ea27fc700 1 -- 172.21.15.2:0/3102431049 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e90007a70 con 0x7f6e90003a00 2023-08-25T03:35:12.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.649+0000 7f6ea2ffd700 1 --2- 172.21.15.2:0/3102431049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e90004850 0x7f6e90004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:12.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.650+0000 7f6ea37fe700 1 --2- 172.21.15.2:0/3102431049 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6e90003a00 0x7f6e90003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:12.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.650+0000 7f6ea3fff700 1 --2- 172.21.15.2:0/3102431049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e90005210 0x7f6e9000fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:12.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.650+0000 7f6ea2ffd700 1 -- 172.21.15.2:0/3102431049 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6e90003a00 msgr2=0x7f6e90003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:12.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.650+0000 7f6ea2ffd700 1 --2- 172.21.15.2:0/3102431049 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6e90003a00 0x7f6e90003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:12.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.650+0000 7f6ea2ffd700 1 -- 172.21.15.2:0/3102431049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e90005210 msgr2=0x7f6e9000fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:12.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.650+0000 7f6ea2ffd700 1 --2- 172.21.15.2:0/3102431049 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6e90005210 0x7f6e9000fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:12.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.650+0000 7f6ea2ffd700 1 -- 172.21.15.2:0/3102431049 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e900035b0 con 0x7f6e90004850 2023-08-25T03:35:12.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.650+0000 7f6ea3fff700 1 -- 172.21.15.2:0/3102431049 reap_dead start 2023-08-25T03:35:12.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.650+0000 7f6ea2ffd700 1 --2- 172.21.15.2:0/3102431049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e90004850 0x7f6e90004cd0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00c210 tx=0x7f6e90005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bcbf41d06e0179b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:12.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.650+0000 7f6ea27fc700 1 -- 172.21.15.2:0/3102431049 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c00cdb0 con 0x7f6e90004850 2023-08-25T03:35:12.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.650+0000 7f6ea27fc700 1 -- 172.21.15.2:0/3102431049 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8c005960 con 0x7f6e90004850 2023-08-25T03:35:12.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.650+0000 7f6ea27fc700 1 -- 172.21.15.2:0/3102431049 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c005d10 con 0x7f6e90004850 2023-08-25T03:35:12.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.651+0000 7f6eaa041700 1 -- 172.21.15.2:0/3102431049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e90004850 msgr2=0x7f6e90004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:12.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.651+0000 7f6eaa041700 1 --2- 172.21.15.2:0/3102431049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e90004850 0x7f6e90004cd0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00c210 tx=0x7f6e90005880 comp rx=0 tx=0).stop 2023-08-25T03:35:12.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.651+0000 7f6eaa041700 1 -- 172.21.15.2:0/3102431049 shutdown_connections 2023-08-25T03:35:12.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.651+0000 7f6eaa041700 1 --2- 172.21.15.2:0/3102431049 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6e90004850 0x7f6e90004cd0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:12.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.651+0000 7f6eaa041700 1 -- 172.21.15.2:0/3102431049 >> 172.21.15.2:0/3102431049 conn(0x7f6ea40d0000 msgr2=0x7f6ea40d03e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:12.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.651+0000 7f6eaa041700 1 -- 172.21.15.2:0/3102431049 shutdown_connections 2023-08-25T03:35:12.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.651+0000 7f6eaa041700 1 -- 172.21.15.2:0/3102431049 wait complete. 2023-08-25T03:35:12.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.652+0000 7f6eaa041700 1 Processor -- start 2023-08-25T03:35:12.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.652+0000 7f6eaa041700 1 -- start start 2023-08-25T03:35:12.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.653+0000 7f6eaa041700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6e90004850 0x7f6ea414f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:12.660 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.653+0000 7f6eaa041700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ea40b4d70 0x7f6ea40c5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:12.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.653+0000 7f6eaa041700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ea40b51f0 0x7f6ea40c5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:12.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.653+0000 7f6eaa041700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea40c7970 con 0x7f6ea40b4d70 2023-08-25T03:35:12.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.653+0000 7f6eaa041700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea40b44b0 con 0x7f6ea40b51f0 2023-08-25T03:35:12.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.653+0000 7f6eaa041700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea40b3930 con 0x7f6e90004850 2023-08-25T03:35:12.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.653+0000 7f6ea2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ea40b4d70 0x7f6ea40c5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:12.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.653+0000 7f6ea2ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ea40b4d70 0x7f6ea40c5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33246/0 (socket says 172.21.15.2:33246) 2023-08-25T03:35:12.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.653+0000 7f6ea2ffd700 1 -- 172.21.15.2:0/4241989534 learned_addr learned my addr 172.21.15.2:0/4241989534 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:12.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.653+0000 7f6ea3fff700 1 --2- 172.21.15.2:0/4241989534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ea40b51f0 0x7f6ea40c5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:12.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.653+0000 7f6ea37fe700 1 --2- 172.21.15.2:0/4241989534 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6e90004850 0x7f6ea414f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:12.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.653+0000 7f6ea2ffd700 1 -- 172.21.15.2:0/4241989534 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6e90004850 msgr2=0x7f6ea414f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:12.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.653+0000 7f6ea2ffd700 1 --2- 172.21.15.2:0/4241989534 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6e90004850 0x7f6ea414f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:12.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.653+0000 7f6ea2ffd700 1 -- 172.21.15.2:0/4241989534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ea40b51f0 msgr2=0x7f6ea40c5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:12.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.653+0000 7f6ea2ffd700 1 --2- 172.21.15.2:0/4241989534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ea40b51f0 0x7f6ea40c5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:12.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.653+0000 7f6ea2ffd700 1 -- 172.21.15.2:0/4241989534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea40b3c10 con 0x7f6ea40b4d70 2023-08-25T03:35:12.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.653+0000 7f6ea2ffd700 1 --2- 172.21.15.2:0/4241989534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ea40b4d70 0x7f6ea40c5590 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6e900105d0 tx=0x7f6e8c000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:12.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.654+0000 7f6e9bfff700 1 -- 172.21.15.2:0/4241989534 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c007d80 con 0x7f6ea40b4d70 2023-08-25T03:35:12.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.654+0000 7f6e9bfff700 1 -- 172.21.15.2:0/4241989534 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8c004870 con 0x7f6ea40b4d70 2023-08-25T03:35:12.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.654+0000 7f6eaa041700 1 -- 172.21.15.2:0/4241989534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea40b3ef0 con 0x7f6ea40b4d70 2023-08-25T03:35:12.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.654+0000 7f6e9bfff700 1 -- 172.21.15.2:0/4241989534 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c004be0 con 0x7f6ea40b4d70 2023-08-25T03:35:12.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.654+0000 7f6eaa041700 1 -- 172.21.15.2:0/4241989534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea414f940 con 0x7f6ea40b4d70 2023-08-25T03:35:12.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.655+0000 7f6eaa041700 1 -- 172.21.15.2:0/4241989534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea4066f70 con 0x7f6ea40b4d70 2023-08-25T03:35:12.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.655+0000 7f6e9bfff700 1 -- 172.21.15.2:0/4241989534 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6e8c004d40 con 0x7f6ea40b4d70 2023-08-25T03:35:12.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.656+0000 7f6e9bfff700 1 --2- 172.21.15.2:0/4241989534 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6e8406a0e0 0x7f6e8406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:12.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.656+0000 7f6e9bfff700 1 -- 172.21.15.2:0/4241989534 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6e8c0886a0 con 0x7f6ea40b4d70 2023-08-25T03:35:12.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.656+0000 7f6ea37fe700 1 --2- 172.21.15.2:0/4241989534 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6e8406a0e0 0x7f6e8406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:12.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.656+0000 7f6ea37fe700 1 --2- 172.21.15.2:0/4241989534 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6e8406a0e0 0x7f6e8406c5a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009ce0 tx=0x7f6e94009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:12.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.660+0000 7f6e9bfff700 1 -- 172.21.15.2:0/4241989534 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6e8c053e20 con 0x7f6ea40b4d70 2023-08-25T03:35:12.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.945+0000 7f6eaa041700 1 -- 172.21.15.2:0/4241989534 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f6ea40cc6f0 con 0x7f6ea40b4d70 2023-08-25T03:35:12.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.948+0000 7f6e9bfff700 1 -- 172.21.15.2:0/4241989534 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f6ea40cc6f0 con 0x7f6ea40b4d70 2023-08-25T03:35:12.949 INFO:teuthology.orchestra.run.smithi002.stdout:[client.0] 2023-08-25T03:35:12.949 INFO:teuthology.orchestra.run.smithi002.stdout: key = AQDxIOhkOijPNBAAx276DY0meH7ng2YxV3536A== 2023-08-25T03:35:12.949 INFO:teuthology.orchestra.run.smithi002.stdout: caps mds = "allow" 2023-08-25T03:35:12.950 INFO:teuthology.orchestra.run.smithi002.stdout: caps mon = "allow r" 2023-08-25T03:35:12.950 INFO:teuthology.orchestra.run.smithi002.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-08-25T03:35:12.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.950+0000 7f6eaa041700 1 -- 172.21.15.2:0/4241989534 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6e8406a0e0 msgr2=0x7f6e8406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:12.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.950+0000 7f6eaa041700 1 --2- 172.21.15.2:0/4241989534 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6e8406a0e0 0x7f6e8406c5a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009ce0 tx=0x7f6e94009210 comp rx=0 tx=0).stop 2023-08-25T03:35:12.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.950+0000 7f6eaa041700 1 -- 172.21.15.2:0/4241989534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ea40b4d70 msgr2=0x7f6ea40c5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:12.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.950+0000 7f6eaa041700 1 --2- 172.21.15.2:0/4241989534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ea40b4d70 0x7f6ea40c5590 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6e900105d0 tx=0x7f6e8c000fe0 comp rx=0 tx=0).stop 2023-08-25T03:35:12.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.950+0000 7f6eaa041700 1 -- 172.21.15.2:0/4241989534 shutdown_connections 2023-08-25T03:35:12.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.950+0000 7f6eaa041700 1 --2- 172.21.15.2:0/4241989534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6ea40b51f0 0x7f6ea40c5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:12.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.950+0000 7f6eaa041700 1 --2- 172.21.15.2:0/4241989534 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6e90004850 0x7f6ea414f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:12.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.950+0000 7f6eaa041700 1 --2- 172.21.15.2:0/4241989534 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6e8406a0e0 0x7f6e8406c5a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:12.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.950+0000 7f6eaa041700 1 --2- 172.21.15.2:0/4241989534 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6ea40b4d70 0x7f6ea40c5590 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:12.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.950+0000 7f6eaa041700 1 -- 172.21.15.2:0/4241989534 >> 172.21.15.2:0/4241989534 conn(0x7f6ea40d0000 msgr2=0x7f6ea406e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:12.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.951+0000 7f6eaa041700 1 -- 172.21.15.2:0/4241989534 shutdown_connections 2023-08-25T03:35:12.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:12.951+0000 7f6eaa041700 1 -- 172.21.15.2:0/4241989534 wait complete. 2023-08-25T03:35:12.954 INFO:teuthology.orchestra.run.smithi002.stderr:updated caps for client.0 2023-08-25T03:35:12.963 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T03:35:13.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.088+0000 7f9345306700 1 Processor -- start 2023-08-25T03:35:13.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.088+0000 7f9345306700 1 -- start start 2023-08-25T03:35:13.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.089+0000 7f9345306700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f93400b8aa0 0x7f93400cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:13.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.089+0000 7f9345306700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93400d2b40 0x7f93400ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:13.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.089+0000 7f9345306700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f934015d750 con 0x7f934015d370 2023-08-25T03:35:13.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.089+0000 7f9345306700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93400d1aa0 con 0x7f93400b8aa0 2023-08-25T03:35:13.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.089+0000 7f9345306700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93400d1d80 con 0x7f93400d2b40 2023-08-25T03:35:13.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.089+0000 7f933f7fe700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93400d2b40 0x7f93400ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:13.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.089+0000 7f933e7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f93400b8aa0 0x7f93400cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:13.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.089+0000 7f933e7fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f93400b8aa0 0x7f93400cdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.2:52776/0 (socket says 172.21.15.2:52776) 2023-08-25T03:35:13.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.089+0000 7f933f7fe700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93400d2b40 0x7f93400ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.2:33500/0 (socket says 172.21.15.2:33500) 2023-08-25T03:35:13.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.089+0000 7f933e7fc700 1 -- 172.21.15.2:0/185371316 learned_addr learned my addr 172.21.15.2:0/185371316 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:13.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.090+0000 7f933e7fc700 1 -- 172.21.15.2:0/185371316 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93400d2b40 msgr2=0x7f93400ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:13.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.090+0000 7f933e7fc700 1 --2- 172.21.15.2:0/185371316 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93400d2b40 0x7f93400ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.090+0000 7f933e7fc700 1 -- 172.21.15.2:0/185371316 >> v1:172.21.15.31:6789/0 conn(0x7f934015d370 legacy=0x7f93400d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:13.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.090+0000 7f933e7fc700 1 -- 172.21.15.2:0/185371316 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93400d2060 con 0x7f93400b8aa0 2023-08-25T03:35:13.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.090+0000 7f933e7fc700 1 --2- 172.21.15.2:0/185371316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93400b8aa0 0x7f93400cdde0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f93340098f0 tx=0x7f9334009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61aa2fbdd1a171b3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:13.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.090+0000 7f933dffb700 1 -- 172.21.15.2:0/185371316 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9334011830 con 0x7f93400b8aa0 2023-08-25T03:35:13.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.090+0000 7f933dffb700 1 -- 172.21.15.2:0/185371316 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9334011990 con 0x7f93400b8aa0 2023-08-25T03:35:13.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.090+0000 7f933dffb700 1 -- 172.21.15.2:0/185371316 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9334011d20 con 0x7f93400b8aa0 2023-08-25T03:35:13.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.091+0000 7f9345306700 1 -- 172.21.15.2:0/185371316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93400b8aa0 msgr2=0x7f93400cdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:13.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.091+0000 7f9345306700 1 --2- 172.21.15.2:0/185371316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93400b8aa0 0x7f93400cdde0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f93340098f0 tx=0x7f9334009c00 comp rx=0 tx=0).stop 2023-08-25T03:35:13.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.091+0000 7f9345306700 1 -- 172.21.15.2:0/185371316 shutdown_connections 2023-08-25T03:35:13.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.091+0000 7f9345306700 1 --2- 172.21.15.2:0/185371316 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93400d2b40 0x7f93400ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.091+0000 7f9345306700 1 --2- 172.21.15.2:0/185371316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93400b8aa0 0x7f93400cdde0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.091+0000 7f9345306700 1 -- 172.21.15.2:0/185371316 >> 172.21.15.2:0/185371316 conn(0x7f934006ab50 msgr2=0x7f934006af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:13.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.091+0000 7f9345306700 1 -- 172.21.15.2:0/185371316 shutdown_connections 2023-08-25T03:35:13.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.091+0000 7f9345306700 1 -- 172.21.15.2:0/185371316 wait complete. 2023-08-25T03:35:13.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.092+0000 7f9345306700 1 Processor -- start 2023-08-25T03:35:13.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.093+0000 7f9345306700 1 -- start start 2023-08-25T03:35:13.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.093+0000 7f9345306700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93400b8aa0 0x7f9340154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:13.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.093+0000 7f9345306700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93400d2b40 0x7f93400b7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:13.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.093+0000 7f9345306700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f934015d370 0x7f93401f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:13.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.093+0000 7f9345306700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9340154340 con 0x7f93400b8aa0 2023-08-25T03:35:13.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.093+0000 7f9345306700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9340154ca0 con 0x7f934015d370 2023-08-25T03:35:13.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.093+0000 7f9345306700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9340146de0 con 0x7f93400d2b40 2023-08-25T03:35:13.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.094+0000 7f933effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93400b8aa0 0x7f9340154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:13.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.094+0000 7f933effd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93400b8aa0 0x7f9340154760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33258/0 (socket says 172.21.15.2:33258) 2023-08-25T03:35:13.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.094+0000 7f933effd700 1 -- 172.21.15.2:0/277085807 learned_addr learned my addr 172.21.15.2:0/277085807 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:13.103 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.094+0000 7f933e7fc700 1 --2- 172.21.15.2:0/277085807 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93400d2b40 0x7f93400b7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:13.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.094+0000 7f933f7fe700 1 --2- 172.21.15.2:0/277085807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f934015d370 0x7f93401f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:13.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.094+0000 7f933effd700 1 -- 172.21.15.2:0/277085807 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93400d2b40 msgr2=0x7f93400b7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:13.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.094+0000 7f933effd700 1 --2- 172.21.15.2:0/277085807 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93400d2b40 0x7f93400b7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.094+0000 7f933effd700 1 -- 172.21.15.2:0/277085807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f934015d370 msgr2=0x7f93401f5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:13.104 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.094+0000 7f933effd700 1 --2- 172.21.15.2:0/277085807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f934015d370 0x7f93401f5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.094+0000 7f933effd700 1 -- 172.21.15.2:0/277085807 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9334009550 con 0x7f93400b8aa0 2023-08-25T03:35:13.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.094+0000 7f933effd700 1 --2- 172.21.15.2:0/277085807 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93400b8aa0 0x7f9340154760 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f933000c430 tx=0x7f933000c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:13.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.094+0000 7f93277fe700 1 -- 172.21.15.2:0/277085807 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93300052d0 con 0x7f93400b8aa0 2023-08-25T03:35:13.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.094+0000 7f93277fe700 1 -- 172.21.15.2:0/277085807 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9330005450 con 0x7f93400b8aa0 2023-08-25T03:35:13.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.094+0000 7f9345306700 1 -- 172.21.15.2:0/277085807 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93401f6830 con 0x7f93400b8aa0 2023-08-25T03:35:13.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.094+0000 7f93277fe700 1 -- 172.21.15.2:0/277085807 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9330006aa0 con 0x7f93400b8aa0 2023-08-25T03:35:13.105 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.094+0000 7f9345306700 1 -- 172.21.15.2:0/277085807 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9340153f60 con 0x7f93400b8aa0 2023-08-25T03:35:13.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.095+0000 7f9345306700 1 -- 172.21.15.2:0/277085807 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93401516b0 con 0x7f93400b8aa0 2023-08-25T03:35:13.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.096+0000 7f93277fe700 1 -- 172.21.15.2:0/277085807 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9330003680 con 0x7f93400b8aa0 2023-08-25T03:35:13.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.096+0000 7f93277fe700 1 --2- 172.21.15.2:0/277085807 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f932c06a0e0 0x7f932c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:13.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.096+0000 7f93277fe700 1 -- 172.21.15.2:0/277085807 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f93300877a0 con 0x7f93400b8aa0 2023-08-25T03:35:13.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.096+0000 7f933e7fc700 1 --2- 172.21.15.2:0/277085807 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f932c06a0e0 0x7f932c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:13.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.097+0000 7f933e7fc700 1 --2- 172.21.15.2:0/277085807 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f932c06a0e0 0x7f932c06c5a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f93340098f0 tx=0x7f9334008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:13.106 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.101+0000 7f93277fe700 1 -- 172.21.15.2:0/277085807 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9330052e90 con 0x7f93400b8aa0 2023-08-25T03:35:13.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.347+0000 7f9345306700 1 -- 172.21.15.2:0/277085807 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f934015dcf0 con 0x7f93400b8aa0 2023-08-25T03:35:13.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.347+0000 7f93277fe700 1 -- 172.21.15.2:0/277085807 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4869 (secure 0 0 0) 0x7f93300564b0 con 0x7f93400b8aa0 2023-08-25T03:35:13.350 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:35:13.350 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4751,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/3802947908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3802947908},{"type":"v1","addr":"172.21.15.31:6834","nonce":3802947908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1470640897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6835","nonce":1470640897},{"type":"v1","addr":"172.21.15.2:6837","nonce":1470640897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6836/3706466654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3706466654},{"type":"v1","addr":"172.21.15.2:6836","nonce":3706466654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T03:35:07.333851+0000","modified":"2023-08-25T03:35:10.931788+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5229":{"gid":5229,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1140461626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1140461626},{"type":"v1","addr":"172.21.15.31:6835","nonce":1140461626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-25T03:35:13.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.349+0000 7f9345306700 1 -- 172.21.15.2:0/277085807 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f932c06a0e0 msgr2=0x7f932c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:13.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.349+0000 7f9345306700 1 --2- 172.21.15.2:0/277085807 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f932c06a0e0 0x7f932c06c5a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f93340098f0 tx=0x7f9334008030 comp rx=0 tx=0).stop 2023-08-25T03:35:13.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.349+0000 7f9345306700 1 -- 172.21.15.2:0/277085807 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93400b8aa0 msgr2=0x7f9340154760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:13.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.349+0000 7f9345306700 1 --2- 172.21.15.2:0/277085807 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93400b8aa0 0x7f9340154760 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f933000c430 tx=0x7f933000c7f0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.349+0000 7f9345306700 1 -- 172.21.15.2:0/277085807 shutdown_connections 2023-08-25T03:35:13.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.349+0000 7f9345306700 1 --2- 172.21.15.2:0/277085807 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f932c06a0e0 0x7f932c06c5a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.349+0000 7f9345306700 1 --2- 172.21.15.2:0/277085807 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f93400b8aa0 0x7f9340154760 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.349+0000 7f9345306700 1 --2- 172.21.15.2:0/277085807 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f934015d370 0x7f93401f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.349+0000 7f9345306700 1 --2- 172.21.15.2:0/277085807 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93400d2b40 0x7f93400b7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.349+0000 7f9345306700 1 -- 172.21.15.2:0/277085807 >> 172.21.15.2:0/277085807 conn(0x7f934006ab50 msgr2=0x7f93400b6710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:13.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.349+0000 7f9345306700 1 -- 172.21.15.2:0/277085807 shutdown_connections 2023-08-25T03:35:13.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.349+0000 7f9345306700 1 -- 172.21.15.2:0/277085807 wait complete. 2023-08-25T03:35:13.353 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2023-08-25T03:35:13.767 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-08-25T03:35:07.333851+0000', 'modified': '2023-08-25T03:35:10.931788+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5229}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5229': {'gid': 5229, 'name': 'd', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.31:6835/1140461626', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6833', 'nonce': 1140461626}, {'type': 'v1', 'addr': '172.21.15.31:6835', 'nonce': 1140461626}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-08-25T03:35:13.767 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-08-25T03:35:13.767 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-08-25T03:35:13.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.898+0000 7fa69f6fc700 1 Processor -- start 2023-08-25T03:35:13.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.899+0000 7fa69f6fc700 1 -- start start 2023-08-25T03:35:13.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.899+0000 7fa69f6fc700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fa69815a410 0x7fa6980ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:13.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.899+0000 7fa69f6fc700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa6980ce6b0 0x7fa6980ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:13.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.899+0000 7fa69f6fc700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa698071dc0 con 0x7fa69815a030 2023-08-25T03:35:13.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.900+0000 7fa69f6fc700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6980ceb00 con 0x7fa6980ce6b0 2023-08-25T03:35:13.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.900+0000 7fa69f6fc700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6980720a0 con 0x7fa69815a410 2023-08-25T03:35:13.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.900+0000 7fa69dc99700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa6980ce6b0 0x7fa6980ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:13.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.900+0000 7fa69dc99700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa6980ce6b0 0x7fa6980ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33270/0 (socket says 172.21.15.2:33270) 2023-08-25T03:35:13.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.900+0000 7fa69dc99700 1 -- 172.21.15.2:0/2359879394 learned_addr learned my addr 172.21.15.2:0/2359879394 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:13.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.900+0000 7fa69cc97700 1 --2- 172.21.15.2:0/2359879394 >> v2:172.21.15.31:3300/0 conn(0x7fa69815a410 0x7fa6980ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:13.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.900+0000 7fa69dc99700 1 -- 172.21.15.2:0/2359879394 >> v2:172.21.15.31:3300/0 conn(0x7fa69815a410 msgr2=0x7fa6980ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:13.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.900+0000 7fa69dc99700 1 --2- 172.21.15.2:0/2359879394 >> v2:172.21.15.31:3300/0 conn(0x7fa69815a410 0x7fa6980ce170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.900+0000 7fa69dc99700 1 -- 172.21.15.2:0/2359879394 >> v1:172.21.15.31:6789/0 conn(0x7fa69815a030 legacy=0x7fa6980cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:13.903 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.900+0000 7fa69dc99700 1 -- 172.21.15.2:0/2359879394 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6980cb4b0 con 0x7fa6980ce6b0 2023-08-25T03:35:13.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.900+0000 7fa69dc99700 1 --2- 172.21.15.2:0/2359879394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6980ce6b0 0x7fa6980ccd50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa69806de90 tx=0x7fa69000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593d58ba3024778c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:13.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.900+0000 7fa68bfff700 1 -- 172.21.15.2:0/2359879394 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69000e070 con 0x7fa6980ce6b0 2023-08-25T03:35:13.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.900+0000 7fa68bfff700 1 -- 172.21.15.2:0/2359879394 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa690003a20 con 0x7fa6980ce6b0 2023-08-25T03:35:13.904 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.900+0000 7fa68bfff700 1 -- 172.21.15.2:0/2359879394 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa690003dd0 con 0x7fa6980ce6b0 2023-08-25T03:35:13.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.901+0000 7fa69f6fc700 1 -- 172.21.15.2:0/2359879394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6980ce6b0 msgr2=0x7fa6980ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:13.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.901+0000 7fa69f6fc700 1 --2- 172.21.15.2:0/2359879394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6980ce6b0 0x7fa6980ccd50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa69806de90 tx=0x7fa69000b330 comp rx=0 tx=0).stop 2023-08-25T03:35:13.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.901+0000 7fa69f6fc700 1 -- 172.21.15.2:0/2359879394 shutdown_connections 2023-08-25T03:35:13.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.901+0000 7fa69f6fc700 1 --2- 172.21.15.2:0/2359879394 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6980ce6b0 0x7fa6980ccd50 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.901+0000 7fa69f6fc700 1 --2- 172.21.15.2:0/2359879394 >> v2:172.21.15.31:3300/0 conn(0x7fa69815a410 0x7fa6980ce170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.901+0000 7fa69f6fc700 1 -- 172.21.15.2:0/2359879394 >> 172.21.15.2:0/2359879394 conn(0x7fa6980b1cf0 msgr2=0x7fa6980b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:13.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.901+0000 7fa69f6fc700 1 -- 172.21.15.2:0/2359879394 shutdown_connections 2023-08-25T03:35:13.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.901+0000 7fa69f6fc700 1 -- 172.21.15.2:0/2359879394 wait complete. 2023-08-25T03:35:13.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.902+0000 7fa69f6fc700 1 Processor -- start 2023-08-25T03:35:13.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.902+0000 7fa69f6fc700 1 -- start start 2023-08-25T03:35:13.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.903+0000 7fa69f6fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6980ce6b0 0x7fa6980ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:13.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.903+0000 7fa69f6fc700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa69815a030 0x7fa6980c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:13.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.903+0000 7fa69f6fc700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa69815a410 0x7fa6980adee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:13.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.903+0000 7fa69f6fc700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa698150980 con 0x7fa6980ce6b0 2023-08-25T03:35:13.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.903+0000 7fa69f6fc700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6980ace20 con 0x7fa69815a030 2023-08-25T03:35:13.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.903+0000 7fa69f6fc700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6980adbb0 con 0x7fa69815a410 2023-08-25T03:35:13.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.903+0000 7fa69d498700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6980ce6b0 0x7fa6980ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:13.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.903+0000 7fa69d498700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6980ce6b0 0x7fa6980ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33272/0 (socket says 172.21.15.2:33272) 2023-08-25T03:35:13.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.903+0000 7fa69d498700 1 -- 172.21.15.2:0/1241189276 learned_addr learned my addr 172.21.15.2:0/1241189276 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:13.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.903+0000 7fa69dc99700 1 --2- 172.21.15.2:0/1241189276 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa69815a410 0x7fa6980adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:13.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.903+0000 7fa69d498700 1 -- 172.21.15.2:0/1241189276 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa69815a410 msgr2=0x7fa6980adee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:13.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.903+0000 7fa69cc97700 1 --2- 172.21.15.2:0/1241189276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa69815a030 0x7fa6980c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:13.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.903+0000 7fa69d498700 1 --2- 172.21.15.2:0/1241189276 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa69815a410 0x7fa6980adee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.903+0000 7fa69d498700 1 -- 172.21.15.2:0/1241189276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa69815a030 msgr2=0x7fa6980c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:13.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.903+0000 7fa69d498700 1 --2- 172.21.15.2:0/1241189276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa69815a030 0x7fa6980c9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.903+0000 7fa69d498700 1 -- 172.21.15.2:0/1241189276 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa690009b30 con 0x7fa6980ce6b0 2023-08-25T03:35:13.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.903+0000 7fa69d498700 1 --2- 172.21.15.2:0/1241189276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6980ce6b0 0x7fa6980ac8e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa69400c500 tx=0x7fa69400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:13.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.904+0000 7fa689ffb700 1 -- 172.21.15.2:0/1241189276 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa694011790 con 0x7fa6980ce6b0 2023-08-25T03:35:13.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.904+0000 7fa689ffb700 1 -- 172.21.15.2:0/1241189276 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6940118f0 con 0x7fa6980ce6b0 2023-08-25T03:35:13.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.904+0000 7fa69f6fc700 1 -- 172.21.15.2:0/1241189276 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6980ad030 con 0x7fa6980ce6b0 2023-08-25T03:35:13.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.904+0000 7fa689ffb700 1 -- 172.21.15.2:0/1241189276 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa694011ca0 con 0x7fa6980ce6b0 2023-08-25T03:35:13.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.904+0000 7fa69f6fc700 1 -- 172.21.15.2:0/1241189276 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6980ad5f0 con 0x7fa6980ce6b0 2023-08-25T03:35:13.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.905+0000 7fa689ffb700 1 -- 172.21.15.2:0/1241189276 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa69401c460 con 0x7fa6980ce6b0 2023-08-25T03:35:13.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.906+0000 7fa689ffb700 1 --2- 172.21.15.2:0/1241189276 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa67c06a0e0 0x7fa67c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:13.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.906+0000 7fa689ffb700 1 -- 172.21.15.2:0/1241189276 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa694088440 con 0x7fa6980ce6b0 2023-08-25T03:35:13.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.906+0000 7fa69cc97700 1 --2- 172.21.15.2:0/1241189276 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa67c06a0e0 0x7fa67c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:13.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.906+0000 7fa69cc97700 1 --2- 172.21.15.2:0/1241189276 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa67c06a0e0 0x7fa67c06c5a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa6980c9e80 tx=0x7fa68c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:13.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.908+0000 7fa6ab8a0b80 1 Processor -- start 2023-08-25T03:35:13.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.908+0000 7fa6ab8a0b80 1 -- start start 2023-08-25T03:35:13.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.908+0000 7fa6ab8a0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bce83c7790 0x55bce83c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:13.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.908+0000 7fa6ab8a0b80 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x55bce83c2db0 0x55bce840dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:13.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.908+0000 7fa6ab8a0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c4f80 0x55bce83e9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:13.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.908+0000 7fa6ab8a0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bce832e760 con 0x55bce83c4f80 2023-08-25T03:35:13.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.908+0000 7fa6ab8a0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bce831f730 con 0x55bce83c7790 2023-08-25T03:35:13.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.908+0000 7fa6ab8a0b80 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bce831d050 con 0x55bce83c2db0 2023-08-25T03:35:13.916 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.908+0000 7fa69dc99700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c4f80 0x55bce83e9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:13.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.909+0000 7fa69dc99700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c4f80 0x55bce83e9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33284/0 (socket says 172.21.15.2:33284) 2023-08-25T03:35:13.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.909+0000 7fa69dc99700 1 -- 172.21.15.2:0/4208123188 learned_addr learned my addr 172.21.15.2:0/4208123188 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:13.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.909+0000 7fa69cc97700 1 --2- 172.21.15.2:0/4208123188 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x55bce83c2db0 0x55bce840dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:13.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.909+0000 7fa69d498700 1 --2- 172.21.15.2:0/4208123188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bce83c7790 0x55bce83c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:13.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.909+0000 7fa69dc99700 1 -- 172.21.15.2:0/4208123188 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x55bce83c2db0 msgr2=0x55bce840dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:13.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.909+0000 7fa69dc99700 1 --2- 172.21.15.2:0/4208123188 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x55bce83c2db0 0x55bce840dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.909+0000 7fa69dc99700 1 -- 172.21.15.2:0/4208123188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bce83c7790 msgr2=0x55bce83c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:13.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.909+0000 7fa69dc99700 1 --2- 172.21.15.2:0/4208123188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bce83c7790 0x55bce83c7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.909+0000 7fa69dc99700 1 -- 172.21.15.2:0/4208123188 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bce81985e0 con 0x55bce83c4f80 2023-08-25T03:35:13.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.909+0000 7fa69dc99700 1 --2- 172.21.15.2:0/4208123188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c4f80 0x55bce83e9e60 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa69000b370 tx=0x7fa690010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb730b0e0ded301b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:13.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.909+0000 7fa682ffd700 1 -- 172.21.15.2:0/4208123188 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa690003d90 con 0x55bce83c4f80 2023-08-25T03:35:13.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.909+0000 7fa682ffd700 1 -- 172.21.15.2:0/4208123188 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa69000b980 con 0x55bce83c4f80 2023-08-25T03:35:13.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.909+0000 7fa682ffd700 1 -- 172.21.15.2:0/4208123188 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa69000bcf0 con 0x55bce83c4f80 2023-08-25T03:35:13.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.910+0000 7fa6ab8a0b80 1 -- 172.21.15.2:0/4208123188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c4f80 msgr2=0x55bce83e9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:13.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.910+0000 7fa6ab8a0b80 1 --2- 172.21.15.2:0/4208123188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c4f80 0x55bce83e9e60 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa69000b370 tx=0x7fa690010000 comp rx=0 tx=0).stop 2023-08-25T03:35:13.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.910+0000 7fa6ab8a0b80 1 -- 172.21.15.2:0/4208123188 shutdown_connections 2023-08-25T03:35:13.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.910+0000 7fa6ab8a0b80 1 --2- 172.21.15.2:0/4208123188 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c4f80 0x55bce83e9e60 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.910+0000 7fa6ab8a0b80 1 --2- 172.21.15.2:0/4208123188 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bce83c7790 0x55bce83c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.910+0000 7fa6ab8a0b80 1 --2- 172.21.15.2:0/4208123188 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x55bce83c2db0 0x55bce840dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.910+0000 7fa6ab8a0b80 1 -- 172.21.15.2:0/4208123188 >> 172.21.15.2:0/4208123188 conn(0x55bce833b090 msgr2=0x55bce840c920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:13.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.910+0000 7fa6ab8a0b80 1 -- 172.21.15.2:0/4208123188 shutdown_connections 2023-08-25T03:35:13.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.910+0000 7fa6ab8a0b80 1 -- 172.21.15.2:0/4208123188 wait complete. 2023-08-25T03:35:13.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.910+0000 7fa6ab8a0b80 1 -- start start 2023-08-25T03:35:13.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.910+0000 7fa6ab8a0b80 1 Processor -- start 2023-08-25T03:35:13.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.911+0000 7fa6ab8a0b80 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x55bce83c2db0 0x55bce83e73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:13.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.911+0000 7fa6ab8a0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c4f80 0x55bce8409780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:13.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.911+0000 7fa6ab8a0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bce83c7790 0x55bce84038f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:13.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.911+0000 7fa6ab8a0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bce8318940 con 0x55bce83c4f80 2023-08-25T03:35:13.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.911+0000 7fa6ab8a0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bce82b6b60 con 0x55bce83c7790 2023-08-25T03:35:13.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.911+0000 7fa6ab8a0b80 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bce82b6860 con 0x55bce83c2db0 2023-08-25T03:35:13.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.911+0000 7fa6ab8a0b80 10 client.0 _ll_register_callbacks cb 0x55bce8200a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T03:35:13.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.912+0000 7fa69cc97700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c4f80 0x55bce8409780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:13.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.912+0000 7fa69cc97700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c4f80 0x55bce8409780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33288/0 (socket says 172.21.15.2:33288) 2023-08-25T03:35:13.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.912+0000 7fa69cc97700 1 -- 172.21.15.2:0/2706447456 learned_addr learned my addr 172.21.15.2:0/2706447456 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:13.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.912+0000 7fa69dc99700 1 --2- 172.21.15.2:0/2706447456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bce83c7790 0x55bce84038f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:13.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.912+0000 7fa69d498700 1 --2- 172.21.15.2:0/2706447456 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x55bce83c2db0 0x55bce83e73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:13.921 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.912+0000 7fa69cc97700 1 -- 172.21.15.2:0/2706447456 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x55bce83c2db0 msgr2=0x55bce83e73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:13.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.912+0000 7fa69cc97700 1 --2- 172.21.15.2:0/2706447456 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x55bce83c2db0 0x55bce83e73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.912+0000 7fa69cc97700 1 -- 172.21.15.2:0/2706447456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bce83c7790 msgr2=0x55bce84038f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:13.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.912+0000 7fa69cc97700 1 --2- 172.21.15.2:0/2706447456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bce83c7790 0x55bce84038f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.912+0000 7fa69cc97700 1 -- 172.21.15.2:0/2706447456 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6980cb4b0 con 0x55bce83c4f80 2023-08-25T03:35:13.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.912+0000 7fa69cc97700 1 --2- 172.21.15.2:0/2706447456 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c4f80 0x55bce8409780 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa68c015c20 tx=0x7fa68c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a0d66af63a8e1f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:13.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.912+0000 7fa677fff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-08-25T03:35:13.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.912+0000 7fa677fff700 1 -- 172.21.15.2:0/2706447456 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c013a90 con 0x55bce83c4f80 2023-08-25T03:35:13.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.912+0000 7fa677fff700 1 -- 172.21.15.2:0/2706447456 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa68c013bf0 con 0x55bce83c4f80 2023-08-25T03:35:13.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.912+0000 7fa6ab8a0b80 1 -- 172.21.15.2:0/2706447456 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bce83b9e00 con 0x55bce83c4f80 2023-08-25T03:35:13.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.912+0000 7fa677fff700 1 -- 172.21.15.2:0/2706447456 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa68c013db0 con 0x55bce83c4f80 2023-08-25T03:35:13.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.912+0000 7fa6ab8a0b80 10 client.5409 fetch_fsmap learned FSMap version 17 2023-08-25T03:35:13.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.912+0000 7fa6ab8a0b80 1 -- 172.21.15.2:0/2706447456 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55bce8144d90 con 0x55bce83c4f80 2023-08-25T03:35:13.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.913+0000 7fa677fff700 1 -- 172.21.15.2:0/2706447456 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bce83b9e00 con 0x55bce83c4f80 2023-08-25T03:35:13.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.913+0000 7fa677fff700 1 -- 172.21.15.2:0/2706447456 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa68c021420 con 0x55bce83c4f80 2023-08-25T03:35:13.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.913+0000 7fa6ab8a0b80 10 client.5409 fetch_fsmap finished waiting for FSMap version 17 2023-08-25T03:35:13.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.913+0000 7fa6ab8a0b80 10 client.5409 resolve_mds: resolved name 'd' to daemon mds.d 2023-08-25T03:35:13.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.913+0000 7fa6ab8a0b80 20 client.5409 populate_metadata read hostname 'smithi002' 2023-08-25T03:35:13.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.913+0000 7fa6ab8a0b80 1 --2- 172.21.15.2:0/2706447456 >> [v2:172.21.15.31:6833/1140461626,v1:172.21.15.31:6835/1140461626] conn(0x55bce8410010 0x55bce84103f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:13.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.913+0000 7fa6ab8a0b80 4 client.5409 mds_command: new command op to 5229 tid=0[{"prefix": "get_command_descriptions"}] 2023-08-25T03:35:13.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.913+0000 7fa6ab8a0b80 1 -- 172.21.15.2:0/2706447456 --> [v2:172.21.15.31:6833/1140461626,v1:172.21.15.31:6835/1140461626] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bce82beb50 con 0x55bce8410010 2023-08-25T03:35:13.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.913+0000 7fa69d498700 1 --2- 172.21.15.2:0/2706447456 >> [v2:172.21.15.31:6833/1140461626,v1:172.21.15.31:6835/1140461626] conn(0x55bce8410010 0x55bce84103f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:13.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.913+0000 7fa677fff700 10 client.5409 ms_handle_connect on v2:172.21.15.31:6833/1140461626 2023-08-25T03:35:13.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.913+0000 7fa69d498700 1 --2- 172.21.15.2:0/2706447456 >> [v2:172.21.15.31:6833/1140461626,v1:172.21.15.31:6835/1140461626] conn(0x55bce8410010 0x55bce84103f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab1608c8badfe023 server_cookie=59a44283f78628b4 in_seq=0 out_seq=0 2023-08-25T03:35:13.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.914+0000 7fa677fff700 1 -- 172.21.15.2:0/2706447456 <== mds.0 v2:172.21.15.31:6833/1140461626 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x55bce82beb50 con 0x55bce8410010 2023-08-25T03:35:13.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.914+0000 7fa677fff700 10 client.5409 handle_command_reply: tid=0 2023-08-25T03:35:13.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.914+0000 7fa6ab8a0b80 1 client.5409 shutdown 2023-08-25T03:35:13.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.914+0000 7fa6ab8a0b80 1 -- 172.21.15.2:0/2706447456 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c4f80 msgr2=0x55bce8409780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:13.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.914+0000 7fa6ab8a0b80 1 --2- 172.21.15.2:0/2706447456 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c4f80 0x55bce8409780 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa68c015c20 tx=0x7fa68c0047c0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.914+0000 7fa6ab8a0b80 1 -- 172.21.15.2:0/2706447456 shutdown_connections 2023-08-25T03:35:13.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.914+0000 7fa6ab8a0b80 1 --2- 172.21.15.2:0/2706447456 >> [v2:172.21.15.31:6833/1140461626,v1:172.21.15.31:6835/1140461626] conn(0x55bce8410010 0x55bce84103f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.914+0000 7fa6ab8a0b80 1 --2- 172.21.15.2:0/2706447456 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bce83c7790 0x55bce84038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.914+0000 7fa6ab8a0b80 1 --2- 172.21.15.2:0/2706447456 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x55bce83c2db0 0x55bce83e73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.914+0000 7fa6ab8a0b80 1 --2- 172.21.15.2:0/2706447456 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c4f80 0x55bce8409780 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.914+0000 7fa677fff700 0 client.5409 ms_handle_reset on v2:172.21.15.31:6833/1140461626 2023-08-25T03:35:13.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.915+0000 7fa6ab8a0b80 1 -- 172.21.15.2:0/2706447456 >> 172.21.15.2:0/2706447456 conn(0x55bce833b090 msgr2=0x55bce83c63f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:13.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.915+0000 7fa6ab8a0b80 1 -- 172.21.15.2:0/2706447456 shutdown_connections 2023-08-25T03:35:13.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.915+0000 7fa6ab8a0b80 1 -- 172.21.15.2:0/2706447456 wait complete. 2023-08-25T03:35:13.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.915+0000 7fa6ab8a0b80 20 client.5409 trim_cache size 0 max 16384 2023-08-25T03:35:13.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.932+0000 7fa6ab8a0b80 1 Processor -- start 2023-08-25T03:35:13.934 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.932+0000 7fa6ab8a0b80 1 -- start start 2023-08-25T03:35:13.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.933+0000 7fa6ab8a0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bce83c2db0 0x55bce83bdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:13.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.933+0000 7fa6ab8a0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c4f80 0x55bce83f5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:13.936 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.933+0000 7fa6ab8a0b80 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x55bce83c7790 0x55bce83f5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:13.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.933+0000 7fa6ab8a0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bce819a8d0 con 0x55bce83c4f80 2023-08-25T03:35:13.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.933+0000 7fa6ab8a0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bce83e8d40 con 0x55bce83c2db0 2023-08-25T03:35:13.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.933+0000 7fa6ab8a0b80 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bce84081d0 con 0x55bce83c7790 2023-08-25T03:35:13.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.933+0000 7fa69cc97700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c4f80 0x55bce83f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:13.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.933+0000 7fa69cc97700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c4f80 0x55bce83f5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33296/0 (socket says 172.21.15.2:33296) 2023-08-25T03:35:13.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.933+0000 7fa69cc97700 1 -- 172.21.15.2:0/990368105 learned_addr learned my addr 172.21.15.2:0/990368105 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:13.937 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.933+0000 7fa69d498700 1 --2- 172.21.15.2:0/990368105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bce83c2db0 0x55bce83bdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:13.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.933+0000 7fa69dc99700 1 --2- 172.21.15.2:0/990368105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x55bce83c7790 0x55bce83f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:13.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.933+0000 7fa69cc97700 1 -- 172.21.15.2:0/990368105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x55bce83c7790 msgr2=0x55bce83f5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:13.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.933+0000 7fa69cc97700 1 --2- 172.21.15.2:0/990368105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x55bce83c7790 0x55bce83f5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.933+0000 7fa69cc97700 1 -- 172.21.15.2:0/990368105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bce83c2db0 msgr2=0x55bce83bdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:13.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.933+0000 7fa69cc97700 1 --2- 172.21.15.2:0/990368105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bce83c2db0 0x55bce83bdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.938 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.934+0000 7fa69cc97700 1 -- 172.21.15.2:0/990368105 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bce83c2660 con 0x55bce83c4f80 2023-08-25T03:35:13.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.934+0000 7fa69cc97700 1 --2- 172.21.15.2:0/990368105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c4f80 0x55bce83f5410 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00e1c0 tx=0x7fa68c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78ee1fcccf4e5ac server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:13.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.934+0000 7fa6837fe700 1 -- 172.21.15.2:0/990368105 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c021420 con 0x55bce83c4f80 2023-08-25T03:35:13.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.934+0000 7fa6837fe700 1 -- 172.21.15.2:0/990368105 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68c021580 con 0x55bce83c4f80 2023-08-25T03:35:13.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.934+0000 7fa6837fe700 1 -- 172.21.15.2:0/990368105 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c013a90 con 0x55bce83c4f80 2023-08-25T03:35:13.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.934+0000 7fa6ab8a0b80 1 -- 172.21.15.2:0/990368105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c4f80 msgr2=0x55bce83f5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:13.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.934+0000 7fa6ab8a0b80 1 --2- 172.21.15.2:0/990368105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c4f80 0x55bce83f5410 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00e1c0 tx=0x7fa68c01f000 comp rx=0 tx=0).stop 2023-08-25T03:35:13.939 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.934+0000 7fa6ab8a0b80 1 -- 172.21.15.2:0/990368105 shutdown_connections 2023-08-25T03:35:13.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.934+0000 7fa6ab8a0b80 1 --2- 172.21.15.2:0/990368105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x55bce83c7790 0x55bce83f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.934+0000 7fa6ab8a0b80 1 --2- 172.21.15.2:0/990368105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bce83c2db0 0x55bce83bdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.934+0000 7fa6ab8a0b80 1 --2- 172.21.15.2:0/990368105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c4f80 0x55bce83f5410 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.934+0000 7fa6ab8a0b80 1 -- 172.21.15.2:0/990368105 >> 172.21.15.2:0/990368105 conn(0x55bce833b090 msgr2=0x55bce842e790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:13.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.934+0000 7fa6ab8a0b80 1 -- 172.21.15.2:0/990368105 shutdown_connections 2023-08-25T03:35:13.940 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.934+0000 7fa6ab8a0b80 1 -- 172.21.15.2:0/990368105 wait complete. 2023-08-25T03:35:13.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.935+0000 7fa6ab8a0b80 1 -- start start 2023-08-25T03:35:13.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.935+0000 7fa6ab8a0b80 1 Processor -- start 2023-08-25T03:35:13.944 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.936+0000 7fa6ab8a0b80 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x55bce83c2db0 0x55bce8402400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:13.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.936+0000 7fa6ab8a0b80 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bce83c4f80 0x55bce83f5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:13.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.936+0000 7fa6ab8a0b80 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c7790 0x55bce83f5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:13.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.936+0000 7fa6ab8a0b80 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bce840e580 con 0x55bce83c7790 2023-08-25T03:35:13.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.936+0000 7fa6ab8a0b80 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bce83e1a20 con 0x55bce83c4f80 2023-08-25T03:35:13.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.936+0000 7fa6ab8a0b80 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bce84733a0 con 0x55bce83c2db0 2023-08-25T03:35:13.945 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.936+0000 7fa6ab8a0b80 10 client.0 _ll_register_callbacks cb 0x7fa68c0157d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-08-25T03:35:13.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.936+0000 7fa69dc99700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c7790 0x55bce83f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:13.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.936+0000 7fa69dc99700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c7790 0x55bce83f5950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33304/0 (socket says 172.21.15.2:33304) 2023-08-25T03:35:13.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.936+0000 7fa69dc99700 1 -- 172.21.15.2:0/1055992850 learned_addr learned my addr 172.21.15.2:0/1055992850 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:13.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.936+0000 7fa69d498700 1 --2- 172.21.15.2:0/1055992850 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x55bce83c2db0 0x55bce8402400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:13.946 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.936+0000 7fa69cc97700 1 --2- 172.21.15.2:0/1055992850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bce83c4f80 0x55bce83f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:13.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.936+0000 7fa69dc99700 1 -- 172.21.15.2:0/1055992850 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x55bce83c2db0 msgr2=0x55bce8402400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:13.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.936+0000 7fa69dc99700 1 --2- 172.21.15.2:0/1055992850 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x55bce83c2db0 0x55bce8402400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.936+0000 7fa69dc99700 1 -- 172.21.15.2:0/1055992850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bce83c4f80 msgr2=0x55bce83f5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:13.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.936+0000 7fa69dc99700 1 --2- 172.21.15.2:0/1055992850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bce83c4f80 0x55bce83f5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.936+0000 7fa69dc99700 1 -- 172.21.15.2:0/1055992850 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bce8144d90 con 0x55bce83c7790 2023-08-25T03:35:13.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.937+0000 7fa69dc99700 1 --2- 172.21.15.2:0/1055992850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c7790 0x55bce83f5950 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa69001f040 tx=0x7fa69000e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e975ff445dbf98e9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:13.947 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.937+0000 7fa677fff700 10 client.0 ms_handle_connect on v2:172.21.15.2:3300/0 2023-08-25T03:35:13.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.937+0000 7fa677fff700 1 -- 172.21.15.2:0/1055992850 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa690004380 con 0x55bce83c7790 2023-08-25T03:35:13.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.937+0000 7fa677fff700 1 -- 172.21.15.2:0/1055992850 <== mon.0 v2:172.21.15.2:3300/0 2 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa69002fc70 con 0x55bce83c7790 2023-08-25T03:35:13.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.937+0000 7fa6ab8a0b80 1 -- 172.21.15.2:0/1055992850 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bce8311920 con 0x55bce83c7790 2023-08-25T03:35:13.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.937+0000 7fa677fff700 1 -- 172.21.15.2:0/1055992850 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa69002fe30 con 0x55bce83c7790 2023-08-25T03:35:13.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.937+0000 7fa6ab8a0b80 10 client.5415 fetch_fsmap learned FSMap version 17 2023-08-25T03:35:13.948 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.937+0000 7fa6ab8a0b80 1 -- 172.21.15.2:0/1055992850 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x55bce812dc10 con 0x55bce83c7790 2023-08-25T03:35:13.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.937+0000 7fa677fff700 1 -- 172.21.15.2:0/1055992850 <== mon.0 v2:172.21.15.2:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55bce8311920 con 0x55bce83c7790 2023-08-25T03:35:13.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.937+0000 7fa677fff700 1 -- 172.21.15.2:0/1055992850 <== mon.0 v2:172.21.15.2:3300/0 5 ==== fsmap(e 17) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa6900044e0 con 0x55bce83c7790 2023-08-25T03:35:13.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.937+0000 7fa6ab8a0b80 10 client.5415 fetch_fsmap finished waiting for FSMap version 17 2023-08-25T03:35:13.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.937+0000 7fa6ab8a0b80 10 client.5415 resolve_mds: resolved name 'd' to daemon mds.d 2023-08-25T03:35:13.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.937+0000 7fa6ab8a0b80 20 client.5415 populate_metadata read hostname 'smithi002' 2023-08-25T03:35:13.949 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.937+0000 7fa6ab8a0b80 1 --2- 172.21.15.2:0/1055992850 >> [v2:172.21.15.31:6833/1140461626,v1:172.21.15.31:6835/1140461626] conn(0x55bce84738d0 0x55bce8473cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:13.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.937+0000 7fa6ab8a0b80 4 client.5415 mds_command: new command op to 5229 tid=0[{"prefix": "status"}] 2023-08-25T03:35:13.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.937+0000 7fa6ab8a0b80 1 -- 172.21.15.2:0/1055992850 --> [v2:172.21.15.31:6833/1140461626,v1:172.21.15.31:6835/1140461626] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55bce82d2000 con 0x55bce84738d0 2023-08-25T03:35:13.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.938+0000 7fa69d498700 1 --2- 172.21.15.2:0/1055992850 >> [v2:172.21.15.31:6833/1140461626,v1:172.21.15.31:6835/1140461626] conn(0x55bce84738d0 0x55bce8473cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:13.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.938+0000 7fa69d498700 1 --2- 172.21.15.2:0/1055992850 >> [v2:172.21.15.31:6833/1140461626,v1:172.21.15.31:6835/1140461626] conn(0x55bce84738d0 0x55bce8473cb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0c627aaff977ff server_cookie=aa743f59c207d867 in_seq=0 out_seq=0 2023-08-25T03:35:13.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.938+0000 7fa677fff700 10 client.5415 ms_handle_connect on v2:172.21.15.31:6833/1140461626 2023-08-25T03:35:13.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.938+0000 7fa677fff700 1 -- 172.21.15.2:0/1055992850 <== mds.0 v2:172.21.15.31:6833/1140461626 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55bce82d2000 con 0x55bce84738d0 2023-08-25T03:35:13.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.938+0000 7fa677fff700 10 client.5415 handle_command_reply: tid=0 2023-08-25T03:35:13.950 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.938+0000 7fa6ab8a0b80 1 client.5415 shutdown 2023-08-25T03:35:13.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.938+0000 7fa6ab8a0b80 1 -- 172.21.15.2:0/1055992850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c7790 msgr2=0x55bce83f5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:13.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.938+0000 7fa6ab8a0b80 1 --2- 172.21.15.2:0/1055992850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c7790 0x55bce83f5950 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fa69001f040 tx=0x7fa69000e000 comp rx=0 tx=0).stop 2023-08-25T03:35:13.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.938+0000 7fa6ab8a0b80 1 -- 172.21.15.2:0/1055992850 shutdown_connections 2023-08-25T03:35:13.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.938+0000 7fa6ab8a0b80 1 --2- 172.21.15.2:0/1055992850 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x55bce83c7790 0x55bce83f5950 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.938+0000 7fa6ab8a0b80 1 --2- 172.21.15.2:0/1055992850 >> [v2:172.21.15.31:6833/1140461626,v1:172.21.15.31:6835/1140461626] conn(0x55bce84738d0 0x55bce8473cb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.938+0000 7fa6ab8a0b80 1 --2- 172.21.15.2:0/1055992850 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x55bce83c2db0 0x55bce8402400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.938+0000 7fa677fff700 0 client.5415 ms_handle_reset on v2:172.21.15.31:6833/1140461626 2023-08-25T03:35:13.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.938+0000 7fa6ab8a0b80 1 --2- 172.21.15.2:0/1055992850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x55bce83c4f80 0x55bce83f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.938+0000 7fa6ab8a0b80 1 -- 172.21.15.2:0/1055992850 >> 172.21.15.2:0/1055992850 conn(0x55bce833b090 msgr2=0x55bce83bdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:13.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.939+0000 7fa6ab8a0b80 1 -- 172.21.15.2:0/1055992850 shutdown_connections 2023-08-25T03:35:13.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.939+0000 7fa6ab8a0b80 1 -- 172.21.15.2:0/1055992850 wait complete. 2023-08-25T03:35:13.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.939+0000 7fa6ab8a0b80 20 client.5415 trim_cache size 0 max 16384 2023-08-25T03:35:13.953 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-08-25T03:35:13.953 INFO:teuthology.orchestra.run.smithi002.stdout: "cluster_fsid": "dc54136d-3683-431b-8a68-994ec618505c", 2023-08-25T03:35:13.953 INFO:teuthology.orchestra.run.smithi002.stdout: "whoami": 0, 2023-08-25T03:35:13.953 INFO:teuthology.orchestra.run.smithi002.stdout: "id": 5229, 2023-08-25T03:35:13.953 INFO:teuthology.orchestra.run.smithi002.stdout: "want_state": "up:active", 2023-08-25T03:35:13.954 INFO:teuthology.orchestra.run.smithi002.stdout: "state": "up:active", 2023-08-25T03:35:13.954 INFO:teuthology.orchestra.run.smithi002.stdout: "fs_name": "cephfs", 2023-08-25T03:35:13.954 INFO:teuthology.orchestra.run.smithi002.stdout: "rank_uptime": 6.0257140749999998, 2023-08-25T03:35:13.954 INFO:teuthology.orchestra.run.smithi002.stdout: "mdsmap_epoch": 17, 2023-08-25T03:35:13.954 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch": 37, 2023-08-25T03:35:13.954 INFO:teuthology.orchestra.run.smithi002.stdout: "osdmap_epoch_barrier": 37, 2023-08-25T03:35:13.954 INFO:teuthology.orchestra.run.smithi002.stdout: "uptime": 13.879062445000001 2023-08-25T03:35:13.955 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-08-25T03:35:13.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.939+0000 7fa69f6fc700 1 -- 172.21.15.2:0/1241189276 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa67c06a0e0 msgr2=0x7fa67c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:13.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.939+0000 7fa69f6fc700 1 --2- 172.21.15.2:0/1241189276 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa67c06a0e0 0x7fa67c06c5a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa6980c9e80 tx=0x7fa68c008040 comp rx=0 tx=0).stop 2023-08-25T03:35:13.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.939+0000 7fa69f6fc700 1 -- 172.21.15.2:0/1241189276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6980ce6b0 msgr2=0x7fa6980ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:13.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.939+0000 7fa69f6fc700 1 --2- 172.21.15.2:0/1241189276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6980ce6b0 0x7fa6980ac8e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fa69400c500 tx=0x7fa69400c8c0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.939+0000 7fa69f6fc700 1 -- 172.21.15.2:0/1241189276 shutdown_connections 2023-08-25T03:35:13.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.939+0000 7fa69f6fc700 1 --2- 172.21.15.2:0/1241189276 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa67c06a0e0 0x7fa67c06c5a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.939+0000 7fa69f6fc700 1 --2- 172.21.15.2:0/1241189276 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6980ce6b0 0x7fa6980ac8e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.939+0000 7fa69f6fc700 1 --2- 172.21.15.2:0/1241189276 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa69815a410 0x7fa6980adee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.939+0000 7fa69f6fc700 1 --2- 172.21.15.2:0/1241189276 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa69815a030 0x7fa6980c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:13.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.939+0000 7fa69f6fc700 1 -- 172.21.15.2:0/1241189276 >> 172.21.15.2:0/1241189276 conn(0x7fa6980b1cf0 msgr2=0x7fa6980b20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:13.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.939+0000 7fa69f6fc700 1 -- 172.21.15.2:0/1241189276 shutdown_connections 2023-08-25T03:35:13.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:13.939+0000 7fa69f6fc700 1 -- 172.21.15.2:0/1241189276 wait complete. 2023-08-25T03:35:13.958 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T03:35:14.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.081+0000 7f14b699d700 1 Processor -- start 2023-08-25T03:35:14.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.082+0000 7f14b699d700 1 -- start start 2023-08-25T03:35:14.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.082+0000 7f14b699d700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f14b00ceab0 0x7f14b00d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:14.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.082+0000 7f14b699d700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b00cf120 con 0x7f14b015d1f0 2023-08-25T03:35:14.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.082+0000 7f14b699d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b00b29f0 con 0x7f14b006cfe0 2023-08-25T03:35:14.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.082+0000 7f14b699d700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b00cef20 con 0x7f14b00ceab0 2023-08-25T03:35:14.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.082+0000 7f14affff700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f14b015d1f0 0x7f14b006c8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48206/0 (socket says 172.21.15.2:48206) 2023-08-25T03:35:14.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.082+0000 7f14affff700 1 -- 172.21.15.2:0/2986839033 learned_addr learned my addr 172.21.15.2:0/2986839033 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:14.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.083+0000 7f14b4f3a700 1 --2- 172.21.15.2:0/2986839033 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f14b00ceab0 0x7f14b00d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:14.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.083+0000 7f14aeffd700 1 -- 172.21.15.2:0/2986839033 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 331035179 0 0) 0x7f14b00cf120 con 0x7f14b015d1f0 2023-08-25T03:35:14.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.083+0000 7f14aeffd700 1 -- 172.21.15.2:0/2986839033 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14980035b0 con 0x7f14b015d1f0 2023-08-25T03:35:14.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.083+0000 7f14aeffd700 1 -- 172.21.15.2:0/2986839033 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2194020354 0 0) 0x7f14980035b0 con 0x7f14b015d1f0 2023-08-25T03:35:14.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.083+0000 7f14aeffd700 1 -- 172.21.15.2:0/2986839033 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f14b00ceab0 msgr2=0x7f14b00d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:14.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.083+0000 7f14aeffd700 1 --2- 172.21.15.2:0/2986839033 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f14b00ceab0 0x7f14b00d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:14.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.083+0000 7f14aeffd700 1 -- 172.21.15.2:0/2986839033 >> v1:172.21.15.31:6789/0 conn(0x7f14b006cfe0 legacy=0x7f14b015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:14.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.083+0000 7f14aeffd700 1 -- 172.21.15.2:0/2986839033 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b00b2cd0 con 0x7f14b015d1f0 2023-08-25T03:35:14.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.083+0000 7f14aeffd700 1 -- 172.21.15.2:0/2986839033 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f14a0001670 con 0x7f14b015d1f0 2023-08-25T03:35:14.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.083+0000 7f14aeffd700 1 -- 172.21.15.2:0/2986839033 >> v1:172.21.15.2:6789/0 conn(0x7f14b015d1f0 legacy=0x7f14b006c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:14.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.083+0000 7f14aeffd700 1 --2- 172.21.15.2:0/2986839033 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1498003a30 0x7f1498003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:14.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.083+0000 7f14aeffd700 1 --2- 172.21.15.2:0/2986839033 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f14b00ceab0 0x7f1498004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:14.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.083+0000 7f14aeffd700 1 --2- 172.21.15.2:0/2986839033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1498004d90 0x7f1498005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:14.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.083+0000 7f14aeffd700 1 -- 172.21.15.2:0/2986839033 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b00cef20 con 0x7f1498003a30 2023-08-25T03:35:14.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.083+0000 7f14aeffd700 1 -- 172.21.15.2:0/2986839033 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f149800baf0 con 0x7f1498004d90 2023-08-25T03:35:14.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.083+0000 7f14aeffd700 1 -- 172.21.15.2:0/2986839033 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f149800bcb0 con 0x7f14b00ceab0 2023-08-25T03:35:14.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.083+0000 7f14affff700 1 --2- 172.21.15.2:0/2986839033 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1498003a30 0x7f1498003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:14.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.083+0000 7f14af7fe700 1 --2- 172.21.15.2:0/2986839033 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f14b00ceab0 0x7f1498004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:14.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.083+0000 7f14b4f3a700 1 --2- 172.21.15.2:0/2986839033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1498004d90 0x7f1498005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:14.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.083+0000 7f14affff700 1 -- 172.21.15.2:0/2986839033 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f14b00ceab0 msgr2=0x7f1498004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:14.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.083+0000 7f14affff700 1 --2- 172.21.15.2:0/2986839033 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f14b00ceab0 0x7f1498004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:14.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.083+0000 7f14affff700 1 -- 172.21.15.2:0/2986839033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1498004d90 msgr2=0x7f1498005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:14.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.083+0000 7f14affff700 1 --2- 172.21.15.2:0/2986839033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1498004d90 0x7f1498005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:14.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.083+0000 7f14affff700 1 -- 172.21.15.2:0/2986839033 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14980035b0 con 0x7f1498003a30 2023-08-25T03:35:14.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.084+0000 7f14b4f3a700 1 --2- 172.21.15.2:0/2986839033 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1498004d90 0x7f1498005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:14.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.084+0000 7f14affff700 1 --2- 172.21.15.2:0/2986839033 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1498003a30 0x7f1498003e50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f14a000c210 tx=0x7f1498005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=781a2290973fb2b5 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:14.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.084+0000 7f14aeffd700 1 -- 172.21.15.2:0/2986839033 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0007c10 con 0x7f1498003a30 2023-08-25T03:35:14.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.084+0000 7f14aeffd700 1 -- 172.21.15.2:0/2986839033 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a000cdd0 con 0x7f1498003a30 2023-08-25T03:35:14.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.084+0000 7f14aeffd700 1 -- 172.21.15.2:0/2986839033 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0005ba0 con 0x7f1498003a30 2023-08-25T03:35:14.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.084+0000 7f14b699d700 1 -- 172.21.15.2:0/2986839033 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1498003a30 msgr2=0x7f1498003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:14.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.084+0000 7f14b699d700 1 --2- 172.21.15.2:0/2986839033 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1498003a30 0x7f1498003e50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f14a000c210 tx=0x7f1498005880 comp rx=0 tx=0).stop 2023-08-25T03:35:14.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.084+0000 7f14b4f3a700 1 -- 172.21.15.2:0/2986839033 reap_dead start 2023-08-25T03:35:14.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.084+0000 7f14b699d700 1 -- 172.21.15.2:0/2986839033 shutdown_connections 2023-08-25T03:35:14.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.084+0000 7f14b699d700 1 -- 172.21.15.2:0/2986839033 >> 172.21.15.2:0/2986839033 conn(0x7f14b00d0f70 msgr2=0x7f14b00d1350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:14.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.084+0000 7f14b699d700 1 -- 172.21.15.2:0/2986839033 shutdown_connections 2023-08-25T03:35:14.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.084+0000 7f14b699d700 1 -- 172.21.15.2:0/2986839033 wait complete. 2023-08-25T03:35:14.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.086+0000 7f14b699d700 1 Processor -- start 2023-08-25T03:35:14.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.086+0000 7f14b699d700 1 -- start start 2023-08-25T03:35:14.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.086+0000 7f14b699d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1498003a30 0x7f14b0160310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:14.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.086+0000 7f14b699d700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f14b00ca330 0x7f14b015c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:14.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.086+0000 7f14b699d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14b00cc340 0x7f14b00cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:14.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.086+0000 7f14b699d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b01f2690 con 0x7f1498003a30 2023-08-25T03:35:14.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.086+0000 7f14b699d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b00c84c0 con 0x7f14b00cc340 2023-08-25T03:35:14.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.086+0000 7f14b699d700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b01488f0 con 0x7f14b00ca330 2023-08-25T03:35:14.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.087+0000 7f14affff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1498003a30 0x7f14b0160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:14.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.087+0000 7f14affff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1498003a30 0x7f14b0160310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33320/0 (socket says 172.21.15.2:33320) 2023-08-25T03:35:14.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.087+0000 7f14affff700 1 -- 172.21.15.2:0/1160900994 learned_addr learned my addr 172.21.15.2:0/1160900994 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:14.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.087+0000 7f14af7fe700 1 --2- 172.21.15.2:0/1160900994 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f14b00ca330 0x7f14b015c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:14.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.087+0000 7f14affff700 1 -- 172.21.15.2:0/1160900994 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f14b00ca330 msgr2=0x7f14b015c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:14.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.087+0000 7f14affff700 1 --2- 172.21.15.2:0/1160900994 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f14b00ca330 0x7f14b015c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:14.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.087+0000 7f14affff700 1 -- 172.21.15.2:0/1160900994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14b00cc340 msgr2=0x7f14b00cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:14.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.087+0000 7f14affff700 1 --2- 172.21.15.2:0/1160900994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14b00cc340 0x7f14b00cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:14.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.087+0000 7f14affff700 1 -- 172.21.15.2:0/1160900994 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b00caeb0 con 0x7f1498003a30 2023-08-25T03:35:14.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.087+0000 7f14affff700 1 --2- 172.21.15.2:0/1160900994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1498003a30 0x7f14b0160310 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f14a000c800 tx=0x7f14a0001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:14.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.087+0000 7f14acff9700 1 -- 172.21.15.2:0/1160900994 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0001630 con 0x7f1498003a30 2023-08-25T03:35:14.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.087+0000 7f14acff9700 1 -- 172.21.15.2:0/1160900994 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a00070d0 con 0x7f1498003a30 2023-08-25T03:35:14.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.087+0000 7f14acff9700 1 -- 172.21.15.2:0/1160900994 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a0007440 con 0x7f1498003a30 2023-08-25T03:35:14.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.087+0000 7f14b699d700 1 -- 172.21.15.2:0/1160900994 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b00ca8f0 con 0x7f1498003a30 2023-08-25T03:35:14.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.087+0000 7f14b699d700 1 -- 172.21.15.2:0/1160900994 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b00c8150 con 0x7f1498003a30 2023-08-25T03:35:14.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.088+0000 7f14b699d700 1 -- 172.21.15.2:0/1160900994 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b00c87a0 con 0x7f1498003a30 2023-08-25T03:35:14.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.089+0000 7f14acff9700 1 -- 172.21.15.2:0/1160900994 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f14a0019030 con 0x7f1498003a30 2023-08-25T03:35:14.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.089+0000 7f14acff9700 1 --2- 172.21.15.2:0/1160900994 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f149c06a0e0 0x7f149c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:14.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.089+0000 7f14acff9700 1 -- 172.21.15.2:0/1160900994 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f14a0089410 con 0x7f1498003a30 2023-08-25T03:35:14.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.089+0000 7f14af7fe700 1 --2- 172.21.15.2:0/1160900994 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f149c06a0e0 0x7f149c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:14.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.090+0000 7f14af7fe700 1 --2- 172.21.15.2:0/1160900994 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f149c06a0e0 0x7f149c06c5a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f14b00c9f90 tx=0x7f14a4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:14.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.094+0000 7f14acff9700 1 -- 172.21.15.2:0/1160900994 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f14a0054b90 con 0x7f1498003a30 2023-08-25T03:35:14.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.339+0000 7f14b699d700 1 -- 172.21.15.2:0/1160900994 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f14b00c7fb0 con 0x7f1498003a30 2023-08-25T03:35:14.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.340+0000 7f14acff9700 1 -- 172.21.15.2:0/1160900994 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f14b00c7fb0 con 0x7f1498003a30 2023-08-25T03:35:14.342 INFO:teuthology.orchestra.run.smithi002.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T03:35:14.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.342+0000 7f14b699d700 1 -- 172.21.15.2:0/1160900994 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f149c06a0e0 msgr2=0x7f149c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:14.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.342+0000 7f14b699d700 1 --2- 172.21.15.2:0/1160900994 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f149c06a0e0 0x7f149c06c5a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f14b00c9f90 tx=0x7f14a4009040 comp rx=0 tx=0).stop 2023-08-25T03:35:14.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.342+0000 7f14b699d700 1 -- 172.21.15.2:0/1160900994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1498003a30 msgr2=0x7f14b0160310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:14.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.342+0000 7f14b699d700 1 --2- 172.21.15.2:0/1160900994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1498003a30 0x7f14b0160310 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f14a000c800 tx=0x7f14a0001a00 comp rx=0 tx=0).stop 2023-08-25T03:35:14.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.342+0000 7f14b699d700 1 -- 172.21.15.2:0/1160900994 shutdown_connections 2023-08-25T03:35:14.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.342+0000 7f14b699d700 1 --2- 172.21.15.2:0/1160900994 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f149c06a0e0 0x7f149c06c5a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:14.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.342+0000 7f14b699d700 1 --2- 172.21.15.2:0/1160900994 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1498003a30 0x7f14b0160310 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:14.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.342+0000 7f14b699d700 1 --2- 172.21.15.2:0/1160900994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f14b00cc340 0x7f14b00cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:14.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.342+0000 7f14b699d700 1 --2- 172.21.15.2:0/1160900994 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f14b00ca330 0x7f14b015c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:14.348 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.342+0000 7f14b699d700 1 -- 172.21.15.2:0/1160900994 >> 172.21.15.2:0/1160900994 conn(0x7f14b00d0f70 msgr2=0x7f14b0160ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:14.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.342+0000 7f14b699d700 1 -- 172.21.15.2:0/1160900994 shutdown_connections 2023-08-25T03:35:14.349 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:14.342+0000 7f14b699d700 1 -- 172.21.15.2:0/1160900994 wait complete. 2023-08-25T03:35:14.356 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-25T03:35:14.357 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T03:35:14.357 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi002.front.sepia.ceph.com 2023-08-25T03:35:14.357 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-08-25T03:35:14.358 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-08-25T03:35:14.358 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-25T03:35:14.358 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-25T03:35:14.359 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T03:35:14.359 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-25T03:35:14.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:14.360 DEBUG:teuthology.orchestra.run.smithi002:> ip addr 2023-08-25T03:35:14.378 INFO:teuthology.orchestra.run.smithi002.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T03:35:14.379 INFO:teuthology.orchestra.run.smithi002.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T03:35:14.379 INFO:teuthology.orchestra.run.smithi002.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T03:35:14.379 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:35:14.379 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 ::1/128 scope host 2023-08-25T03:35:14.379 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:35:14.379 INFO:teuthology.orchestra.run.smithi002.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T03:35:14.379 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:02 brd ff:ff:ff:ff:ff:ff 2023-08-25T03:35:14.380 INFO:teuthology.orchestra.run.smithi002.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T03:35:14.380 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:6b:fe:03 brd ff:ff:ff:ff:ff:ff 2023-08-25T03:35:14.380 INFO:teuthology.orchestra.run.smithi002.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T03:35:14.380 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:20 brd ff:ff:ff:ff:ff:ff 2023-08-25T03:35:14.380 INFO:teuthology.orchestra.run.smithi002.stdout: inet 172.21.15.2/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-25T03:35:14.381 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft 42266sec preferred_lft 42266sec 2023-08-25T03:35:14.381 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 fe80::ec4:7aff:febd:1920/64 scope link 2023-08-25T03:35:14.381 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:35:14.381 INFO:teuthology.orchestra.run.smithi002.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T03:35:14.381 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 0c:c4:7a:bd:19:21 brd ff:ff:ff:ff:ff:ff 2023-08-25T03:35:14.381 INFO:teuthology.orchestra.run.smithi002.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T03:35:14.381 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 92:c3:1c:fd:40:24 brd ff:ff:ff:ff:ff:ff 2023-08-25T03:35:14.382 INFO:teuthology.orchestra.run.smithi002.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-25T03:35:14.382 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:35:14.382 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 fe80::4058:70ff:fea4:c945/64 scope link 2023-08-25T03:35:14.382 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:35:14.382 INFO:teuthology.orchestra.run.smithi002.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T03:35:14.382 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether f2:17:15:9f:24:a0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-25T03:35:14.382 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 fe80::f017:15ff:fe9f:24a0/64 scope link 2023-08-25T03:35:14.383 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:35:14.383 INFO:teuthology.orchestra.run.smithi002.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T03:35:14.383 INFO:teuthology.orchestra.run.smithi002.stdout: link/ether 92:c3:1c:fd:40:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-25T03:35:14.383 INFO:teuthology.orchestra.run.smithi002.stdout: inet6 fe80::90c3:1cff:fefd:4024/64 scope link 2023-08-25T03:35:14.383 INFO:teuthology.orchestra.run.smithi002.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:35:14.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:14.384 DEBUG:teuthology.orchestra.run.smithi002:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T03:35:14.444 INFO:teuthology.orchestra.run.smithi002.stdout:1 2023-08-25T03:35:14.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:14.446 DEBUG:teuthology.orchestra.run.smithi002:> route 2023-08-25T03:35:14.507 INFO:teuthology.orchestra.run.smithi002.stdout:Kernel IP routing table 2023-08-25T03:35:14.507 INFO:teuthology.orchestra.run.smithi002.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T03:35:14.507 INFO:teuthology.orchestra.run.smithi002.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-25T03:35:14.508 INFO:teuthology.orchestra.run.smithi002.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-25T03:35:14.508 INFO:teuthology.orchestra.run.smithi002.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T03:35:14.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:14.509 DEBUG:teuthology.orchestra.run.smithi002:> (cd / && exec bash -c ' 2023-08-25T03:35:14.509 DEBUG:teuthology.orchestra.run.smithi002:> set -e 2023-08-25T03:35:14.509 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-25T03:35:14.510 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-25T03:35:14.511 DEBUG:teuthology.orchestra.run.smithi002:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-25T03:35:14.511 DEBUG:teuthology.orchestra.run.smithi002:> ') 2023-08-25T03:35:14.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:14.615 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2023-08-25T03:35:14.631 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-25T03:35:14.632 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-25T03:35:14.632 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-08-25T03:35:14.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:14.633 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link set brx.0 up 2023-08-25T03:35:14.700 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T03:35:14.701 DEBUG:teuthology.orchestra.run.smithi002:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-08-25T03:35:14.757 INFO:teuthology.orchestra.run.smithi002.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-08-25T03:35:14.757 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T03:35:14.758 DEBUG:teuthology.orchestra.run.smithi002:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-08-25T03:35:14.814 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-25T03:35:14.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:14.815 DEBUG:teuthology.orchestra.run.smithi002:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-25T03:35:14.900 INFO:teuthology.orchestra.run.smithi002.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-25T03:35:14.901 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-25T03:35:14.901 INFO:teuthology.orchestra.run.smithi002.stdout:invalid new device string format 2023-08-25T03:35:14.909 INFO:teuthology.orchestra.run.smithi002.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-25T03:35:14.910 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-25T03:35:14.910 INFO:teuthology.orchestra.run.smithi002.stdout:invalid new device string format 2023-08-25T03:35:14.910 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: resolved to: "172.21.15.2:3300,172.21.15.31:3300,172.21.15.31:3301" 2023-08-25T03:35:14.910 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-25T03:35:14.938 INFO:teuthology.orchestra.run.smithi002.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2023-08-25T03:35:14.938 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2023-08-25T03:35:14.938 INFO:teuthology.orchestra.run.smithi002.stdout:invalid new device string format 2023-08-25T03:35:14.938 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: resolved to: "172.21.15.2:3300,172.21.15.31:3300,172.21.15.31:3301" 2023-08-25T03:35:14.938 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-25T03:35:14.939 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: resolved to: "172.21.15.2:6789,172.21.15.31:6789,172.21.15.31:6790" 2023-08-25T03:35:14.939 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.2:6789,172.21.15.31:6789,172.21.15.31:6790:/ 2023-08-25T03:35:14.939 INFO:teuthology.orchestra.run.smithi002.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=dc54136d-3683-431b-8a68-994ec618505c" will pass to kernel 2023-08-25T03:35:14.944 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-25T03:35:14.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:14.945 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-08-25T03:35:14.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:14.987 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-25T03:35:15.073 INFO:teuthology.orchestra.run.smithi002.stdout:client5424 2023-08-25T03:35:15.073 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:35:15.074 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:35:15.092 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-08-25T03:35:15.256 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.254+0000 7f53f5416700 1 Processor -- start 2023-08-25T03:35:15.257 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.254+0000 7f53f5416700 1 -- start start 2023-08-25T03:35:15.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.255+0000 7f53f5416700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f53f00cdb70 0x7f53f006b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:15.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.255+0000 7f53f5416700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f53f00ce000 0x7f53f006be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:15.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.255+0000 7f53f5416700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53f0161620 con 0x7f53f00b1fb0 2023-08-25T03:35:15.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.255+0000 7f53f5416700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53f006d450 con 0x7f53f00cdb70 2023-08-25T03:35:15.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.255+0000 7f53f5416700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53f006d730 con 0x7f53f00ce000 2023-08-25T03:35:15.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.255+0000 7f53ee7fc700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f53f00cdb70 0x7f53f006b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:15.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.255+0000 7f53eeffd700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f53f00b1fb0 0x7f53f0160f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48230/0 (socket says 172.21.15.2:48230) 2023-08-25T03:35:15.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.255+0000 7f53eeffd700 1 -- 172.21.15.2:0/2949185141 learned_addr learned my addr 172.21.15.2:0/2949185141 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:15.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.255+0000 7f53ee7fc700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f53f00cdb70 0x7f53f006b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33334/0 (socket says 172.21.15.2:33334) 2023-08-25T03:35:15.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.256+0000 7f53ef7fe700 1 --2- 172.21.15.2:0/2949185141 >> v2:172.21.15.31:3300/0 conn(0x7f53f00ce000 0x7f53f006be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:15.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.256+0000 7f53ee7fc700 1 -- 172.21.15.2:0/2949185141 >> v2:172.21.15.31:3300/0 conn(0x7f53f00ce000 msgr2=0x7f53f006be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:15.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.256+0000 7f53ee7fc700 1 --2- 172.21.15.2:0/2949185141 >> v2:172.21.15.31:3300/0 conn(0x7f53f00ce000 0x7f53f006be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:15.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.256+0000 7f53ee7fc700 1 -- 172.21.15.2:0/2949185141 >> v1:172.21.15.2:6789/0 conn(0x7f53f00b1fb0 legacy=0x7f53f0160f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:15.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.256+0000 7f53ee7fc700 1 -- 172.21.15.2:0/2949185141 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f006da10 con 0x7f53f00cdb70 2023-08-25T03:35:15.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.256+0000 7f53edffb700 1 -- 172.21.15.2:0/2949185141 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1263725165 0 0) 0x7f53f0161620 con 0x7f53f00b1fb0 2023-08-25T03:35:15.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.256+0000 7f53ee7fc700 1 --2- 172.21.15.2:0/2949185141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f00cdb70 0x7f53f006b8c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f53f0072360 tx=0x7f53e0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=250f9c64b38b4bdd server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:15.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.256+0000 7f53edffb700 1 -- 172.21.15.2:0/2949185141 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0011830 con 0x7f53f00cdb70 2023-08-25T03:35:15.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.256+0000 7f53edffb700 1 -- 172.21.15.2:0/2949185141 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e0011990 con 0x7f53f00cdb70 2023-08-25T03:35:15.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.256+0000 7f53edffb700 1 -- 172.21.15.2:0/2949185141 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0011d20 con 0x7f53f00cdb70 2023-08-25T03:35:15.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.256+0000 7f53f5416700 1 -- 172.21.15.2:0/2949185141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f00cdb70 msgr2=0x7f53f006b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:15.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.256+0000 7f53f5416700 1 --2- 172.21.15.2:0/2949185141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f00cdb70 0x7f53f006b8c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f53f0072360 tx=0x7f53e0009b20 comp rx=0 tx=0).stop 2023-08-25T03:35:15.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.257+0000 7f53f5416700 1 -- 172.21.15.2:0/2949185141 shutdown_connections 2023-08-25T03:35:15.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.257+0000 7f53f5416700 1 --2- 172.21.15.2:0/2949185141 >> v2:172.21.15.31:3300/0 conn(0x7f53f00ce000 0x7f53f006be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:15.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.257+0000 7f53f5416700 1 --2- 172.21.15.2:0/2949185141 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f00cdb70 0x7f53f006b8c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:15.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.257+0000 7f53f5416700 1 -- 172.21.15.2:0/2949185141 >> 172.21.15.2:0/2949185141 conn(0x7f53f00cfea0 msgr2=0x7f53f00d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:15.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.257+0000 7f53f5416700 1 -- 172.21.15.2:0/2949185141 shutdown_connections 2023-08-25T03:35:15.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.257+0000 7f53f5416700 1 -- 172.21.15.2:0/2949185141 wait complete. 2023-08-25T03:35:15.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.258+0000 7f53f5416700 1 Processor -- start 2023-08-25T03:35:15.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.258+0000 7f53f5416700 1 -- start start 2023-08-25T03:35:15.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.259+0000 7f53f5416700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53f00b1fb0 0x7f53f015aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:15.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.259+0000 7f53f5416700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f00cdb70 0x7f53f01f6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:15.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.259+0000 7f53f5416700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53f00ce000 0x7f53f00c3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:15.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.259+0000 7f53f5416700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f01f65a0 con 0x7f53f00cdb70 2023-08-25T03:35:15.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.259+0000 7f53f5416700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f00c6cb0 con 0x7f53f00ce000 2023-08-25T03:35:15.267 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.259+0000 7f53f5416700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f00c66f0 con 0x7f53f00b1fb0 2023-08-25T03:35:15.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.259+0000 7f53ee7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f00cdb70 0x7f53f01f6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:15.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.259+0000 7f53ee7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f00cdb70 0x7f53f01f6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33340/0 (socket says 172.21.15.2:33340) 2023-08-25T03:35:15.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.259+0000 7f53ee7fc700 1 -- 172.21.15.2:0/2361448162 learned_addr learned my addr 172.21.15.2:0/2361448162 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:15.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.259+0000 7f53ef7fe700 1 --2- 172.21.15.2:0/2361448162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53f00ce000 0x7f53f00c3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:15.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.259+0000 7f53eeffd700 1 --2- 172.21.15.2:0/2361448162 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53f00b1fb0 0x7f53f015aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:15.268 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.259+0000 7f53ee7fc700 1 -- 172.21.15.2:0/2361448162 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53f00b1fb0 msgr2=0x7f53f015aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:15.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.259+0000 7f53ee7fc700 1 --2- 172.21.15.2:0/2361448162 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53f00b1fb0 0x7f53f015aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:15.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.259+0000 7f53ee7fc700 1 -- 172.21.15.2:0/2361448162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53f00ce000 msgr2=0x7f53f00c3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:15.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.259+0000 7f53ee7fc700 1 --2- 172.21.15.2:0/2361448162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53f00ce000 0x7f53f00c3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:15.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.259+0000 7f53ee7fc700 1 -- 172.21.15.2:0/2361448162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e00094a0 con 0x7f53f00cdb70 2023-08-25T03:35:15.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.259+0000 7f53ee7fc700 1 --2- 172.21.15.2:0/2361448162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f00cdb70 0x7f53f01f6a30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f53e0014040 tx=0x7f53e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:15.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.259+0000 7f53d77fe700 1 -- 172.21.15.2:0/2361448162 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e00038a0 con 0x7f53f00cdb70 2023-08-25T03:35:15.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.259+0000 7f53d77fe700 1 -- 172.21.15.2:0/2361448162 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e001e070 con 0x7f53f00cdb70 2023-08-25T03:35:15.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.260+0000 7f53f5416700 1 -- 172.21.15.2:0/2361448162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f00c69d0 con 0x7f53f00cdb70 2023-08-25T03:35:15.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.260+0000 7f53d77fe700 1 -- 172.21.15.2:0/2361448162 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0003c60 con 0x7f53f00cdb70 2023-08-25T03:35:15.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.260+0000 7f53f5416700 1 -- 172.21.15.2:0/2361448162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f015af70 con 0x7f53f00cdb70 2023-08-25T03:35:15.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.260+0000 7f53f5416700 1 -- 172.21.15.2:0/2361448162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f0066f70 con 0x7f53f00cdb70 2023-08-25T03:35:15.271 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.261+0000 7f53d77fe700 1 -- 172.21.15.2:0/2361448162 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f53e002d450 con 0x7f53f00cdb70 2023-08-25T03:35:15.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.261+0000 7f53d77fe700 1 --2- 172.21.15.2:0/2361448162 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f53dc06a0e0 0x7f53dc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:15.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.262+0000 7f53eeffd700 1 --2- 172.21.15.2:0/2361448162 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f53dc06a0e0 0x7f53dc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:15.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.262+0000 7f53d77fe700 1 -- 172.21.15.2:0/2361448162 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f53e0089640 con 0x7f53f00cdb70 2023-08-25T03:35:15.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.262+0000 7f53eeffd700 1 --2- 172.21.15.2:0/2361448162 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f53dc06a0e0 0x7f53dc06c5a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f53e4000f60 tx=0x7f53e4007c90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:15.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.266+0000 7f53d77fe700 1 -- 172.21.15.2:0/2361448162 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f53e0054dc0 con 0x7f53f00cdb70 2023-08-25T03:35:15.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.513+0000 7f53f5416700 1 -- 172.21.15.2:0/2361448162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f53f006dcd0 con 0x7f53f00cdb70 2023-08-25T03:35:15.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.514+0000 7f53d77fe700 1 -- 172.21.15.2:0/2361448162 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f53f006dcd0 con 0x7f53f00cdb70 2023-08-25T03:35:15.516 INFO:teuthology.orchestra.run.smithi002.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-08-25T03:35:15.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.516+0000 7f53f5416700 1 -- 172.21.15.2:0/2361448162 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f53dc06a0e0 msgr2=0x7f53dc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:15.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.516+0000 7f53f5416700 1 --2- 172.21.15.2:0/2361448162 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f53dc06a0e0 0x7f53dc06c5a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f53e4000f60 tx=0x7f53e4007c90 comp rx=0 tx=0).stop 2023-08-25T03:35:15.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.516+0000 7f53f5416700 1 -- 172.21.15.2:0/2361448162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f00cdb70 msgr2=0x7f53f01f6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:15.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.516+0000 7f53f5416700 1 --2- 172.21.15.2:0/2361448162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f00cdb70 0x7f53f01f6a30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f53e0014040 tx=0x7f53e0004750 comp rx=0 tx=0).stop 2023-08-25T03:35:15.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.516+0000 7f53f5416700 1 -- 172.21.15.2:0/2361448162 shutdown_connections 2023-08-25T03:35:15.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.516+0000 7f53f5416700 1 --2- 172.21.15.2:0/2361448162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f53f00ce000 0x7f53f00c3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:15.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.516+0000 7f53f5416700 1 --2- 172.21.15.2:0/2361448162 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f53f00b1fb0 0x7f53f015aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:15.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.516+0000 7f53f5416700 1 --2- 172.21.15.2:0/2361448162 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f53dc06a0e0 0x7f53dc06c5a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:15.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.516+0000 7f53f5416700 1 --2- 172.21.15.2:0/2361448162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f53f00cdb70 0x7f53f01f6a30 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:15.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.516+0000 7f53f5416700 1 -- 172.21.15.2:0/2361448162 >> 172.21.15.2:0/2361448162 conn(0x7f53f00cfea0 msgr2=0x7f53f00cb030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:15.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.516+0000 7f53f5416700 1 -- 172.21.15.2:0/2361448162 shutdown_connections 2023-08-25T03:35:15.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:15.516+0000 7f53f5416700 1 -- 172.21.15.2:0/2361448162 wait complete. 2023-08-25T03:35:15.531 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-08-25T03:35:15.531 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-08-25T03:35:15.532 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi031.front.sepia.ceph.com 2023-08-25T03:35:15.533 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-08-25T03:35:15.533 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-08-25T03:35:15.533 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-08-25T03:35:15.534 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-08-25T03:35:15.534 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-08-25T03:35:15.534 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-25T03:35:15.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:15.536 DEBUG:teuthology.orchestra.run.smithi031:> ip addr 2023-08-25T03:35:15.559 INFO:teuthology.orchestra.run.smithi031.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-08-25T03:35:15.560 INFO:teuthology.orchestra.run.smithi031.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-08-25T03:35:15.560 INFO:teuthology.orchestra.run.smithi031.stdout: inet 127.0.0.1/8 scope host lo 2023-08-25T03:35:15.560 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:35:15.560 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 ::1/128 scope host 2023-08-25T03:35:15.560 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:35:15.560 INFO:teuthology.orchestra.run.smithi031.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T03:35:15.560 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cc brd ff:ff:ff:ff:ff:ff 2023-08-25T03:35:15.561 INFO:teuthology.orchestra.run.smithi031.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T03:35:15.561 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether 0c:c4:7a:6b:f6:cd brd ff:ff:ff:ff:ff:ff 2023-08-25T03:35:15.561 INFO:teuthology.orchestra.run.smithi031.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-08-25T03:35:15.561 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:ba brd ff:ff:ff:ff:ff:ff 2023-08-25T03:35:15.561 INFO:teuthology.orchestra.run.smithi031.stdout: inet 172.21.15.31/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-08-25T03:35:15.562 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft 42249sec preferred_lft 42249sec 2023-08-25T03:35:15.562 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::ae1f:6bff:fea5:caba/64 scope link 2023-08-25T03:35:15.562 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:35:15.562 INFO:teuthology.orchestra.run.smithi031.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-08-25T03:35:15.563 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ac:1f:6b:a5:ca:bb brd ff:ff:ff:ff:ff:ff 2023-08-25T03:35:15.563 INFO:teuthology.orchestra.run.smithi031.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-08-25T03:35:15.563 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ba:9d:dd:5f:f1:13 brd ff:ff:ff:ff:ff:ff 2023-08-25T03:35:15.563 INFO:teuthology.orchestra.run.smithi031.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-08-25T03:35:15.563 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:35:15.563 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::947b:51ff:fef5:313a/64 scope link 2023-08-25T03:35:15.563 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:35:15.564 INFO:teuthology.orchestra.run.smithi031.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T03:35:15.564 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether c6:68:ce:93:64:ff brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-25T03:35:15.564 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::c468:ceff:fe93:64ff/64 scope link 2023-08-25T03:35:15.564 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:35:15.564 INFO:teuthology.orchestra.run.smithi031.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-08-25T03:35:15.564 INFO:teuthology.orchestra.run.smithi031.stdout: link/ether ba:9d:dd:5f:f1:13 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-25T03:35:15.564 INFO:teuthology.orchestra.run.smithi031.stdout: inet6 fe80::b89d:ddff:fe5f:f113/64 scope link 2023-08-25T03:35:15.564 INFO:teuthology.orchestra.run.smithi031.stdout: valid_lft forever preferred_lft forever 2023-08-25T03:35:15.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:15.565 DEBUG:teuthology.orchestra.run.smithi031:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-08-25T03:35:15.633 INFO:teuthology.orchestra.run.smithi031.stdout:1 2023-08-25T03:35:15.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:15.635 DEBUG:teuthology.orchestra.run.smithi031:> route 2023-08-25T03:35:15.697 INFO:teuthology.orchestra.run.smithi031.stdout:Kernel IP routing table 2023-08-25T03:35:15.697 INFO:teuthology.orchestra.run.smithi031.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-08-25T03:35:15.698 INFO:teuthology.orchestra.run.smithi031.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-08-25T03:35:15.698 INFO:teuthology.orchestra.run.smithi031.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-08-25T03:35:15.698 INFO:teuthology.orchestra.run.smithi031.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-08-25T03:35:15.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:15.699 DEBUG:teuthology.orchestra.run.smithi031:> (cd / && exec bash -c ' 2023-08-25T03:35:15.699 DEBUG:teuthology.orchestra.run.smithi031:> set -e 2023-08-25T03:35:15.699 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-08-25T03:35:15.700 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-08-25T03:35:15.700 DEBUG:teuthology.orchestra.run.smithi031:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-08-25T03:35:15.700 DEBUG:teuthology.orchestra.run.smithi031:> ') 2023-08-25T03:35:15.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:15.814 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-08-25T03:35:15.833 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-08-25T03:35:15.833 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-25T03:35:15.833 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-08-25T03:35:15.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:15.834 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link set brx.0 up 2023-08-25T03:35:15.902 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T03:35:15.903 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-08-25T03:35:15.960 INFO:teuthology.orchestra.run.smithi031.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-08-25T03:35:15.961 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-08-25T03:35:15.961 DEBUG:teuthology.orchestra.run.smithi031:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-08-25T03:35:16.017 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2023-08-25T03:35:16.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:16.018 DEBUG:teuthology.orchestra.run.smithi031:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-25T03:35:16.104 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-25T03:35:16.104 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-25T03:35:16.104 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2023-08-25T03:35:16.113 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-25T03:35:16.114 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-25T03:35:16.114 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2023-08-25T03:35:16.114 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.2:3300,172.21.15.31:3300,172.21.15.31:3301" 2023-08-25T03:35:16.114 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-25T03:35:16.186 INFO:teuthology.orchestra.run.smithi031.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2023-08-25T03:35:16.187 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2023-08-25T03:35:16.187 INFO:teuthology.orchestra.run.smithi031.stdout:invalid new device string format 2023-08-25T03:35:16.187 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.2:3300,172.21.15.31:3300,172.21.15.31:3301" 2023-08-25T03:35:16.187 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: switching to using v1 address with old syntax 2023-08-25T03:35:16.188 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: resolved to: "172.21.15.2:6789,172.21.15.31:6789,172.21.15.31:6790" 2023-08-25T03:35:16.188 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.2:6789,172.21.15.31:6789,172.21.15.31:6790:/ 2023-08-25T03:35:16.188 INFO:teuthology.orchestra.run.smithi031.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=dc54136d-3683-431b-8a68-994ec618505c" will pass to kernel 2023-08-25T03:35:16.189 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-08-25T03:35:16.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:16.190 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-08-25T03:35:16.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:16.219 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-08-25T03:35:16.291 INFO:teuthology.orchestra.run.smithi031.stdout:client5436 2023-08-25T03:35:16.292 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:35:16.292 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:35:16.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:16.350 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2023-08-25T03:35:16.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:16.376 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quota.max_bytes -v 0 ./subdir) 2023-08-25T03:35:16.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:16.447 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.quota.max_bytes ./subdir) 2023-08-25T03:35:16.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T03:35:16.508 INFO:teuthology.orchestra.run.smithi002.stderr:./subdir: ceph.quota.max_bytes: No such attribute 2023-08-25T03:35:16.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:16.509 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quota.max_bytes -v 1K ./subdir) 2023-08-25T03:35:16.567 INFO:teuthology.orchestra.run.smithi002.stderr:setfattr: ./subdir: Invalid argument 2023-08-25T03:35:16.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T03:35:16.568 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-08-25T03:35:16.569 DEBUG:teuthology.orchestra.run.smithi002:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T03:35:16.656 DEBUG:teuthology.orchestra.run.smithi031:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-08-25T03:35:16.697 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-08-25T03:35:16.697 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:35:16.698 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:35:16.719 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:35:16.719 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:35:16.780 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:35:16.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.951+0000 7f511c1f5700 1 Processor -- start 2023-08-25T03:35:16.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.951+0000 7f511c1f5700 1 -- start start 2023-08-25T03:35:16.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.952+0000 7f511c1f5700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f511415d370 0x7f51140d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:16.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.952+0000 7f511c1f5700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f511415d750 con 0x7f51140ce4f0 2023-08-25T03:35:16.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.952+0000 7f511c1f5700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51140d1aa0 con 0x7f51140b8aa0 2023-08-25T03:35:16.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.952+0000 7f511c1f5700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51140d1d80 con 0x7f511415d370 2023-08-25T03:35:16.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.952+0000 7f5119f91700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f511415d370 0x7f51140d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:16.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.952+0000 7f511a792700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f51140ce4f0 0x7f51140b62b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48246/0 (socket says 172.21.15.2:48246) 2023-08-25T03:35:16.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.952+0000 7f511a792700 1 -- 172.21.15.2:0/673902702 learned_addr learned my addr 172.21.15.2:0/673902702 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:16.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.952+0000 7f5119f91700 1 -- 172.21.15.2:0/673902702 >> v1:172.21.15.31:6789/0 conn(0x7f51140b8aa0 legacy=0x7f51140cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:16.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.952+0000 7f5119f91700 1 -- 172.21.15.2:0/673902702 >> v1:172.21.15.2:6789/0 conn(0x7f51140ce4f0 legacy=0x7f51140b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:16.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.952+0000 7f5119f91700 1 -- 172.21.15.2:0/673902702 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51140d2060 con 0x7f511415d370 2023-08-25T03:35:16.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.952+0000 7f5118f8f700 1 -- 172.21.15.2:0/673902702 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1155274181 0 0) 0x7f511415d750 con 0x7f51140ce4f0 2023-08-25T03:35:16.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.952+0000 7f5119f91700 1 --2- 172.21.15.2:0/673902702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f511415d370 0x7f51140d1250 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5110009840 tx=0x7f5110009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55372f655f3601d9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:16.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.952+0000 7f5118f8f700 1 -- 172.21.15.2:0/673902702 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5110004230 con 0x7f511415d370 2023-08-25T03:35:16.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.952+0000 7f5118f8f700 1 -- 172.21.15.2:0/673902702 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5110004390 con 0x7f511415d370 2023-08-25T03:35:16.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.952+0000 7f5118f8f700 1 -- 172.21.15.2:0/673902702 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51100036a0 con 0x7f511415d370 2023-08-25T03:35:16.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.953+0000 7f511c1f5700 1 -- 172.21.15.2:0/673902702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f511415d370 msgr2=0x7f51140d1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:16.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.953+0000 7f511c1f5700 1 --2- 172.21.15.2:0/673902702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f511415d370 0x7f51140d1250 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5110009840 tx=0x7f5110009b50 comp rx=0 tx=0).stop 2023-08-25T03:35:16.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.953+0000 7f511c1f5700 1 -- 172.21.15.2:0/673902702 shutdown_connections 2023-08-25T03:35:16.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.953+0000 7f511c1f5700 1 --2- 172.21.15.2:0/673902702 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f511415d370 0x7f51140d1250 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:16.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.953+0000 7f511c1f5700 1 -- 172.21.15.2:0/673902702 >> 172.21.15.2:0/673902702 conn(0x7f511406ab50 msgr2=0x7f511406af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:16.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.953+0000 7f511c1f5700 1 -- 172.21.15.2:0/673902702 shutdown_connections 2023-08-25T03:35:16.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.953+0000 7f511c1f5700 1 -- 172.21.15.2:0/673902702 wait complete. 2023-08-25T03:35:16.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.954+0000 7f511c1f5700 1 Processor -- start 2023-08-25T03:35:16.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.954+0000 7f511c1f5700 1 -- start start 2023-08-25T03:35:16.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.955+0000 7f511c1f5700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51140b8aa0 0x7f51140cb230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:16.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.955+0000 7f511c1f5700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51140ce4f0 0x7f51141f5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:16.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.955+0000 7f511c1f5700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f511415d370 0x7f51141f6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:16.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.955+0000 7f511c1f5700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51140cc720 con 0x7f511415d370 2023-08-25T03:35:16.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.955+0000 7f511c1f5700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51140ccbb0 con 0x7f51140ce4f0 2023-08-25T03:35:16.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.955+0000 7f511c1f5700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51140cbb10 con 0x7f51140b8aa0 2023-08-25T03:35:16.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.955+0000 7f511a792700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f511415d370 0x7f51141f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:16.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.955+0000 7f511a792700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f511415d370 0x7f51141f6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33354/0 (socket says 172.21.15.2:33354) 2023-08-25T03:35:16.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.955+0000 7f511a792700 1 -- 172.21.15.2:0/3113731633 learned_addr learned my addr 172.21.15.2:0/3113731633 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:16.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.955+0000 7f5119790700 1 --2- 172.21.15.2:0/3113731633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51140ce4f0 0x7f51141f5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:16.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.955+0000 7f5119f91700 1 --2- 172.21.15.2:0/3113731633 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51140b8aa0 0x7f51140cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:16.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.956+0000 7f511a792700 1 -- 172.21.15.2:0/3113731633 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51140b8aa0 msgr2=0x7f51140cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:16.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.956+0000 7f511a792700 1 --2- 172.21.15.2:0/3113731633 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51140b8aa0 0x7f51140cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:16.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.956+0000 7f511a792700 1 -- 172.21.15.2:0/3113731633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51140ce4f0 msgr2=0x7f51141f5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:16.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.956+0000 7f511a792700 1 --2- 172.21.15.2:0/3113731633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51140ce4f0 0x7f51141f5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:16.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.956+0000 7f511a792700 1 -- 172.21.15.2:0/3113731633 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51100094a0 con 0x7f511415d370 2023-08-25T03:35:16.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.956+0000 7f511a792700 1 --2- 172.21.15.2:0/3113731633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f511415d370 0x7f51141f6480 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f510c00cca0 tx=0x7f510c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:16.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.956+0000 7f510a7fc700 1 -- 172.21.15.2:0/3113731633 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c00ab70 con 0x7f511415d370 2023-08-25T03:35:16.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.956+0000 7f510a7fc700 1 -- 172.21.15.2:0/3113731633 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510c00acd0 con 0x7f511415d370 2023-08-25T03:35:16.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.956+0000 7f511c1f5700 1 -- 172.21.15.2:0/3113731633 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51140cbdf0 con 0x7f511415d370 2023-08-25T03:35:16.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.956+0000 7f510a7fc700 1 -- 172.21.15.2:0/3113731633 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c007310 con 0x7f511415d370 2023-08-25T03:35:16.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.956+0000 7f511c1f5700 1 -- 172.21.15.2:0/3113731633 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51140cc3b0 con 0x7f511415d370 2023-08-25T03:35:16.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.957+0000 7f511c1f5700 1 -- 172.21.15.2:0/3113731633 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51140cc8d0 con 0x7f511415d370 2023-08-25T03:35:16.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.957+0000 7f510a7fc700 1 -- 172.21.15.2:0/3113731633 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f510c007470 con 0x7f511415d370 2023-08-25T03:35:16.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.958+0000 7f510a7fc700 1 --2- 172.21.15.2:0/3113731633 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f50f806a0e0 0x7f50f806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:16.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.958+0000 7f510a7fc700 1 -- 172.21.15.2:0/3113731633 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f510c0882e0 con 0x7f511415d370 2023-08-25T03:35:16.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.958+0000 7f5119f91700 1 --2- 172.21.15.2:0/3113731633 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f50f806a0e0 0x7f50f806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:16.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.958+0000 7f5119f91700 1 --2- 172.21.15.2:0/3113731633 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f50f806a0e0 0x7f50f806c5a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f51100090b0 tx=0x7f5110008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:16.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:16.962+0000 7f510a7fc700 1 -- 172.21.15.2:0/3113731633 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f51140cc8d0 con 0x7f511415d370 2023-08-25T03:35:17.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.168+0000 7f511c1f5700 1 -- 172.21.15.2:0/3113731633 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51140cc860 con 0x7f511415d370 2023-08-25T03:35:17.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.169+0000 7f510a7fc700 1 -- 172.21.15.2:0/3113731633 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18207 (secure 0 0 0) 0x7f510c0539d0 con 0x7f511415d370 2023-08-25T03:35:17.171 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:35:17.171 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":37,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:35:08.407381+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:35:05.870790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-08-25T03:35:06.296959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:35:17.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.171+0000 7f511c1f5700 1 -- 172.21.15.2:0/3113731633 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f50f806a0e0 msgr2=0x7f50f806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:17.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.171+0000 7f511c1f5700 1 --2- 172.21.15.2:0/3113731633 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f50f806a0e0 0x7f50f806c5a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f51100090b0 tx=0x7f5110008030 comp rx=0 tx=0).stop 2023-08-25T03:35:17.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.171+0000 7f511c1f5700 1 -- 172.21.15.2:0/3113731633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f511415d370 msgr2=0x7f51141f6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:17.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.171+0000 7f511c1f5700 1 --2- 172.21.15.2:0/3113731633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f511415d370 0x7f51141f6480 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f510c00cca0 tx=0x7f510c00a060 comp rx=0 tx=0).stop 2023-08-25T03:35:17.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.171+0000 7f511c1f5700 1 -- 172.21.15.2:0/3113731633 shutdown_connections 2023-08-25T03:35:17.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.171+0000 7f511c1f5700 1 --2- 172.21.15.2:0/3113731633 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f50f806a0e0 0x7f50f806c5a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:17.174 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.171+0000 7f511c1f5700 1 --2- 172.21.15.2:0/3113731633 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f511415d370 0x7f51141f6480 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:17.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.171+0000 7f511c1f5700 1 --2- 172.21.15.2:0/3113731633 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51140b8aa0 0x7f51140cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:17.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.171+0000 7f511c1f5700 1 --2- 172.21.15.2:0/3113731633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51140ce4f0 0x7f51141f5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:17.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.171+0000 7f511c1f5700 1 -- 172.21.15.2:0/3113731633 >> 172.21.15.2:0/3113731633 conn(0x7f511406ab50 msgr2=0x7f51140b6910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:17.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.171+0000 7f511c1f5700 1 -- 172.21.15.2:0/3113731633 shutdown_connections 2023-08-25T03:35:17.175 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.171+0000 7f511c1f5700 1 -- 172.21.15.2:0/3113731633 wait complete. 2023-08-25T03:35:17.185 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T03:33:49.560643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T03:33:56.265184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T03:35:05.870790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T03:35:06.296959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T03:35:17.186 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T03:35:17.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.313+0000 7f518740d700 1 Processor -- start 2023-08-25T03:35:17.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.313+0000 7f518740d700 1 -- start start 2023-08-25T03:35:17.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.314+0000 7f518740d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f518015d360 0x7f518006b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:17.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.314+0000 7f518740d700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f51800cfd80 0x7f518006ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:17.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.314+0000 7f518740d700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51800d1b40 con 0x7f518006bf80 2023-08-25T03:35:17.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.314+0000 7f518740d700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f518006e0e0 con 0x7f51800cfd80 2023-08-25T03:35:17.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.314+0000 7f518740d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51800d0620 con 0x7f518015d360 2023-08-25T03:35:17.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.314+0000 7f51849a8700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f51800cfd80 0x7f518006ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:17.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.314+0000 7f51859aa700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f518006bf80 0x7f5180160ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48260/0 (socket says 172.21.15.2:48260) 2023-08-25T03:35:17.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.314+0000 7f51859aa700 1 -- 172.21.15.2:0/3832316041 learned_addr learned my addr 172.21.15.2:0/3832316041 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:17.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.315+0000 7f51851a9700 1 --2- 172.21.15.2:0/3832316041 >> v2:172.21.15.31:3300/0 conn(0x7f518015d360 0x7f518006b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:17.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.315+0000 7f51849a8700 1 -- 172.21.15.2:0/3832316041 >> v2:172.21.15.31:3300/0 conn(0x7f518015d360 msgr2=0x7f518006b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:17.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.315+0000 7f51849a8700 1 --2- 172.21.15.2:0/3832316041 >> v2:172.21.15.31:3300/0 conn(0x7f518015d360 0x7f518006b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:17.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.315+0000 7f51849a8700 1 -- 172.21.15.2:0/3832316041 >> v1:172.21.15.2:6789/0 conn(0x7f518006bf80 legacy=0x7f5180160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:17.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.315+0000 7f51849a8700 1 -- 172.21.15.2:0/3832316041 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51800d0340 con 0x7f51800cfd80 2023-08-25T03:35:17.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.315+0000 7f5177fff700 1 -- 172.21.15.2:0/3832316041 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2149037760 0 0) 0x7f51800d1b40 con 0x7f518006bf80 2023-08-25T03:35:17.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.315+0000 7f51849a8700 1 --2- 172.21.15.2:0/3832316041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51800cfd80 0x7f518006ba40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5180066e60 tx=0x7f517c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f43cf387ca49406 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:17.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.315+0000 7f5177fff700 1 -- 172.21.15.2:0/3832316041 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c011830 con 0x7f51800cfd80 2023-08-25T03:35:17.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.315+0000 7f5177fff700 1 -- 172.21.15.2:0/3832316041 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517c011990 con 0x7f51800cfd80 2023-08-25T03:35:17.319 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.315+0000 7f5177fff700 1 -- 172.21.15.2:0/3832316041 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c011d20 con 0x7f51800cfd80 2023-08-25T03:35:17.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.315+0000 7f518740d700 1 -- 172.21.15.2:0/3832316041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51800cfd80 msgr2=0x7f518006ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:17.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.315+0000 7f518740d700 1 --2- 172.21.15.2:0/3832316041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51800cfd80 0x7f518006ba40 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5180066e60 tx=0x7f517c009b20 comp rx=0 tx=0).stop 2023-08-25T03:35:17.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.316+0000 7f518740d700 1 -- 172.21.15.2:0/3832316041 shutdown_connections 2023-08-25T03:35:17.321 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.316+0000 7f518740d700 1 --2- 172.21.15.2:0/3832316041 >> v2:172.21.15.31:3300/0 conn(0x7f518015d360 0x7f518006b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:17.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.316+0000 7f518740d700 1 --2- 172.21.15.2:0/3832316041 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51800cfd80 0x7f518006ba40 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:17.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.316+0000 7f518740d700 1 -- 172.21.15.2:0/3832316041 >> 172.21.15.2:0/3832316041 conn(0x7f518009a200 msgr2=0x7f51800d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:17.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.316+0000 7f518740d700 1 -- 172.21.15.2:0/3832316041 shutdown_connections 2023-08-25T03:35:17.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.316+0000 7f518740d700 1 -- 172.21.15.2:0/3832316041 wait complete. 2023-08-25T03:35:17.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.317+0000 7f518740d700 1 Processor -- start 2023-08-25T03:35:17.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.317+0000 7f518740d700 1 -- start start 2023-08-25T03:35:17.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.318+0000 7f518740d700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f518006bf80 0x7f51801f35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:17.322 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.318+0000 7f518740d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51800cfd80 0x7f5180153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:17.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.318+0000 7f518740d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f518015d360 0x7f51801536b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:17.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.318+0000 7f518740d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51801f2fc0 con 0x7f51800cfd80 2023-08-25T03:35:17.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.318+0000 7f518740d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518015e9d0 con 0x7f518015d360 2023-08-25T03:35:17.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.318+0000 7f518740d700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518015ef90 con 0x7f518006bf80 2023-08-25T03:35:17.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.318+0000 7f51849a8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51800cfd80 0x7f5180153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:17.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.318+0000 7f51849a8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51800cfd80 0x7f5180153170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33366/0 (socket says 172.21.15.2:33366) 2023-08-25T03:35:17.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.318+0000 7f51849a8700 1 -- 172.21.15.2:0/4129272803 learned_addr learned my addr 172.21.15.2:0/4129272803 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:17.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.318+0000 7f51849a8700 1 -- 172.21.15.2:0/4129272803 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f518006bf80 msgr2=0x7f51801f35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:17.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.318+0000 7f51849a8700 1 --2- 172.21.15.2:0/4129272803 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f518006bf80 0x7f51801f35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:17.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.318+0000 7f51849a8700 1 -- 172.21.15.2:0/4129272803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f518015d360 msgr2=0x7f51801536b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:17.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.318+0000 7f51859aa700 1 --2- 172.21.15.2:0/4129272803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f518015d360 0x7f51801536b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:17.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.318+0000 7f51849a8700 1 --2- 172.21.15.2:0/4129272803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f518015d360 0x7f51801536b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:17.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.318+0000 7f51849a8700 1 -- 172.21.15.2:0/4129272803 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f517c0094a0 con 0x7f51800cfd80 2023-08-25T03:35:17.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.318+0000 7f51849a8700 1 --2- 172.21.15.2:0/4129272803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51800cfd80 0x7f5180153170 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f517c0090b0 tx=0x7f517c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:17.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.318+0000 7f5175ffb700 1 -- 172.21.15.2:0/4129272803 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c0038a0 con 0x7f51800cfd80 2023-08-25T03:35:17.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.318+0000 7f5175ffb700 1 -- 172.21.15.2:0/4129272803 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517c01f070 con 0x7f51800cfd80 2023-08-25T03:35:17.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.318+0000 7f518740d700 1 -- 172.21.15.2:0/4129272803 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5180155ee0 con 0x7f51800cfd80 2023-08-25T03:35:17.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.318+0000 7f5175ffb700 1 -- 172.21.15.2:0/4129272803 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c003c00 con 0x7f51800cfd80 2023-08-25T03:35:17.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.319+0000 7f518740d700 1 -- 172.21.15.2:0/4129272803 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f518015ec20 con 0x7f51800cfd80 2023-08-25T03:35:17.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.319+0000 7f518740d700 1 -- 172.21.15.2:0/4129272803 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51800746d0 con 0x7f51800cfd80 2023-08-25T03:35:17.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.320+0000 7f5175ffb700 1 -- 172.21.15.2:0/4129272803 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f517c01a430 con 0x7f51800cfd80 2023-08-25T03:35:17.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.320+0000 7f5175ffb700 1 --2- 172.21.15.2:0/4129272803 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f516406a0e0 0x7f516406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:17.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.320+0000 7f5175ffb700 1 -- 172.21.15.2:0/4129272803 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f517c021070 con 0x7f51800cfd80 2023-08-25T03:35:17.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.321+0000 7f51851a9700 1 --2- 172.21.15.2:0/4129272803 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f516406a0e0 0x7f516406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:17.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.321+0000 7f51851a9700 1 --2- 172.21.15.2:0/4129272803 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f516406a0e0 0x7f516406c5a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5170006eb0 tx=0x7f5170008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:17.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.325+0000 7f5175ffb700 1 -- 172.21.15.2:0/4129272803 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f517c08d030 con 0x7f51800cfd80 2023-08-25T03:35:17.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.530+0000 7f518740d700 1 -- 172.21.15.2:0/4129272803 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f51800ce620 con 0x7f51800cfd80 2023-08-25T03:35:17.532 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.530+0000 7f5175ffb700 1 -- 172.21.15.2:0/4129272803 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f517c02b0a0 con 0x7f51800cfd80 2023-08-25T03:35:17.532 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-08-25T03:35:17.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.532+0000 7f518740d700 1 -- 172.21.15.2:0/4129272803 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f516406a0e0 msgr2=0x7f516406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:17.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.532+0000 7f518740d700 1 --2- 172.21.15.2:0/4129272803 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f516406a0e0 0x7f516406c5a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5170006eb0 tx=0x7f5170008040 comp rx=0 tx=0).stop 2023-08-25T03:35:17.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.532+0000 7f518740d700 1 -- 172.21.15.2:0/4129272803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51800cfd80 msgr2=0x7f5180153170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:17.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.532+0000 7f518740d700 1 --2- 172.21.15.2:0/4129272803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51800cfd80 0x7f5180153170 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f517c0090b0 tx=0x7f517c004750 comp rx=0 tx=0).stop 2023-08-25T03:35:17.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.532+0000 7f518740d700 1 -- 172.21.15.2:0/4129272803 shutdown_connections 2023-08-25T03:35:17.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.532+0000 7f518740d700 1 --2- 172.21.15.2:0/4129272803 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f518015d360 0x7f51801536b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:17.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.532+0000 7f518740d700 1 --2- 172.21.15.2:0/4129272803 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f518006bf80 0x7f51801f35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:17.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.532+0000 7f518740d700 1 --2- 172.21.15.2:0/4129272803 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f516406a0e0 0x7f516406c5a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:17.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.532+0000 7f518740d700 1 --2- 172.21.15.2:0/4129272803 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f51800cfd80 0x7f5180153170 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:17.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.532+0000 7f518740d700 1 -- 172.21.15.2:0/4129272803 >> 172.21.15.2:0/4129272803 conn(0x7f518009a200 msgr2=0x7f518015c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:17.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.533+0000 7f518740d700 1 -- 172.21.15.2:0/4129272803 shutdown_connections 2023-08-25T03:35:17.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.533+0000 7f518740d700 1 -- 172.21.15.2:0/4129272803 wait complete. 2023-08-25T03:35:17.546 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T03:35:17.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.673+0000 7fa6da3b3700 1 Processor -- start 2023-08-25T03:35:17.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.673+0000 7fa6da3b3700 1 -- start start 2023-08-25T03:35:17.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.674+0000 7fa6da3b3700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa6d41607f0 0x7fa6d4070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:17.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.674+0000 7fa6da3b3700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa6d40af180 0x7fa6d40ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:17.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.674+0000 7fa6da3b3700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6d4160bd0 con 0x7fa6d406a1e0 2023-08-25T03:35:17.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.674+0000 7fa6da3b3700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6d40713a0 con 0x7fa6d40af180 2023-08-25T03:35:17.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.674+0000 7fa6da3b3700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d40718a0 con 0x7fa6d41607f0 2023-08-25T03:35:17.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.674+0000 7fa6d37fe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fa6d40af180 0x7fa6d40ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:17.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.674+0000 7fa6d8950700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fa6d406a1e0 0x7fa6d406ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48274/0 (socket says 172.21.15.2:48274) 2023-08-25T03:35:17.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.674+0000 7fa6d8950700 1 -- 172.21.15.2:0/556512844 learned_addr learned my addr 172.21.15.2:0/556512844 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:17.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.674+0000 7fa6d3fff700 1 --2- 172.21.15.2:0/556512844 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa6d41607f0 0x7fa6d4070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:17.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.674+0000 7fa6d37fe700 1 -- 172.21.15.2:0/556512844 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa6d41607f0 msgr2=0x7fa6d4070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:17.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.674+0000 7fa6d37fe700 1 --2- 172.21.15.2:0/556512844 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa6d41607f0 0x7fa6d4070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:17.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.674+0000 7fa6d37fe700 1 -- 172.21.15.2:0/556512844 >> v1:172.21.15.2:6789/0 conn(0x7fa6d406a1e0 legacy=0x7fa6d406ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:17.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.674+0000 7fa6d37fe700 1 -- 172.21.15.2:0/556512844 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d40cdd00 con 0x7fa6d40af180 2023-08-25T03:35:17.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.675+0000 7fa6d37fe700 1 --2- 172.21.15.2:0/556512844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6d40af180 0x7fa6d40ac8e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009840 tx=0x7fa6c4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304c32d8740c30ec server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:17.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.675+0000 7fa6d2ffd700 1 -- 172.21.15.2:0/556512844 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4011830 con 0x7fa6d40af180 2023-08-25T03:35:17.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.675+0000 7fa6d2ffd700 1 -- 172.21.15.2:0/556512844 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c4011990 con 0x7fa6d40af180 2023-08-25T03:35:17.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.675+0000 7fa6d2ffd700 1 -- 172.21.15.2:0/556512844 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4011d20 con 0x7fa6d40af180 2023-08-25T03:35:17.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.675+0000 7fa6da3b3700 1 -- 172.21.15.2:0/556512844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6d40af180 msgr2=0x7fa6d40ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:17.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.675+0000 7fa6da3b3700 1 --2- 172.21.15.2:0/556512844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6d40af180 0x7fa6d40ac8e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009840 tx=0x7fa6c4009b50 comp rx=0 tx=0).stop 2023-08-25T03:35:17.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.675+0000 7fa6da3b3700 1 -- 172.21.15.2:0/556512844 shutdown_connections 2023-08-25T03:35:17.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.675+0000 7fa6da3b3700 1 --2- 172.21.15.2:0/556512844 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa6d41607f0 0x7fa6d4070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:17.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.675+0000 7fa6da3b3700 1 --2- 172.21.15.2:0/556512844 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6d40af180 0x7fa6d40ac8e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:17.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.675+0000 7fa6da3b3700 1 -- 172.21.15.2:0/556512844 >> 172.21.15.2:0/556512844 conn(0x7fa6d40cfd90 msgr2=0x7fa6d40d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:17.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.675+0000 7fa6da3b3700 1 -- 172.21.15.2:0/556512844 shutdown_connections 2023-08-25T03:35:17.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.675+0000 7fa6da3b3700 1 -- 172.21.15.2:0/556512844 wait complete. 2023-08-25T03:35:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.677+0000 7fa6da3b3700 1 Processor -- start 2023-08-25T03:35:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.677+0000 7fa6da3b3700 1 -- start start 2023-08-25T03:35:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.678+0000 7fa6da3b3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6d406a1e0 0x7fa6d4153040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.678+0000 7fa6da3b3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6d40af180 0x7fa6d40c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.678+0000 7fa6da3b3700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa6d41607f0 0x7fa6d40cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.678+0000 7fa6da3b3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d40c7dc0 con 0x7fa6d40af180 2023-08-25T03:35:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.678+0000 7fa6da3b3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d40c80a0 con 0x7fa6d406a1e0 2023-08-25T03:35:17.683 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.678+0000 7fa6da3b3700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6d40c8380 con 0x7fa6d41607f0 2023-08-25T03:35:17.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.678+0000 7fa6d37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6d40af180 0x7fa6d40c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:17.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.678+0000 7fa6d37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6d40af180 0x7fa6d40c8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33380/0 (socket says 172.21.15.2:33380) 2023-08-25T03:35:17.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.678+0000 7fa6d37fe700 1 -- 172.21.15.2:0/2090414437 learned_addr learned my addr 172.21.15.2:0/2090414437 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:17.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.678+0000 7fa6d37fe700 1 -- 172.21.15.2:0/2090414437 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa6d41607f0 msgr2=0x7fa6d40cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:17.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.678+0000 7fa6d8950700 1 --2- 172.21.15.2:0/2090414437 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa6d41607f0 0x7fa6d40cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:17.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.678+0000 7fa6d3fff700 1 --2- 172.21.15.2:0/2090414437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6d406a1e0 0x7fa6d4153040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:17.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.678+0000 7fa6d37fe700 1 --2- 172.21.15.2:0/2090414437 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa6d41607f0 0x7fa6d40cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:17.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.678+0000 7fa6d37fe700 1 -- 172.21.15.2:0/2090414437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6d406a1e0 msgr2=0x7fa6d4153040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:17.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.678+0000 7fa6d37fe700 1 --2- 172.21.15.2:0/2090414437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6d406a1e0 0x7fa6d4153040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:17.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.678+0000 7fa6d37fe700 1 -- 172.21.15.2:0/2090414437 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c40094a0 con 0x7fa6d40af180 2023-08-25T03:35:17.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.678+0000 7fa6d3fff700 1 --2- 172.21.15.2:0/2090414437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6d406a1e0 0x7fa6d4153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:35:17.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.678+0000 7fa6d37fe700 1 --2- 172.21.15.2:0/2090414437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6d40af180 0x7fa6d40c8e60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009b90 tx=0x7fa6c4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:17.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.678+0000 7fa6d0ff9700 1 -- 172.21.15.2:0/2090414437 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c40036a0 con 0x7fa6d40af180 2023-08-25T03:35:17.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.678+0000 7fa6d0ff9700 1 -- 172.21.15.2:0/2090414437 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6c401f070 con 0x7fa6d40af180 2023-08-25T03:35:17.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.678+0000 7fa6da3b3700 1 -- 172.21.15.2:0/2090414437 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6d40c8660 con 0x7fa6d40af180 2023-08-25T03:35:17.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.678+0000 7fa6d0ff9700 1 -- 172.21.15.2:0/2090414437 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6c4003a40 con 0x7fa6d40af180 2023-08-25T03:35:17.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.678+0000 7fa6da3b3700 1 -- 172.21.15.2:0/2090414437 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d40c93a0 con 0x7fa6d40af180 2023-08-25T03:35:17.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.679+0000 7fa6da3b3700 1 -- 172.21.15.2:0/2090414437 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6d40c8b80 con 0x7fa6d40af180 2023-08-25T03:35:17.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.680+0000 7fa6d0ff9700 1 -- 172.21.15.2:0/2090414437 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fa6c4003ba0 con 0x7fa6d40af180 2023-08-25T03:35:17.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.680+0000 7fa6d0ff9700 1 --2- 172.21.15.2:0/2090414437 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa6c006a0e0 0x7fa6c006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:17.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.681+0000 7fa6d3fff700 1 --2- 172.21.15.2:0/2090414437 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa6c006a0e0 0x7fa6c006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:17.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.681+0000 7fa6d0ff9700 1 -- 172.21.15.2:0/2090414437 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fa6c4021070 con 0x7fa6d40af180 2023-08-25T03:35:17.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.681+0000 7fa6d3fff700 1 --2- 172.21.15.2:0/2090414437 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa6c006a0e0 0x7fa6c006c5a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8009820 tx=0x7fa6c8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:17.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.685+0000 7fa6d0ff9700 1 -- 172.21.15.2:0/2090414437 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fa6c40556f0 con 0x7fa6d40af180 2023-08-25T03:35:17.882 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.881+0000 7fa6da3b3700 1 -- 172.21.15.2:0/2090414437 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa6d40746d0 con 0x7fa6d40af180 2023-08-25T03:35:17.883 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.881+0000 7fa6d0ff9700 1 -- 172.21.15.2:0/2090414437 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v37) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa6c402b0a0 con 0x7fa6d40af180 2023-08-25T03:35:17.883 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 8 2023-08-25T03:35:17.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.883+0000 7fa6da3b3700 1 -- 172.21.15.2:0/2090414437 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa6c006a0e0 msgr2=0x7fa6c006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:17.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.883+0000 7fa6da3b3700 1 --2- 172.21.15.2:0/2090414437 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa6c006a0e0 0x7fa6c006c5a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8009820 tx=0x7fa6c8008040 comp rx=0 tx=0).stop 2023-08-25T03:35:17.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.883+0000 7fa6da3b3700 1 -- 172.21.15.2:0/2090414437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6d40af180 msgr2=0x7fa6d40c8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:17.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.883+0000 7fa6da3b3700 1 --2- 172.21.15.2:0/2090414437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6d40af180 0x7fa6d40c8e60 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009b90 tx=0x7fa6c4004750 comp rx=0 tx=0).stop 2023-08-25T03:35:17.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.883+0000 7fa6da3b3700 1 -- 172.21.15.2:0/2090414437 shutdown_connections 2023-08-25T03:35:17.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.883+0000 7fa6da3b3700 1 --2- 172.21.15.2:0/2090414437 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa6d41607f0 0x7fa6d40cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:17.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.883+0000 7fa6da3b3700 1 --2- 172.21.15.2:0/2090414437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa6d406a1e0 0x7fa6d4153040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:17.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.883+0000 7fa6da3b3700 1 --2- 172.21.15.2:0/2090414437 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fa6c006a0e0 0x7fa6c006c5a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:17.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.883+0000 7fa6da3b3700 1 --2- 172.21.15.2:0/2090414437 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fa6d40af180 0x7fa6d40c8e60 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:17.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.883+0000 7fa6da3b3700 1 -- 172.21.15.2:0/2090414437 >> 172.21.15.2:0/2090414437 conn(0x7fa6d40cfd90 msgr2=0x7fa6d40af9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:17.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.883+0000 7fa6da3b3700 1 -- 172.21.15.2:0/2090414437 shutdown_connections 2023-08-25T03:35:17.886 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:17.883+0000 7fa6da3b3700 1 -- 172.21.15.2:0/2090414437 wait complete. 2023-08-25T03:35:17.897 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T03:35:18.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.024+0000 7f750c71b700 1 Processor -- start 2023-08-25T03:35:18.026 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.024+0000 7f750c71b700 1 -- start start 2023-08-25T03:35:18.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.025+0000 7f750c71b700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75040af180 0x7f7504070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:18.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.025+0000 7f750c71b700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f750406a1e0 0x7f750406a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:18.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.025+0000 7f750c71b700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7504160bd0 con 0x7f75041607f0 2023-08-25T03:35:18.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.025+0000 7f750c71b700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75040713a0 con 0x7f750406a1e0 2023-08-25T03:35:18.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.025+0000 7f750c71b700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75040718a0 con 0x7f75040af180 2023-08-25T03:35:18.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.025+0000 7f750acb8700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f750406a1e0 0x7f750406a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:18.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.026+0000 7f750acb8700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f750406a1e0 0x7f750406a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33390/0 (socket says 172.21.15.2:33390) 2023-08-25T03:35:18.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.026+0000 7f750acb8700 1 -- 172.21.15.2:0/4077014022 learned_addr learned my addr 172.21.15.2:0/4077014022 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:18.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.026+0000 7f7509cb6700 1 --2- 172.21.15.2:0/4077014022 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75040af180 0x7f7504070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:18.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.026+0000 7f750acb8700 1 -- 172.21.15.2:0/4077014022 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75040af180 msgr2=0x7f7504070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:18.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.026+0000 7f750acb8700 1 --2- 172.21.15.2:0/4077014022 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75040af180 0x7f7504070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.026+0000 7f750acb8700 1 -- 172.21.15.2:0/4077014022 >> v1:172.21.15.31:6789/0 conn(0x7f75041607f0 legacy=0x7f75040ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:18.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.026+0000 7f750acb8700 1 -- 172.21.15.2:0/4077014022 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75040cdd00 con 0x7f750406a1e0 2023-08-25T03:35:18.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.026+0000 7f750acb8700 1 --2- 172.21.15.2:0/4077014022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f750406a1e0 0x7f750406a620 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009e50 tx=0x7f74fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0448920a627e40 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:18.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.026+0000 7f75094b5700 1 -- 172.21.15.2:0/4077014022 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74fc00e070 con 0x7f750406a1e0 2023-08-25T03:35:18.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.026+0000 7f75094b5700 1 -- 172.21.15.2:0/4077014022 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74fc00b9c0 con 0x7f750406a1e0 2023-08-25T03:35:18.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.026+0000 7f75094b5700 1 -- 172.21.15.2:0/4077014022 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74fc00bd70 con 0x7f750406a1e0 2023-08-25T03:35:18.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.027+0000 7f750c71b700 1 -- 172.21.15.2:0/4077014022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f750406a1e0 msgr2=0x7f750406a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:18.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.027+0000 7f750c71b700 1 --2- 172.21.15.2:0/4077014022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f750406a1e0 0x7f750406a620 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f74fc009e50 tx=0x7f74fc00b330 comp rx=0 tx=0).stop 2023-08-25T03:35:18.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.027+0000 7f750c71b700 1 -- 172.21.15.2:0/4077014022 shutdown_connections 2023-08-25T03:35:18.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.027+0000 7f750c71b700 1 --2- 172.21.15.2:0/4077014022 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f750406a1e0 0x7f750406a620 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.027+0000 7f750c71b700 1 --2- 172.21.15.2:0/4077014022 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75040af180 0x7f7504070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.027+0000 7f750c71b700 1 -- 172.21.15.2:0/4077014022 >> 172.21.15.2:0/4077014022 conn(0x7f75040cfd90 msgr2=0x7f75040d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:18.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.027+0000 7f750c71b700 1 -- 172.21.15.2:0/4077014022 shutdown_connections 2023-08-25T03:35:18.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.027+0000 7f750c71b700 1 -- 172.21.15.2:0/4077014022 wait complete. 2023-08-25T03:35:18.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.028+0000 7f750c71b700 1 Processor -- start 2023-08-25T03:35:18.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.028+0000 7f750c71b700 1 -- start start 2023-08-25T03:35:18.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.029+0000 7f750c71b700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f750406a1e0 0x7f75040c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:18.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.029+0000 7f750c71b700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75040af180 0x7f75041577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:18.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.029+0000 7f750c71b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75041607f0 0x7f75040c4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:18.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.029+0000 7f750c71b700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75040cbbb0 con 0x7f75041607f0 2023-08-25T03:35:18.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.029+0000 7f750c71b700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75040c8a20 con 0x7f750406a1e0 2023-08-25T03:35:18.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.029+0000 7f750c71b700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75040c8d00 con 0x7f75040af180 2023-08-25T03:35:18.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.029+0000 7f750acb8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75041607f0 0x7f75040c4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:18.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.029+0000 7f750acb8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75041607f0 0x7f75040c4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33396/0 (socket says 172.21.15.2:33396) 2023-08-25T03:35:18.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.029+0000 7f750acb8700 1 -- 172.21.15.2:0/854142364 learned_addr learned my addr 172.21.15.2:0/854142364 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:18.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.029+0000 7f750a4b7700 1 --2- 172.21.15.2:0/854142364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f750406a1e0 0x7f75040c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:18.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.029+0000 7f7509cb6700 1 --2- 172.21.15.2:0/854142364 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75040af180 0x7f75041577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:18.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.029+0000 7f750acb8700 1 -- 172.21.15.2:0/854142364 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75040af180 msgr2=0x7f75041577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:18.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.029+0000 7f750acb8700 1 --2- 172.21.15.2:0/854142364 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75040af180 0x7f75041577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.029+0000 7f750acb8700 1 -- 172.21.15.2:0/854142364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f750406a1e0 msgr2=0x7f75040c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:18.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.029+0000 7f750acb8700 1 --2- 172.21.15.2:0/854142364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f750406a1e0 0x7f75040c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.029+0000 7f750acb8700 1 -- 172.21.15.2:0/854142364 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74fc009b30 con 0x7f75041607f0 2023-08-25T03:35:18.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.029+0000 7f7509cb6700 1 --2- 172.21.15.2:0/854142364 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75040af180 0x7f75041577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:35:18.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.029+0000 7f750acb8700 1 --2- 172.21.15.2:0/854142364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75041607f0 0x7f75040c4550 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f74fc00b370 tx=0x7f74fc004b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:18.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.029+0000 7f74f6ffd700 1 -- 172.21.15.2:0/854142364 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74fc0092e0 con 0x7f75041607f0 2023-08-25T03:35:18.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.029+0000 7f74f6ffd700 1 -- 172.21.15.2:0/854142364 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74fc0099b0 con 0x7f75041607f0 2023-08-25T03:35:18.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.029+0000 7f750c71b700 1 -- 172.21.15.2:0/854142364 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75040c8fe0 con 0x7f75041607f0 2023-08-25T03:35:18.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.029+0000 7f74f6ffd700 1 -- 172.21.15.2:0/854142364 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74fc01b650 con 0x7f75041607f0 2023-08-25T03:35:18.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.030+0000 7f750c71b700 1 -- 172.21.15.2:0/854142364 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75040c9d20 con 0x7f75041607f0 2023-08-25T03:35:18.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.030+0000 7f750c71b700 1 -- 172.21.15.2:0/854142364 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75040c94a0 con 0x7f75041607f0 2023-08-25T03:35:18.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.031+0000 7f74f6ffd700 1 -- 172.21.15.2:0/854142364 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f74fc01b7b0 con 0x7f75041607f0 2023-08-25T03:35:18.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.031+0000 7f74f6ffd700 1 --2- 172.21.15.2:0/854142364 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f74e806a0e0 0x7f74e806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:18.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.031+0000 7f74f6ffd700 1 -- 172.21.15.2:0/854142364 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f74fc089b50 con 0x7f75041607f0 2023-08-25T03:35:18.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.031+0000 7f750a4b7700 1 --2- 172.21.15.2:0/854142364 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f74e806a0e0 0x7f74e806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:18.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.032+0000 7f750a4b7700 1 --2- 172.21.15.2:0/854142364 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f74e806a0e0 0x7f74e806c5a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7500000f60 tx=0x7f7500009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:18.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.036+0000 7f74f6ffd700 1 -- 172.21.15.2:0/854142364 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f74fc0552d0 con 0x7f75041607f0 2023-08-25T03:35:18.237 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.236+0000 7f750c71b700 1 -- 172.21.15.2:0/854142364 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f75040746d0 con 0x7f75041607f0 2023-08-25T03:35:18.238 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.236+0000 7f74f6ffd700 1 -- 172.21.15.2:0/854142364 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7f74fc01f0a0 con 0x7f75041607f0 2023-08-25T03:35:18.238 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-08-25T03:35:18.241 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.238+0000 7f750c71b700 1 -- 172.21.15.2:0/854142364 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f74e806a0e0 msgr2=0x7f74e806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.238+0000 7f750c71b700 1 --2- 172.21.15.2:0/854142364 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f74e806a0e0 0x7f74e806c5a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7500000f60 tx=0x7f7500009040 comp rx=0 tx=0).stop 2023-08-25T03:35:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.238+0000 7f750c71b700 1 -- 172.21.15.2:0/854142364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75041607f0 msgr2=0x7f75040c4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:18.242 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.238+0000 7f750c71b700 1 --2- 172.21.15.2:0/854142364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75041607f0 0x7f75040c4550 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f74fc00b370 tx=0x7f74fc004b20 comp rx=0 tx=0).stop 2023-08-25T03:35:18.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.238+0000 7f750c71b700 1 -- 172.21.15.2:0/854142364 shutdown_connections 2023-08-25T03:35:18.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.238+0000 7f750c71b700 1 --2- 172.21.15.2:0/854142364 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f74e806a0e0 0x7f74e806c5a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.238+0000 7f750c71b700 1 --2- 172.21.15.2:0/854142364 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f75041607f0 0x7f75040c4550 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.243 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.238+0000 7f750c71b700 1 --2- 172.21.15.2:0/854142364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f750406a1e0 0x7f75040c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.238+0000 7f750c71b700 1 --2- 172.21.15.2:0/854142364 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75040af180 0x7f75041577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.238+0000 7f750c71b700 1 -- 172.21.15.2:0/854142364 >> 172.21.15.2:0/854142364 conn(0x7f75040cfd90 msgr2=0x7f75040afd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:18.244 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.238+0000 7f750c71b700 1 -- 172.21.15.2:0/854142364 shutdown_connections 2023-08-25T03:35:18.245 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.238+0000 7f750c71b700 1 -- 172.21.15.2:0/854142364 wait complete. 2023-08-25T03:35:18.252 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T03:35:18.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.379+0000 7f6384943700 1 Processor -- start 2023-08-25T03:35:18.381 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.379+0000 7f6384943700 1 -- start start 2023-08-25T03:35:18.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.380+0000 7f6384943700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f638015d360 0x7f638006b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:18.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.380+0000 7f6384943700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f63800cfd80 0x7f638006ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:18.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.380+0000 7f6384943700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63800d1b40 con 0x7f638006bf80 2023-08-25T03:35:18.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.380+0000 7f6384943700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f638006e0e0 con 0x7f638015d360 2023-08-25T03:35:18.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.380+0000 7f6384943700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63800d0620 con 0x7f63800cfd80 2023-08-25T03:35:18.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.380+0000 7f637e59c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f638015d360 0x7f638006b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:18.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.380+0000 7f637ed9d700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f638006bf80 0x7f6380160ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48302/0 (socket says 172.21.15.2:48302) 2023-08-25T03:35:18.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.380+0000 7f637ed9d700 1 -- 172.21.15.2:0/3490138233 learned_addr learned my addr 172.21.15.2:0/3490138233 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:18.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.380+0000 7f637e59c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f638015d360 0x7f638006b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33400/0 (socket says 172.21.15.2:33400) 2023-08-25T03:35:18.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.381+0000 7f637dd9b700 1 --2- 172.21.15.2:0/3490138233 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f63800cfd80 0x7f638006ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:18.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.381+0000 7f637e59c700 1 -- 172.21.15.2:0/3490138233 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f63800cfd80 msgr2=0x7f638006ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:18.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.381+0000 7f637e59c700 1 --2- 172.21.15.2:0/3490138233 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f63800cfd80 0x7f638006ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.381+0000 7f637e59c700 1 -- 172.21.15.2:0/3490138233 >> v1:172.21.15.2:6789/0 conn(0x7f638006bf80 legacy=0x7f6380160ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:18.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.381+0000 7f637e59c700 1 -- 172.21.15.2:0/3490138233 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63800d0340 con 0x7f638015d360 2023-08-25T03:35:18.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.381+0000 7f637d59a700 1 -- 172.21.15.2:0/3490138233 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2510011329 0 0) 0x7f63800d1b40 con 0x7f638006bf80 2023-08-25T03:35:18.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.381+0000 7f637e59c700 1 --2- 172.21.15.2:0/3490138233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f638015d360 0x7f638006b500 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6368009820 tx=0x7f6368009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c497e2a48d0a3a4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:18.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.381+0000 7f637d59a700 1 -- 172.21.15.2:0/3490138233 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6368005e30 con 0x7f638015d360 2023-08-25T03:35:18.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.381+0000 7f637d59a700 1 -- 172.21.15.2:0/3490138233 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6368006030 con 0x7f638015d360 2023-08-25T03:35:18.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.381+0000 7f637d59a700 1 -- 172.21.15.2:0/3490138233 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63680063e0 con 0x7f638015d360 2023-08-25T03:35:18.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.382+0000 7f6384943700 1 -- 172.21.15.2:0/3490138233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f638015d360 msgr2=0x7f638006b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:18.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.382+0000 7f6384943700 1 --2- 172.21.15.2:0/3490138233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f638015d360 0x7f638006b500 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6368009820 tx=0x7f6368009b30 comp rx=0 tx=0).stop 2023-08-25T03:35:18.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.382+0000 7f6384943700 1 -- 172.21.15.2:0/3490138233 shutdown_connections 2023-08-25T03:35:18.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.382+0000 7f6384943700 1 --2- 172.21.15.2:0/3490138233 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f638015d360 0x7f638006b500 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.382+0000 7f6384943700 1 --2- 172.21.15.2:0/3490138233 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f63800cfd80 0x7f638006ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.382+0000 7f6384943700 1 -- 172.21.15.2:0/3490138233 >> 172.21.15.2:0/3490138233 conn(0x7f638009a200 msgr2=0x7f63800d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:18.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.382+0000 7f6384943700 1 -- 172.21.15.2:0/3490138233 shutdown_connections 2023-08-25T03:35:18.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.382+0000 7f6384943700 1 -- 172.21.15.2:0/3490138233 wait complete. 2023-08-25T03:35:18.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.383+0000 7f6384943700 1 Processor -- start 2023-08-25T03:35:18.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.383+0000 7f6384943700 1 -- start start 2023-08-25T03:35:18.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.384+0000 7f6384943700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f638006bf80 0x7f638015f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:18.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.384+0000 7f6384943700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f63800cfd80 0x7f638009ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:18.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.384+0000 7f6384943700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f638015d360 0x7f638009e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:18.390 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.384+0000 7f6384943700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63800b1210 con 0x7f63800cfd80 2023-08-25T03:35:18.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.384+0000 7f6384943700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6380149b40 con 0x7f638006bf80 2023-08-25T03:35:18.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.384+0000 7f6384943700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63800b0870 con 0x7f638015d360 2023-08-25T03:35:18.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.384+0000 7f637dd9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f63800cfd80 0x7f638009ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:18.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.384+0000 7f637dd9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f63800cfd80 0x7f638009ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33408/0 (socket says 172.21.15.2:33408) 2023-08-25T03:35:18.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.384+0000 7f637dd9b700 1 -- 172.21.15.2:0/2112920089 learned_addr learned my addr 172.21.15.2:0/2112920089 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:18.391 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.384+0000 7f637e59c700 1 --2- 172.21.15.2:0/2112920089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f638006bf80 0x7f638015f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:18.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.384+0000 7f637ed9d700 1 --2- 172.21.15.2:0/2112920089 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f638015d360 0x7f638009e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:18.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.384+0000 7f637dd9b700 1 -- 172.21.15.2:0/2112920089 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f638015d360 msgr2=0x7f638009e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:18.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.384+0000 7f637dd9b700 1 --2- 172.21.15.2:0/2112920089 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f638015d360 0x7f638009e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.384+0000 7f637dd9b700 1 -- 172.21.15.2:0/2112920089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f638006bf80 msgr2=0x7f638015f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:18.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.384+0000 7f637dd9b700 1 --2- 172.21.15.2:0/2112920089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f638006bf80 0x7f638015f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.384+0000 7f637dd9b700 1 -- 172.21.15.2:0/2112920089 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6368009430 con 0x7f63800cfd80 2023-08-25T03:35:18.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.385+0000 7f637dd9b700 1 --2- 172.21.15.2:0/2112920089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f63800cfd80 0x7f638009ded0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f637000c7f0 tx=0x7f637000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:18.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.385+0000 7f6376ffd700 1 -- 172.21.15.2:0/2112920089 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6370004220 con 0x7f63800cfd80 2023-08-25T03:35:18.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.385+0000 7f6376ffd700 1 -- 172.21.15.2:0/2112920089 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6370004380 con 0x7f63800cfd80 2023-08-25T03:35:18.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.385+0000 7f6376ffd700 1 -- 172.21.15.2:0/2112920089 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63700036a0 con 0x7f63800cfd80 2023-08-25T03:35:18.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.385+0000 7f6384943700 1 -- 172.21.15.2:0/2112920089 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63800b0b50 con 0x7f63800cfd80 2023-08-25T03:35:18.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.385+0000 7f6384943700 1 -- 172.21.15.2:0/2112920089 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63801f6a40 con 0x7f63800cfd80 2023-08-25T03:35:18.393 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.386+0000 7f6384943700 1 -- 172.21.15.2:0/2112920089 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63800b1350 con 0x7f63800cfd80 2023-08-25T03:35:18.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.386+0000 7f6376ffd700 1 -- 172.21.15.2:0/2112920089 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6370003800 con 0x7f63800cfd80 2023-08-25T03:35:18.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.387+0000 7f6376ffd700 1 --2- 172.21.15.2:0/2112920089 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f636006a0e0 0x7f636006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:18.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.387+0000 7f6376ffd700 1 -- 172.21.15.2:0/2112920089 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6370088030 con 0x7f63800cfd80 2023-08-25T03:35:18.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.387+0000 7f637e59c700 1 --2- 172.21.15.2:0/2112920089 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f636006a0e0 0x7f636006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:18.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.387+0000 7f637e59c700 1 --2- 172.21.15.2:0/2112920089 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f636006a0e0 0x7f636006c5a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6368013040 tx=0x7f6368008070 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:18.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.391+0000 7f6376ffd700 1 -- 172.21.15.2:0/2112920089 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f63700ba990 con 0x7f63800cfd80 2023-08-25T03:35:18.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.591+0000 7f6384943700 1 -- 172.21.15.2:0/2112920089 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f63800746d0 con 0x7f63800cfd80 2023-08-25T03:35:18.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.592+0000 7f6376ffd700 1 -- 172.21.15.2:0/2112920089 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7f637001e0a0 con 0x7f63800cfd80 2023-08-25T03:35:18.594 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-08-25T03:35:18.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.594+0000 7f6384943700 1 -- 172.21.15.2:0/2112920089 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f636006a0e0 msgr2=0x7f636006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:18.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.594+0000 7f6384943700 1 --2- 172.21.15.2:0/2112920089 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f636006a0e0 0x7f636006c5a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6368013040 tx=0x7f6368008070 comp rx=0 tx=0).stop 2023-08-25T03:35:18.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.594+0000 7f6384943700 1 -- 172.21.15.2:0/2112920089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f63800cfd80 msgr2=0x7f638009ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:18.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.594+0000 7f6384943700 1 --2- 172.21.15.2:0/2112920089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f63800cfd80 0x7f638009ded0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f637000c7f0 tx=0x7f637000cb00 comp rx=0 tx=0).stop 2023-08-25T03:35:18.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.594+0000 7f6384943700 1 -- 172.21.15.2:0/2112920089 shutdown_connections 2023-08-25T03:35:18.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.594+0000 7f6384943700 1 --2- 172.21.15.2:0/2112920089 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f638015d360 0x7f638009e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.594+0000 7f6384943700 1 --2- 172.21.15.2:0/2112920089 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f638006bf80 0x7f638015f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.594+0000 7f6384943700 1 --2- 172.21.15.2:0/2112920089 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f636006a0e0 0x7f636006c5a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.594+0000 7f6384943700 1 --2- 172.21.15.2:0/2112920089 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f63800cfd80 0x7f638009ded0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.594+0000 7f6384943700 1 -- 172.21.15.2:0/2112920089 >> 172.21.15.2:0/2112920089 conn(0x7f638009a200 msgr2=0x7f63800af9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:18.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.594+0000 7f6384943700 1 -- 172.21.15.2:0/2112920089 shutdown_connections 2023-08-25T03:35:18.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.594+0000 7f6384943700 1 -- 172.21.15.2:0/2112920089 wait complete. 2023-08-25T03:35:18.608 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T03:35:18.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.732+0000 7f7d4e784700 1 Processor -- start 2023-08-25T03:35:18.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.732+0000 7f7d4e784700 1 -- start start 2023-08-25T03:35:18.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.733+0000 7f7d4e784700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7d480cdb70 0x7f7d4806b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:18.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.733+0000 7f7d4e784700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7d480ce000 0x7f7d4806be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:18.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.733+0000 7f7d4e784700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d48161620 con 0x7f7d480b1fb0 2023-08-25T03:35:18.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.733+0000 7f7d4e784700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d4806d450 con 0x7f7d480ce000 2023-08-25T03:35:18.742 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.733+0000 7f7d4e784700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d4806d730 con 0x7f7d480cdb70 2023-08-25T03:35:18.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.733+0000 7f7d477fe700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7d480cdb70 0x7f7d4806b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:18.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.733+0000 7f7d4cd21700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7d480ce000 0x7f7d4806be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:18.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.733+0000 7f7d477fe700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7d480cdb70 0x7f7d4806b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.2:33660/0 (socket says 172.21.15.2:33660) 2023-08-25T03:35:18.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.733+0000 7f7d477fe700 1 -- 172.21.15.2:0/1399801223 learned_addr learned my addr 172.21.15.2:0/1399801223 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:18.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.734+0000 7f7d477fe700 1 -- 172.21.15.2:0/1399801223 >> v2:172.21.15.31:3300/0 conn(0x7f7d480ce000 msgr2=0x7f7d4806be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:18.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.734+0000 7f7d477fe700 1 --2- 172.21.15.2:0/1399801223 >> v2:172.21.15.31:3300/0 conn(0x7f7d480ce000 0x7f7d4806be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.743 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.734+0000 7f7d477fe700 1 -- 172.21.15.2:0/1399801223 >> v1:172.21.15.31:6789/0 conn(0x7f7d480b1fb0 legacy=0x7f7d48160f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:18.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.734+0000 7f7d477fe700 1 -- 172.21.15.2:0/1399801223 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4806da10 con 0x7f7d480cdb70 2023-08-25T03:35:18.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.734+0000 7f7d477fe700 1 --2- 172.21.15.2:0/1399801223 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7d480cdb70 0x7f7d4806b8c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7d380098f0 tx=0x7f7d38009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=838835d4b8ed9f63 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:18.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.734+0000 7f7d46ffd700 1 -- 172.21.15.2:0/1399801223 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d38011830 con 0x7f7d480cdb70 2023-08-25T03:35:18.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.734+0000 7f7d46ffd700 1 -- 172.21.15.2:0/1399801223 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d38011990 con 0x7f7d480cdb70 2023-08-25T03:35:18.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.734+0000 7f7d46ffd700 1 -- 172.21.15.2:0/1399801223 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d38011d20 con 0x7f7d480cdb70 2023-08-25T03:35:18.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.734+0000 7f7d4e784700 1 -- 172.21.15.2:0/1399801223 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7d480cdb70 msgr2=0x7f7d4806b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:18.744 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.734+0000 7f7d4e784700 1 --2- 172.21.15.2:0/1399801223 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7d480cdb70 0x7f7d4806b8c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7d380098f0 tx=0x7f7d38009c00 comp rx=0 tx=0).stop 2023-08-25T03:35:18.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.734+0000 7f7d4e784700 1 -- 172.21.15.2:0/1399801223 shutdown_connections 2023-08-25T03:35:18.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.734+0000 7f7d4e784700 1 --2- 172.21.15.2:0/1399801223 >> v2:172.21.15.31:3300/0 conn(0x7f7d480ce000 0x7f7d4806be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.734+0000 7f7d4e784700 1 --2- 172.21.15.2:0/1399801223 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7d480cdb70 0x7f7d4806b8c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.734+0000 7f7d4e784700 1 -- 172.21.15.2:0/1399801223 >> 172.21.15.2:0/1399801223 conn(0x7f7d480cfea0 msgr2=0x7f7d480d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:18.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.735+0000 7f7d4e784700 1 -- 172.21.15.2:0/1399801223 shutdown_connections 2023-08-25T03:35:18.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.735+0000 7f7d4e784700 1 -- 172.21.15.2:0/1399801223 wait complete. 2023-08-25T03:35:18.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.736+0000 7f7d4e784700 1 Processor -- start 2023-08-25T03:35:18.745 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.736+0000 7f7d4e784700 1 -- start start 2023-08-25T03:35:18.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.737+0000 7f7d4e784700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7d480b1fb0 0x7f7d481f2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:18.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.737+0000 7f7d4e784700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d480cdb70 0x7f7d480c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:18.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.737+0000 7f7d4e784700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d480ce000 0x7f7d480c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:18.746 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.737+0000 7f7d4e784700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d480c81f0 con 0x7f7d480cdb70 2023-08-25T03:35:18.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.737+0000 7f7d4e784700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d480cc3b0 con 0x7f7d480ce000 2023-08-25T03:35:18.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.737+0000 7f7d4e784700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d480c75e0 con 0x7f7d480b1fb0 2023-08-25T03:35:18.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.737+0000 7f7d477fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d480cdb70 0x7f7d480c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:18.747 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.737+0000 7f7d477fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d480cdb70 0x7f7d480c6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33422/0 (socket says 172.21.15.2:33422) 2023-08-25T03:35:18.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.737+0000 7f7d477fe700 1 -- 172.21.15.2:0/3393635395 learned_addr learned my addr 172.21.15.2:0/3393635395 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:18.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.737+0000 7f7d4cd21700 1 --2- 172.21.15.2:0/3393635395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d480ce000 0x7f7d480c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:18.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.737+0000 7f7d47fff700 1 --2- 172.21.15.2:0/3393635395 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7d480b1fb0 0x7f7d481f2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:18.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.737+0000 7f7d477fe700 1 -- 172.21.15.2:0/3393635395 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7d480b1fb0 msgr2=0x7f7d481f2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:18.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.737+0000 7f7d477fe700 1 --2- 172.21.15.2:0/3393635395 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7d480b1fb0 0x7f7d481f2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.737+0000 7f7d477fe700 1 -- 172.21.15.2:0/3393635395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d480ce000 msgr2=0x7f7d480c6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:18.748 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.737+0000 7f7d477fe700 1 --2- 172.21.15.2:0/3393635395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d480ce000 0x7f7d480c6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.737+0000 7f7d477fe700 1 -- 172.21.15.2:0/3393635395 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d38009550 con 0x7f7d480cdb70 2023-08-25T03:35:18.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.737+0000 7f7d4cd21700 1 --2- 172.21.15.2:0/3393635395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d480ce000 0x7f7d480c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:18.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.737+0000 7f7d477fe700 1 --2- 172.21.15.2:0/3393635395 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d480cdb70 0x7f7d480c6ee0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009c40 tx=0x7f7d38004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:18.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.737+0000 7f7d44ff9700 1 -- 172.21.15.2:0/3393635395 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d380036a0 con 0x7f7d480cdb70 2023-08-25T03:35:18.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.737+0000 7f7d44ff9700 1 -- 172.21.15.2:0/3393635395 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3801e070 con 0x7f7d480cdb70 2023-08-25T03:35:18.749 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.737+0000 7f7d44ff9700 1 -- 172.21.15.2:0/3393635395 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d38003a60 con 0x7f7d480cdb70 2023-08-25T03:35:18.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.737+0000 7f7d4e784700 1 -- 172.21.15.2:0/3393635395 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d480c78c0 con 0x7f7d480cdb70 2023-08-25T03:35:18.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.738+0000 7f7d4e784700 1 -- 172.21.15.2:0/3393635395 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d480c7e80 con 0x7f7d480cdb70 2023-08-25T03:35:18.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.738+0000 7f7d4e784700 1 -- 172.21.15.2:0/3393635395 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d48066f70 con 0x7f7d480cdb70 2023-08-25T03:35:18.750 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.739+0000 7f7d44ff9700 1 -- 172.21.15.2:0/3393635395 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7d38003bc0 con 0x7f7d480cdb70 2023-08-25T03:35:18.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.739+0000 7f7d44ff9700 1 --2- 172.21.15.2:0/3393635395 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f7d3406a110 0x7f7d3406c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:18.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.739+0000 7f7d44ff9700 1 -- 172.21.15.2:0/3393635395 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7d38089400 con 0x7f7d480cdb70 2023-08-25T03:35:18.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.739+0000 7f7d47fff700 1 --2- 172.21.15.2:0/3393635395 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f7d3406a110 0x7f7d3406c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:18.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.740+0000 7f7d47fff700 1 --2- 172.21.15.2:0/3393635395 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f7d3406a110 0x7f7d3406c5d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7d480c9220 tx=0x7f7d3c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:18.751 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.744+0000 7f7d44ff9700 1 -- 172.21.15.2:0/3393635395 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7d38054b80 con 0x7f7d480cdb70 2023-08-25T03:35:18.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.960+0000 7f7d4e784700 1 -- 172.21.15.2:0/3393635395 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7d480c83a0 con 0x7f7d480cdb70 2023-08-25T03:35:18.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.960+0000 7f7d44ff9700 1 -- 172.21.15.2:0/3393635395 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7d480c83a0 con 0x7f7d480cdb70 2023-08-25T03:35:18.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.962+0000 7f7d4e784700 1 -- 172.21.15.2:0/3393635395 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f7d3406a110 msgr2=0x7f7d3406c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:18.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.962+0000 7f7d4e784700 1 --2- 172.21.15.2:0/3393635395 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f7d3406a110 0x7f7d3406c5d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7d480c9220 tx=0x7f7d3c009040 comp rx=0 tx=0).stop 2023-08-25T03:35:18.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.962+0000 7f7d4e784700 1 -- 172.21.15.2:0/3393635395 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d480cdb70 msgr2=0x7f7d480c6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:18.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.962+0000 7f7d4e784700 1 --2- 172.21.15.2:0/3393635395 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d480cdb70 0x7f7d480c6ee0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7d38009c40 tx=0x7f7d38004750 comp rx=0 tx=0).stop 2023-08-25T03:35:18.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.962+0000 7f7d4e784700 1 -- 172.21.15.2:0/3393635395 shutdown_connections 2023-08-25T03:35:18.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.962+0000 7f7d4e784700 1 --2- 172.21.15.2:0/3393635395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d480ce000 0x7f7d480c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.962+0000 7f7d4e784700 1 --2- 172.21.15.2:0/3393635395 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7d480b1fb0 0x7f7d481f2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.962+0000 7f7d4e784700 1 --2- 172.21.15.2:0/3393635395 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f7d3406a110 0x7f7d3406c5d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.962+0000 7f7d4e784700 1 --2- 172.21.15.2:0/3393635395 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d480cdb70 0x7f7d480c6ee0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:18.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.962+0000 7f7d4e784700 1 -- 172.21.15.2:0/3393635395 >> 172.21.15.2:0/3393635395 conn(0x7f7d480cfea0 msgr2=0x7f7d480d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:18.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.962+0000 7f7d4e784700 1 -- 172.21.15.2:0/3393635395 shutdown_connections 2023-08-25T03:35:18.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:18.962+0000 7f7d4e784700 1 -- 172.21.15.2:0/3393635395 wait complete. 2023-08-25T03:35:18.966 INFO:teuthology.orchestra.run.smithi002.stderr:listed 0 entries 2023-08-25T03:35:18.977 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-08-25T03:35:18.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:18.978 DEBUG:teuthology.orchestra.run.smithi002:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-08-25T03:35:19.018 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi002.front.sepia.ceph.com 2023-08-25T03:35:19.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:19.019 DEBUG:teuthology.orchestra.run.smithi002:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-08-25T03:35:19.077 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:35:19.078 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:35:19.101 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:35:19.101 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:35:19.163 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:35:19.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.287+0000 7ff26437c700 1 Processor -- start 2023-08-25T03:35:19.289 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.287+0000 7ff26437c700 1 -- start start 2023-08-25T03:35:19.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.288+0000 7ff26437c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff25c15b060 0x7ff25c15b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:19.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.288+0000 7ff26437c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff25c0cb5a0 0x7ff25c0ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:19.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.288+0000 7ff26437c700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff25c0cd480 0x7ff25c06dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:19.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.288+0000 7ff26437c700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff25c0b6b30 con 0x7ff25c15b060 2023-08-25T03:35:19.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.288+0000 7ff26437c700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff25c0b36a0 con 0x7ff25c0cb5a0 2023-08-25T03:35:19.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.288+0000 7ff26437c700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c15b980 con 0x7ff25c0cd480 2023-08-25T03:35:19.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.288+0000 7ff262118700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff25c15b060 0x7ff25c15b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:19.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.288+0000 7ff262118700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff25c15b060 0x7ff25c15b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33428/0 (socket says 172.21.15.2:33428) 2023-08-25T03:35:19.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.288+0000 7ff262118700 1 -- 172.21.15.2:0/2361191561 learned_addr learned my addr 172.21.15.2:0/2361191561 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:19.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.288+0000 7ff262919700 1 --2- 172.21.15.2:0/2361191561 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff25c0cd480 0x7ff25c06dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:19.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.288+0000 7ff261917700 1 --2- 172.21.15.2:0/2361191561 >> v2:172.21.15.31:3300/0 conn(0x7ff25c0cb5a0 0x7ff25c0ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:19.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.289+0000 7ff262118700 1 -- 172.21.15.2:0/2361191561 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff25c0cd480 msgr2=0x7ff25c06dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:19.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.289+0000 7ff262118700 1 --2- 172.21.15.2:0/2361191561 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff25c0cd480 0x7ff25c06dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:19.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.289+0000 7ff262118700 1 -- 172.21.15.2:0/2361191561 >> v2:172.21.15.31:3300/0 conn(0x7ff25c0cb5a0 msgr2=0x7ff25c0ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:19.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.289+0000 7ff262118700 1 --2- 172.21.15.2:0/2361191561 >> v2:172.21.15.31:3300/0 conn(0x7ff25c0cb5a0 0x7ff25c0ccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:19.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.289+0000 7ff262118700 1 -- 172.21.15.2:0/2361191561 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25c0b2a90 con 0x7ff25c15b060 2023-08-25T03:35:19.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.289+0000 7ff262118700 1 --2- 172.21.15.2:0/2361191561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff25c15b060 0x7ff25c15b440 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0d1b10 tx=0x7ff24c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14ecf7a14f730ec server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:19.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.289+0000 7ff261116700 1 -- 172.21.15.2:0/2361191561 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c00b870 con 0x7ff25c15b060 2023-08-25T03:35:19.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.289+0000 7ff261116700 1 -- 172.21.15.2:0/2361191561 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff24c00b9d0 con 0x7ff25c15b060 2023-08-25T03:35:19.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.289+0000 7ff261116700 1 -- 172.21.15.2:0/2361191561 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24c00bd60 con 0x7ff25c15b060 2023-08-25T03:35:19.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.289+0000 7ff26437c700 1 -- 172.21.15.2:0/2361191561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff25c15b060 msgr2=0x7ff25c15b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:19.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.289+0000 7ff26437c700 1 --2- 172.21.15.2:0/2361191561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff25c15b060 0x7ff25c15b440 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0d1b10 tx=0x7ff24c009bd0 comp rx=0 tx=0).stop 2023-08-25T03:35:19.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.289+0000 7ff26437c700 1 -- 172.21.15.2:0/2361191561 shutdown_connections 2023-08-25T03:35:19.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.289+0000 7ff26437c700 1 --2- 172.21.15.2:0/2361191561 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff25c0cd480 0x7ff25c06dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:19.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.289+0000 7ff26437c700 1 --2- 172.21.15.2:0/2361191561 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff25c15b060 0x7ff25c15b440 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:19.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.289+0000 7ff26437c700 1 --2- 172.21.15.2:0/2361191561 >> v2:172.21.15.31:3300/0 conn(0x7ff25c0cb5a0 0x7ff25c0ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:19.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.289+0000 7ff26437c700 1 -- 172.21.15.2:0/2361191561 >> 172.21.15.2:0/2361191561 conn(0x7ff25c0d0f70 msgr2=0x7ff25c0d1350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:19.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.290+0000 7ff26437c700 1 -- 172.21.15.2:0/2361191561 shutdown_connections 2023-08-25T03:35:19.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.290+0000 7ff26437c700 1 -- 172.21.15.2:0/2361191561 wait complete. 2023-08-25T03:35:19.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.291+0000 7ff26437c700 1 Processor -- start 2023-08-25T03:35:19.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.291+0000 7ff26437c700 1 -- start start 2023-08-25T03:35:19.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.291+0000 7ff26437c700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff25c0cb5a0 0x7ff25c0c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:19.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.291+0000 7ff26437c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff25c0cd480 0x7ff25c0aff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:19.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.291+0000 7ff26437c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff25c15b060 0x7ff25c158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:19.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.291+0000 7ff26437c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c158570 con 0x7ff25c15b060 2023-08-25T03:35:19.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.291+0000 7ff26437c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c06ce30 con 0x7ff25c0cd480 2023-08-25T03:35:19.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.291+0000 7ff26437c700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c06d0d0 con 0x7ff25c0cb5a0 2023-08-25T03:35:19.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.292+0000 7ff262919700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff25c15b060 0x7ff25c158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:19.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.292+0000 7ff262919700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff25c15b060 0x7ff25c158030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33438/0 (socket says 172.21.15.2:33438) 2023-08-25T03:35:19.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.292+0000 7ff262919700 1 -- 172.21.15.2:0/1412067636 learned_addr learned my addr 172.21.15.2:0/1412067636 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:19.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.292+0000 7ff261917700 1 --2- 172.21.15.2:0/1412067636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff25c0cd480 0x7ff25c0aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:19.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.292+0000 7ff262118700 1 --2- 172.21.15.2:0/1412067636 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff25c0cb5a0 0x7ff25c0c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:19.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.292+0000 7ff262919700 1 -- 172.21.15.2:0/1412067636 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff25c0cb5a0 msgr2=0x7ff25c0c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:19.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.292+0000 7ff262919700 1 --2- 172.21.15.2:0/1412067636 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff25c0cb5a0 0x7ff25c0c4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:19.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.292+0000 7ff262919700 1 -- 172.21.15.2:0/1412067636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff25c0cd480 msgr2=0x7ff25c0aff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:19.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.292+0000 7ff262919700 1 --2- 172.21.15.2:0/1412067636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff25c0cd480 0x7ff25c0aff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:19.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.292+0000 7ff262919700 1 -- 172.21.15.2:0/1412067636 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24c009550 con 0x7ff25c15b060 2023-08-25T03:35:19.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.292+0000 7ff262118700 1 --2- 172.21.15.2:0/1412067636 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff25c0cb5a0 0x7ff25c0c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:19.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.292+0000 7ff262919700 1 --2- 172.21.15.2:0/1412067636 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff25c15b060 0x7ff25c158030 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff254007e40 tx=0x7ff25400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:19.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.292+0000 7ff252ffd700 1 -- 172.21.15.2:0/1412067636 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25400eea0 con 0x7ff25c15b060 2023-08-25T03:35:19.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.292+0000 7ff252ffd700 1 -- 172.21.15.2:0/1412067636 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff254014420 con 0x7ff25c15b060 2023-08-25T03:35:19.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.292+0000 7ff26437c700 1 -- 172.21.15.2:0/1412067636 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff25c15cd70 con 0x7ff25c15b060 2023-08-25T03:35:19.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.292+0000 7ff252ffd700 1 -- 172.21.15.2:0/1412067636 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2540147b0 con 0x7ff25c15b060 2023-08-25T03:35:19.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.292+0000 7ff26437c700 1 -- 172.21.15.2:0/1412067636 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff25c158a10 con 0x7ff25c15b060 2023-08-25T03:35:19.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.293+0000 7ff26437c700 1 -- 172.21.15.2:0/1412067636 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff25c15cbd0 con 0x7ff25c15b060 2023-08-25T03:35:19.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.294+0000 7ff252ffd700 1 -- 172.21.15.2:0/1412067636 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff254014910 con 0x7ff25c15b060 2023-08-25T03:35:19.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.294+0000 7ff252ffd700 1 --2- 172.21.15.2:0/1412067636 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff24006a0e0 0x7ff24006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:19.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.294+0000 7ff252ffd700 1 -- 172.21.15.2:0/1412067636 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7ff254088d30 con 0x7ff25c15b060 2023-08-25T03:35:19.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.294+0000 7ff262118700 1 --2- 172.21.15.2:0/1412067636 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff24006a0e0 0x7ff24006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:19.302 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.295+0000 7ff262118700 1 --2- 172.21.15.2:0/1412067636 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff24006a0e0 0x7ff24006c5a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff24c014040 tx=0x7ff24c008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:19.303 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.299+0000 7ff252ffd700 1 -- 172.21.15.2:0/1412067636 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff25c15cbd0 con 0x7ff25c15b060 2023-08-25T03:35:19.502 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.500+0000 7ff26437c700 1 -- 172.21.15.2:0/1412067636 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff25c15d290 con 0x7ff25c15b060 2023-08-25T03:35:19.503 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.501+0000 7ff252ffd700 1 -- 172.21.15.2:0/1412067636 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18207 (secure 0 0 0) 0x7ff25c15d290 con 0x7ff25c15b060 2023-08-25T03:35:19.503 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:35:19.504 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":37,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:35:08.407381+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:35:05.870790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-08-25T03:35:06.296959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:35:19.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.503+0000 7ff26437c700 1 -- 172.21.15.2:0/1412067636 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff24006a0e0 msgr2=0x7ff24006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:19.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.503+0000 7ff26437c700 1 --2- 172.21.15.2:0/1412067636 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff24006a0e0 0x7ff24006c5a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff24c014040 tx=0x7ff24c008030 comp rx=0 tx=0).stop 2023-08-25T03:35:19.506 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.504+0000 7ff26437c700 1 -- 172.21.15.2:0/1412067636 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff25c15b060 msgr2=0x7ff25c158030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:19.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.504+0000 7ff26437c700 1 --2- 172.21.15.2:0/1412067636 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff25c15b060 0x7ff25c158030 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ff254007e40 tx=0x7ff25400e3c0 comp rx=0 tx=0).stop 2023-08-25T03:35:19.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.504+0000 7ff26437c700 1 -- 172.21.15.2:0/1412067636 shutdown_connections 2023-08-25T03:35:19.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.504+0000 7ff26437c700 1 --2- 172.21.15.2:0/1412067636 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff24006a0e0 0x7ff24006c5a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:19.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.504+0000 7ff26437c700 1 --2- 172.21.15.2:0/1412067636 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff25c15b060 0x7ff25c158030 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:19.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.504+0000 7ff26437c700 1 --2- 172.21.15.2:0/1412067636 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff25c0cb5a0 0x7ff25c0c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:19.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.504+0000 7ff26437c700 1 --2- 172.21.15.2:0/1412067636 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff25c0cd480 0x7ff25c0aff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:19.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.504+0000 7ff26437c700 1 -- 172.21.15.2:0/1412067636 >> 172.21.15.2:0/1412067636 conn(0x7ff25c0d0f70 msgr2=0x7ff25c1591f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:19.507 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.504+0000 7ff26437c700 1 -- 172.21.15.2:0/1412067636 shutdown_connections 2023-08-25T03:35:19.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.504+0000 7ff26437c700 1 -- 172.21.15.2:0/1412067636 wait complete. 2023-08-25T03:35:19.523 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T03:33:49.560643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T03:33:56.265184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T03:35:05.870790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T03:35:06.296959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T03:35:19.524 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T03:35:19.653 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.651+0000 7f5d0e48f700 1 Processor -- start 2023-08-25T03:35:19.654 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.652+0000 7f5d0e48f700 1 -- start start 2023-08-25T03:35:19.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.652+0000 7f5d0e48f700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5d081607f0 0x7f5d08070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:19.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.652+0000 7f5d0e48f700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5d080af180 0x7f5d080ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:19.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.652+0000 7f5d0e48f700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5d0806a1e0 0x7f5d0806a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:19.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.652+0000 7f5d0e48f700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d0806abd0 con 0x7f5d080af180 2023-08-25T03:35:19.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.652+0000 7f5d0e48f700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d080713a0 con 0x7f5d081607f0 2023-08-25T03:35:19.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.652+0000 7f5d0e48f700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d080718a0 con 0x7f5d0806a1e0 2023-08-25T03:35:19.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.653+0000 7f5d077fe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5d080af180 0x7f5d080ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:19.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.653+0000 7f5d077fe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5d080af180 0x7f5d080ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33442/0 (socket says 172.21.15.2:33442) 2023-08-25T03:35:19.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.653+0000 7f5d077fe700 1 -- 172.21.15.2:0/3112832916 learned_addr learned my addr 172.21.15.2:0/3112832916 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:19.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.653+0000 7f5d0ca2c700 1 --2- 172.21.15.2:0/3112832916 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5d0806a1e0 0x7f5d0806a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:19.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.653+0000 7f5d07fff700 1 --2- 172.21.15.2:0/3112832916 >> v2:172.21.15.31:3300/0 conn(0x7f5d081607f0 0x7f5d08070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:19.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.653+0000 7f5d077fe700 1 -- 172.21.15.2:0/3112832916 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5d0806a1e0 msgr2=0x7f5d0806a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:19.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.653+0000 7f5d077fe700 1 --2- 172.21.15.2:0/3112832916 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5d0806a1e0 0x7f5d0806a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:19.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.653+0000 7f5d077fe700 1 -- 172.21.15.2:0/3112832916 >> v2:172.21.15.31:3300/0 conn(0x7f5d081607f0 msgr2=0x7f5d08070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:19.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.653+0000 7f5d077fe700 1 --2- 172.21.15.2:0/3112832916 >> v2:172.21.15.31:3300/0 conn(0x7f5d081607f0 0x7f5d08070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:19.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.653+0000 7f5d077fe700 1 -- 172.21.15.2:0/3112832916 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d08160bd0 con 0x7f5d080af180 2023-08-25T03:35:19.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.653+0000 7f5d07fff700 1 --2- 172.21.15.2:0/3112832916 >> v2:172.21.15.31:3300/0 conn(0x7f5d081607f0 0x7f5d08070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:35:19.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.653+0000 7f5d0ca2c700 1 --2- 172.21.15.2:0/3112832916 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5d0806a1e0 0x7f5d0806a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:35:19.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.653+0000 7f5d077fe700 1 --2- 172.21.15.2:0/3112832916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d080af180 0x7f5d080ac8e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5d0806df30 tx=0x7f5cf0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560da11ed964ced4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:19.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.653+0000 7f5d06ffd700 1 -- 172.21.15.2:0/3112832916 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf000b870 con 0x7f5d080af180 2023-08-25T03:35:19.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.653+0000 7f5d06ffd700 1 -- 172.21.15.2:0/3112832916 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf000b9d0 con 0x7f5d080af180 2023-08-25T03:35:19.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.653+0000 7f5d06ffd700 1 -- 172.21.15.2:0/3112832916 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf000bd60 con 0x7f5d080af180 2023-08-25T03:35:19.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.654+0000 7f5d0e48f700 1 -- 172.21.15.2:0/3112832916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d080af180 msgr2=0x7f5d080ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:19.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.654+0000 7f5d0e48f700 1 --2- 172.21.15.2:0/3112832916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d080af180 0x7f5d080ac8e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5d0806df30 tx=0x7f5cf0009bd0 comp rx=0 tx=0).stop 2023-08-25T03:35:19.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.654+0000 7f5d0e48f700 1 -- 172.21.15.2:0/3112832916 shutdown_connections 2023-08-25T03:35:19.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.654+0000 7f5d0e48f700 1 --2- 172.21.15.2:0/3112832916 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5d0806a1e0 0x7f5d0806a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:19.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.654+0000 7f5d0e48f700 1 --2- 172.21.15.2:0/3112832916 >> v2:172.21.15.31:3300/0 conn(0x7f5d081607f0 0x7f5d08070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:19.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.654+0000 7f5d0e48f700 1 --2- 172.21.15.2:0/3112832916 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d080af180 0x7f5d080ac8e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:19.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.654+0000 7f5d0e48f700 1 -- 172.21.15.2:0/3112832916 >> 172.21.15.2:0/3112832916 conn(0x7f5d080cfd90 msgr2=0x7f5d080d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:19.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.654+0000 7f5d0e48f700 1 -- 172.21.15.2:0/3112832916 shutdown_connections 2023-08-25T03:35:19.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.654+0000 7f5d0e48f700 1 -- 172.21.15.2:0/3112832916 wait complete. 2023-08-25T03:35:19.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.655+0000 7f5d0e48f700 1 Processor -- start 2023-08-25T03:35:19.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.655+0000 7f5d0e48f700 1 -- start start 2023-08-25T03:35:19.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.656+0000 7f5d0e48f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d0806a1e0 0x7f5d0814e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:19.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.656+0000 7f5d0e48f700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5d080af180 0x7f5d080c4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:19.671 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.656+0000 7f5d0e48f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d081607f0 0x7f5d0806ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:19.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.656+0000 7f5d0e48f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d0814e8e0 con 0x7f5d081607f0 2023-08-25T03:35:19.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.656+0000 7f5d0e48f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d080c6cc0 con 0x7f5d0806a1e0 2023-08-25T03:35:19.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.656+0000 7f5d0e48f700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d080c3890 con 0x7f5d080af180 2023-08-25T03:35:19.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.656+0000 7f5d0ca2c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d081607f0 0x7f5d0806ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:19.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.656+0000 7f5d0ca2c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d081607f0 0x7f5d0806ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33452/0 (socket says 172.21.15.2:33452) 2023-08-25T03:35:19.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.656+0000 7f5d0ca2c700 1 -- 172.21.15.2:0/213470547 learned_addr learned my addr 172.21.15.2:0/213470547 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:19.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.656+0000 7f5d077fe700 1 --2- 172.21.15.2:0/213470547 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5d080af180 0x7f5d080c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:19.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.656+0000 7f5d07fff700 1 --2- 172.21.15.2:0/213470547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d0806a1e0 0x7f5d0814e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:19.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.657+0000 7f5d0ca2c700 1 -- 172.21.15.2:0/213470547 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5d080af180 msgr2=0x7f5d080c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:19.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.657+0000 7f5d0ca2c700 1 --2- 172.21.15.2:0/213470547 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5d080af180 0x7f5d080c4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:19.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.657+0000 7f5d0ca2c700 1 -- 172.21.15.2:0/213470547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d0806a1e0 msgr2=0x7f5d0814e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:19.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.657+0000 7f5d0ca2c700 1 --2- 172.21.15.2:0/213470547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d0806a1e0 0x7f5d0814e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:19.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.657+0000 7f5d0ca2c700 1 -- 172.21.15.2:0/213470547 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf0009550 con 0x7f5d081607f0 2023-08-25T03:35:19.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.657+0000 7f5d0ca2c700 1 --2- 172.21.15.2:0/213470547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d081607f0 0x7f5d0806ebb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000cd10 tx=0x7f5d0000a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:19.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.657+0000 7f5d04ff9700 1 -- 172.21.15.2:0/213470547 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0000ce60 con 0x7f5d081607f0 2023-08-25T03:35:19.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.657+0000 7f5d04ff9700 1 -- 172.21.15.2:0/213470547 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0000f480 con 0x7f5d081607f0 2023-08-25T03:35:19.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.657+0000 7f5d0e48f700 1 -- 172.21.15.2:0/213470547 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d080c69e0 con 0x7f5d081607f0 2023-08-25T03:35:19.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.657+0000 7f5d04ff9700 1 -- 172.21.15.2:0/213470547 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0000f810 con 0x7f5d081607f0 2023-08-25T03:35:19.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.657+0000 7f5d0e48f700 1 -- 172.21.15.2:0/213470547 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d08151b60 con 0x7f5d081607f0 2023-08-25T03:35:19.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.658+0000 7f5d0e48f700 1 -- 172.21.15.2:0/213470547 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d080c36f0 con 0x7f5d081607f0 2023-08-25T03:35:19.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.658+0000 7f5d04ff9700 1 -- 172.21.15.2:0/213470547 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5d0000b040 con 0x7f5d081607f0 2023-08-25T03:35:19.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.659+0000 7f5d04ff9700 1 --2- 172.21.15.2:0/213470547 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5ce806a0e0 0x7f5ce806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:19.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.659+0000 7f5d04ff9700 1 -- 172.21.15.2:0/213470547 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5d00088bb0 con 0x7f5d081607f0 2023-08-25T03:35:19.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.659+0000 7f5d07fff700 1 --2- 172.21.15.2:0/213470547 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5ce806a0e0 0x7f5ce806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:19.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.659+0000 7f5d07fff700 1 --2- 172.21.15.2:0/213470547 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5ce806a0e0 0x7f5ce806c5a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8006fd0 tx=0x7f5cf8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:19.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.663+0000 7f5d04ff9700 1 -- 172.21.15.2:0/213470547 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5d080c36f0 con 0x7f5d081607f0 2023-08-25T03:35:19.864 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.863+0000 7f5d0e48f700 1 -- 172.21.15.2:0/213470547 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5d080c36f0 con 0x7f5d081607f0 2023-08-25T03:35:19.865 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.863+0000 7f5d04ff9700 1 -- 172.21.15.2:0/213470547 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5d080c36f0 con 0x7f5d081607f0 2023-08-25T03:35:19.865 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-08-25T03:35:19.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.865+0000 7f5d0e48f700 1 -- 172.21.15.2:0/213470547 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5ce806a0e0 msgr2=0x7f5ce806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:19.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.865+0000 7f5d0e48f700 1 --2- 172.21.15.2:0/213470547 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5ce806a0e0 0x7f5ce806c5a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8006fd0 tx=0x7f5cf8008040 comp rx=0 tx=0).stop 2023-08-25T03:35:19.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.865+0000 7f5d0e48f700 1 -- 172.21.15.2:0/213470547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d081607f0 msgr2=0x7f5d0806ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:19.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.865+0000 7f5d0e48f700 1 --2- 172.21.15.2:0/213470547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d081607f0 0x7f5d0806ebb0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f5d0000cd10 tx=0x7f5d0000a560 comp rx=0 tx=0).stop 2023-08-25T03:35:19.867 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.865+0000 7f5d0e48f700 1 -- 172.21.15.2:0/213470547 shutdown_connections 2023-08-25T03:35:19.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.865+0000 7f5d0e48f700 1 --2- 172.21.15.2:0/213470547 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5ce806a0e0 0x7f5ce806c5a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:19.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.865+0000 7f5d0e48f700 1 --2- 172.21.15.2:0/213470547 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5d081607f0 0x7f5d0806ebb0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:19.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.865+0000 7f5d0e48f700 1 --2- 172.21.15.2:0/213470547 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d0806a1e0 0x7f5d0814e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:19.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.865+0000 7f5d0e48f700 1 --2- 172.21.15.2:0/213470547 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5d080af180 0x7f5d080c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:19.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.865+0000 7f5d0e48f700 1 -- 172.21.15.2:0/213470547 >> 172.21.15.2:0/213470547 conn(0x7f5d080cfd90 msgr2=0x7f5d080af830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:19.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.865+0000 7f5d0e48f700 1 -- 172.21.15.2:0/213470547 shutdown_connections 2023-08-25T03:35:19.868 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:19.865+0000 7f5d0e48f700 1 -- 172.21.15.2:0/213470547 wait complete. 2023-08-25T03:35:19.880 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T03:35:20.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.005+0000 7f9d92087700 1 Processor -- start 2023-08-25T03:35:20.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.006+0000 7f9d92087700 1 -- start start 2023-08-25T03:35:20.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.006+0000 7f9d92087700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9d8c0cfb80 0x7f9d8c0cd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:20.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.006+0000 7f9d92087700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f9d8c0cdd30 0x7f9d8c06d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:20.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.006+0000 7f9d92087700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d8c0d0000 con 0x7f9d8c15b170 2023-08-25T03:35:20.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.006+0000 7f9d92087700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d8c0ce1c0 con 0x7f9d8c0cdd30 2023-08-25T03:35:20.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.006+0000 7f9d92087700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d8c0cf6f0 con 0x7f9d8c0cfb80 2023-08-25T03:35:20.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.007+0000 7f9d8bfff700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f9d8c0cdd30 0x7f9d8c06d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:20.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.007+0000 7f9d8bfff700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f9d8c0cdd30 0x7f9d8c06d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33460/0 (socket says 172.21.15.2:33460) 2023-08-25T03:35:20.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.007+0000 7f9d8bfff700 1 -- 172.21.15.2:0/2808527091 learned_addr learned my addr 172.21.15.2:0/2808527091 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:20.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.007+0000 7f9d8affd700 1 --2- 172.21.15.2:0/2808527091 >> v2:172.21.15.31:3300/0 conn(0x7f9d8c0cfb80 0x7f9d8c0cd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:20.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.007+0000 7f9d8bfff700 1 -- 172.21.15.2:0/2808527091 >> v2:172.21.15.31:3300/0 conn(0x7f9d8c0cfb80 msgr2=0x7f9d8c0cd7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:20.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.007+0000 7f9d8bfff700 1 --2- 172.21.15.2:0/2808527091 >> v2:172.21.15.31:3300/0 conn(0x7f9d8c0cfb80 0x7f9d8c0cd7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.007+0000 7f9d8bfff700 1 -- 172.21.15.2:0/2808527091 >> v1:172.21.15.2:6789/0 conn(0x7f9d8c15b170 legacy=0x7f9d8c0cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:20.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.007+0000 7f9d8bfff700 1 -- 172.21.15.2:0/2808527091 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8c0d1d50 con 0x7f9d8c0cdd30 2023-08-25T03:35:20.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.007+0000 7f9d8bfff700 1 --2- 172.21.15.2:0/2808527091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d8c0cdd30 0x7f9d8c06d9f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0c7ef0 tx=0x7f9d8400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25215ad0cd1d9b27 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:20.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.007+0000 7f9d8a7fc700 1 -- 172.21.15.2:0/2808527091 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8400e070 con 0x7f9d8c0cdd30 2023-08-25T03:35:20.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.007+0000 7f9d8a7fc700 1 -- 172.21.15.2:0/2808527091 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8400b9c0 con 0x7f9d8c0cdd30 2023-08-25T03:35:20.012 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.007+0000 7f9d8a7fc700 1 -- 172.21.15.2:0/2808527091 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8400bd70 con 0x7f9d8c0cdd30 2023-08-25T03:35:20.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.008+0000 7f9d92087700 1 -- 172.21.15.2:0/2808527091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d8c0cdd30 msgr2=0x7f9d8c06d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:20.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.008+0000 7f9d92087700 1 --2- 172.21.15.2:0/2808527091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d8c0cdd30 0x7f9d8c06d9f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0c7ef0 tx=0x7f9d8400b330 comp rx=0 tx=0).stop 2023-08-25T03:35:20.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.008+0000 7f9d92087700 1 -- 172.21.15.2:0/2808527091 shutdown_connections 2023-08-25T03:35:20.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.008+0000 7f9d92087700 1 --2- 172.21.15.2:0/2808527091 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d8c0cdd30 0x7f9d8c06d9f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.008+0000 7f9d92087700 1 --2- 172.21.15.2:0/2808527091 >> v2:172.21.15.31:3300/0 conn(0x7f9d8c0cfb80 0x7f9d8c0cd7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.008+0000 7f9d92087700 1 -- 172.21.15.2:0/2808527091 >> 172.21.15.2:0/2808527091 conn(0x7f9d8c09a200 msgr2=0x7f9d8c0d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:20.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.008+0000 7f9d92087700 1 -- 172.21.15.2:0/2808527091 shutdown_connections 2023-08-25T03:35:20.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.008+0000 7f9d92087700 1 -- 172.21.15.2:0/2808527091 wait complete. 2023-08-25T03:35:20.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.009+0000 7f9d92087700 1 Processor -- start 2023-08-25T03:35:20.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.009+0000 7f9d92087700 1 -- start start 2023-08-25T03:35:20.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.010+0000 7f9d92087700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d8c0cdd30 0x7f9d8c157690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:20.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.010+0000 7f9d92087700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d8c0cfb80 0x7f9d8c0b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:20.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.010+0000 7f9d92087700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9d8c15b170 0x7f9d8c15c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:20.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.010+0000 7f9d92087700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c0c83e0 con 0x7f9d8c0cdd30 2023-08-25T03:35:20.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.010+0000 7f9d92087700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c15d6c0 con 0x7f9d8c0cfb80 2023-08-25T03:35:20.016 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.010+0000 7f9d92087700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c15d100 con 0x7f9d8c15b170 2023-08-25T03:35:20.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.010+0000 7f9d8b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d8c0cdd30 0x7f9d8c157690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:20.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.010+0000 7f9d8b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d8c0cdd30 0x7f9d8c157690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33462/0 (socket says 172.21.15.2:33462) 2023-08-25T03:35:20.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.010+0000 7f9d8b7fe700 1 -- 172.21.15.2:0/4149971753 learned_addr learned my addr 172.21.15.2:0/4149971753 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:20.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.010+0000 7f9d8affd700 1 --2- 172.21.15.2:0/4149971753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d8c0cfb80 0x7f9d8c0b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:20.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.010+0000 7f9d8bfff700 1 --2- 172.21.15.2:0/4149971753 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9d8c15b170 0x7f9d8c15c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:20.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.010+0000 7f9d8b7fe700 1 -- 172.21.15.2:0/4149971753 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9d8c15b170 msgr2=0x7f9d8c15c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:20.017 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.011+0000 7f9d8b7fe700 1 --2- 172.21.15.2:0/4149971753 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9d8c15b170 0x7f9d8c15c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.011+0000 7f9d8b7fe700 1 -- 172.21.15.2:0/4149971753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d8c0cfb80 msgr2=0x7f9d8c0b4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:20.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.011+0000 7f9d8b7fe700 1 --2- 172.21.15.2:0/4149971753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d8c0cfb80 0x7f9d8c0b4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.011+0000 7f9d8b7fe700 1 -- 172.21.15.2:0/4149971753 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d84009b30 con 0x7f9d8c0cdd30 2023-08-25T03:35:20.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.011+0000 7f9d8affd700 1 --2- 172.21.15.2:0/4149971753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d8c0cfb80 0x7f9d8c0b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:35:20.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.011+0000 7f9d8b7fe700 1 --2- 172.21.15.2:0/4149971753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d8c0cdd30 0x7f9d8c157690 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c00c430 tx=0x7f9d7c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:20.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.011+0000 7f9d83fff700 1 -- 172.21.15.2:0/4149971753 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c0054e0 con 0x7f9d8c0cdd30 2023-08-25T03:35:20.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.011+0000 7f9d83fff700 1 -- 172.21.15.2:0/4149971753 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d7c003680 con 0x7f9d8c0cdd30 2023-08-25T03:35:20.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.011+0000 7f9d83fff700 1 -- 172.21.15.2:0/4149971753 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d7c003a30 con 0x7f9d8c0cdd30 2023-08-25T03:35:20.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.011+0000 7f9d92087700 1 -- 172.21.15.2:0/4149971753 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d8c15d3e0 con 0x7f9d8c0cdd30 2023-08-25T03:35:20.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.011+0000 7f9d92087700 1 -- 172.21.15.2:0/4149971753 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d8c157bd0 con 0x7f9d8c0cdd30 2023-08-25T03:35:20.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.012+0000 7f9d92087700 1 -- 172.21.15.2:0/4149971753 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d8c0746d0 con 0x7f9d8c0cdd30 2023-08-25T03:35:20.019 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.013+0000 7f9d83fff700 1 -- 172.21.15.2:0/4149971753 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9d7c003b90 con 0x7f9d8c0cdd30 2023-08-25T03:35:20.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.013+0000 7f9d83fff700 1 --2- 172.21.15.2:0/4149971753 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9d6c06a0e0 0x7f9d6c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:20.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.013+0000 7f9d83fff700 1 -- 172.21.15.2:0/4149971753 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f9d7c0877a0 con 0x7f9d8c0cdd30 2023-08-25T03:35:20.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.013+0000 7f9d8affd700 1 --2- 172.21.15.2:0/4149971753 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9d6c06a0e0 0x7f9d6c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:20.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.014+0000 7f9d8affd700 1 --2- 172.21.15.2:0/4149971753 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9d6c06a0e0 0x7f9d6c06c5a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c09edb0 tx=0x7f9d74008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:20.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.018+0000 7f9d83fff700 1 -- 172.21.15.2:0/4149971753 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9d7c052f20 con 0x7f9d8c0cdd30 2023-08-25T03:35:20.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.216+0000 7f9d92087700 1 -- 172.21.15.2:0/4149971753 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f9d8c0d0000 con 0x7f9d8c0cdd30 2023-08-25T03:35:20.218 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.217+0000 7f9d83fff700 1 -- 172.21.15.2:0/4149971753 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v37) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9d8c0d0000 con 0x7f9d8c0cdd30 2023-08-25T03:35:20.219 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 8 2023-08-25T03:35:20.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.218+0000 7f9d92087700 1 -- 172.21.15.2:0/4149971753 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9d6c06a0e0 msgr2=0x7f9d6c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:20.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.218+0000 7f9d92087700 1 --2- 172.21.15.2:0/4149971753 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9d6c06a0e0 0x7f9d6c06c5a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c09edb0 tx=0x7f9d74008040 comp rx=0 tx=0).stop 2023-08-25T03:35:20.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.218+0000 7f9d92087700 1 -- 172.21.15.2:0/4149971753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d8c0cdd30 msgr2=0x7f9d8c157690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:20.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.218+0000 7f9d92087700 1 --2- 172.21.15.2:0/4149971753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d8c0cdd30 0x7f9d8c157690 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c00c430 tx=0x7f9d7c00c7f0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.219+0000 7f9d92087700 1 -- 172.21.15.2:0/4149971753 shutdown_connections 2023-08-25T03:35:20.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.219+0000 7f9d92087700 1 --2- 172.21.15.2:0/4149971753 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9d6c06a0e0 0x7f9d6c06c5a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.219+0000 7f9d92087700 1 --2- 172.21.15.2:0/4149971753 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9d8c0cdd30 0x7f9d8c157690 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.219+0000 7f9d92087700 1 --2- 172.21.15.2:0/4149971753 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9d8c15b170 0x7f9d8c15c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.219+0000 7f9d92087700 1 --2- 172.21.15.2:0/4149971753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d8c0cfb80 0x7f9d8c0b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.219+0000 7f9d92087700 1 -- 172.21.15.2:0/4149971753 >> 172.21.15.2:0/4149971753 conn(0x7f9d8c09a200 msgr2=0x7f9d8c0afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:20.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.219+0000 7f9d92087700 1 -- 172.21.15.2:0/4149971753 shutdown_connections 2023-08-25T03:35:20.223 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.219+0000 7f9d92087700 1 -- 172.21.15.2:0/4149971753 wait complete. 2023-08-25T03:35:20.233 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T03:35:20.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.360+0000 7f4eeefc2700 1 Processor -- start 2023-08-25T03:35:20.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.360+0000 7f4eeefc2700 1 -- start start 2023-08-25T03:35:20.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.361+0000 7f4eeefc2700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4ee8159f10 0x7f4ee815dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:20.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.361+0000 7f4eeefc2700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee806eaf0 con 0x7f4ee80cacf0 2023-08-25T03:35:20.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.361+0000 7f4eeefc2700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee815e020 con 0x7f4ee80cedf0 2023-08-25T03:35:20.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.361+0000 7f4eeefc2700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee806e850 con 0x7f4ee8159f10 2023-08-25T03:35:20.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.361+0000 7f4eed55f700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f4ee80cacf0 0x7f4ee806cf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48370/0 (socket says 172.21.15.2:48370) 2023-08-25T03:35:20.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.361+0000 7f4eed55f700 1 -- 172.21.15.2:0/2258353453 learned_addr learned my addr 172.21.15.2:0/2258353453 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:20.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.361+0000 7f4eecd5e700 1 --2- 172.21.15.2:0/2258353453 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4ee8159f10 0x7f4ee815dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:20.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.361+0000 7f4ee77fe700 1 -- 172.21.15.2:0/2258353453 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1481131303 0 0) 0x7f4ee806eaf0 con 0x7f4ee80cacf0 2023-08-25T03:35:20.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.361+0000 7f4ee77fe700 1 -- 172.21.15.2:0/2258353453 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ed80035b0 con 0x7f4ee80cacf0 2023-08-25T03:35:20.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.362+0000 7f4ee77fe700 1 -- 172.21.15.2:0/2258353453 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 928761540 0 0) 0x7f4ed80035b0 con 0x7f4ee80cacf0 2023-08-25T03:35:20.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.362+0000 7f4ee77fe700 1 -- 172.21.15.2:0/2258353453 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4ee8159f10 msgr2=0x7f4ee815dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:20.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.362+0000 7f4ee77fe700 1 --2- 172.21.15.2:0/2258353453 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4ee8159f10 0x7f4ee815dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.362+0000 7f4ee77fe700 1 -- 172.21.15.2:0/2258353453 >> v1:172.21.15.31:6789/0 conn(0x7f4ee80cedf0 legacy=0x7f4ee806ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:20.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.362+0000 7f4ee77fe700 1 -- 172.21.15.2:0/2258353453 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee815e2c0 con 0x7f4ee80cacf0 2023-08-25T03:35:20.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.362+0000 7f4ee77fe700 1 -- 172.21.15.2:0/2258353453 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f4ee0001a40 con 0x7f4ee80cacf0 2023-08-25T03:35:20.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.362+0000 7f4ee77fe700 1 -- 172.21.15.2:0/2258353453 >> v1:172.21.15.2:6789/0 conn(0x7f4ee80cacf0 legacy=0x7f4ee806cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:20.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.362+0000 7f4ee77fe700 1 --2- 172.21.15.2:0/2258353453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ed8003a30 0x7f4ed8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:20.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.362+0000 7f4ee77fe700 1 --2- 172.21.15.2:0/2258353453 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4ee8159f10 0x7f4ed8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:20.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.362+0000 7f4ee77fe700 1 --2- 172.21.15.2:0/2258353453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ed8004d90 0x7f4ed8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:20.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.362+0000 7f4ee77fe700 1 -- 172.21.15.2:0/2258353453 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee806e850 con 0x7f4ed8004d90 2023-08-25T03:35:20.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.362+0000 7f4ee77fe700 1 -- 172.21.15.2:0/2258353453 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed800bac0 con 0x7f4ed8003a30 2023-08-25T03:35:20.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.362+0000 7f4ee77fe700 1 -- 172.21.15.2:0/2258353453 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed800de40 con 0x7f4ee8159f10 2023-08-25T03:35:20.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.362+0000 7f4eed55f700 1 --2- 172.21.15.2:0/2258353453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ed8004d90 0x7f4ed8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:20.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.362+0000 7f4eecd5e700 1 --2- 172.21.15.2:0/2258353453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ed8003a30 0x7f4ed8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:20.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.362+0000 7f4ee7fff700 1 --2- 172.21.15.2:0/2258353453 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4ee8159f10 0x7f4ed8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:20.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.362+0000 7f4eed55f700 1 -- 172.21.15.2:0/2258353453 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4ee8159f10 msgr2=0x7f4ed8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:20.376 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.362+0000 7f4eed55f700 1 --2- 172.21.15.2:0/2258353453 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4ee8159f10 0x7f4ed8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.362+0000 7f4eed55f700 1 -- 172.21.15.2:0/2258353453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ed8003a30 msgr2=0x7f4ed8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:20.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.362+0000 7f4eed55f700 1 --2- 172.21.15.2:0/2258353453 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ed8003a30 0x7f4ed8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.362+0000 7f4eed55f700 1 -- 172.21.15.2:0/2258353453 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed80035b0 con 0x7f4ed8004d90 2023-08-25T03:35:20.377 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.362+0000 7f4eed55f700 1 --2- 172.21.15.2:0/2258353453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ed8004d90 0x7f4ed8005210 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4ee000a7b0 tx=0x7f4ed8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a027e9a5381baa2a server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:20.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.363+0000 7f4ee77fe700 1 -- 172.21.15.2:0/2258353453 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0012600 con 0x7f4ed8004d90 2023-08-25T03:35:20.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.363+0000 7f4ee77fe700 1 -- 172.21.15.2:0/2258353453 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee00127a0 con 0x7f4ed8004d90 2023-08-25T03:35:20.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.363+0000 7f4ee77fe700 1 -- 172.21.15.2:0/2258353453 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0012b50 con 0x7f4ed8004d90 2023-08-25T03:35:20.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.363+0000 7f4eeefc2700 1 -- 172.21.15.2:0/2258353453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ed8004d90 msgr2=0x7f4ed8005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:20.378 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.363+0000 7f4eeefc2700 1 --2- 172.21.15.2:0/2258353453 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ed8004d90 0x7f4ed8005210 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4ee000a7b0 tx=0x7f4ed8005880 comp rx=0 tx=0).stop 2023-08-25T03:35:20.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.363+0000 7f4eed55f700 1 -- 172.21.15.2:0/2258353453 reap_dead start 2023-08-25T03:35:20.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.363+0000 7f4eeefc2700 1 -- 172.21.15.2:0/2258353453 shutdown_connections 2023-08-25T03:35:20.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.363+0000 7f4eeefc2700 1 -- 172.21.15.2:0/2258353453 >> 172.21.15.2:0/2258353453 conn(0x7f4ee80cfd90 msgr2=0x7f4ee80d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:20.379 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.363+0000 7f4eeefc2700 1 -- 172.21.15.2:0/2258353453 shutdown_connections 2023-08-25T03:35:20.380 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.363+0000 7f4eeefc2700 1 -- 172.21.15.2:0/2258353453 wait complete. 2023-08-25T03:35:20.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.364+0000 7f4eeefc2700 1 Processor -- start 2023-08-25T03:35:20.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.364+0000 7f4eeefc2700 1 -- start start 2023-08-25T03:35:20.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.365+0000 7f4eeefc2700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ee8156200 0x7f4ee80b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:20.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.365+0000 7f4eeefc2700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ee815d520 0x7f4ee80b1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:20.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.365+0000 7f4eeefc2700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4ee8071ab0 0x7f4ee806a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:20.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.365+0000 7f4eeefc2700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee81f26d0 con 0x7f4ee815d520 2023-08-25T03:35:20.383 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.365+0000 7f4eeefc2700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee81f1a50 con 0x7f4ee8156200 2023-08-25T03:35:20.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.365+0000 7f4eeefc2700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee81f1d30 con 0x7f4ee8071ab0 2023-08-25T03:35:20.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.365+0000 7f4ee7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ee815d520 0x7f4ee80b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:20.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.365+0000 7f4ee7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ee815d520 0x7f4ee80b1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33482/0 (socket says 172.21.15.2:33482) 2023-08-25T03:35:20.384 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.365+0000 7f4ee7fff700 1 -- 172.21.15.2:0/4189093985 learned_addr learned my addr 172.21.15.2:0/4189093985 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:20.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.366+0000 7f4eed55f700 1 --2- 172.21.15.2:0/4189093985 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4ee8071ab0 0x7f4ee806a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:20.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.366+0000 7f4eecd5e700 1 --2- 172.21.15.2:0/4189093985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ee8156200 0x7f4ee80b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:20.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.366+0000 7f4ee7fff700 1 -- 172.21.15.2:0/4189093985 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4ee8071ab0 msgr2=0x7f4ee806a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:20.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.366+0000 7f4ee7fff700 1 --2- 172.21.15.2:0/4189093985 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4ee8071ab0 0x7f4ee806a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.366+0000 7f4ee7fff700 1 -- 172.21.15.2:0/4189093985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ee8156200 msgr2=0x7f4ee80b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:20.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.366+0000 7f4ee7fff700 1 --2- 172.21.15.2:0/4189093985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ee8156200 0x7f4ee80b0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.385 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.366+0000 7f4ee7fff700 1 -- 172.21.15.2:0/4189093985 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee81f2010 con 0x7f4ee815d520 2023-08-25T03:35:20.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.366+0000 7f4eed55f700 1 --2- 172.21.15.2:0/4189093985 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4ee8071ab0 0x7f4ee806a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:35:20.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.366+0000 7f4eecd5e700 1 --2- 172.21.15.2:0/4189093985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ee8156200 0x7f4ee80b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:35:20.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.366+0000 7f4ee7fff700 1 --2- 172.21.15.2:0/4189093985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ee815d520 0x7f4ee80b1160 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400f770 tx=0x7f4ed400fb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:20.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.366+0000 7f4ee57fa700 1 -- 172.21.15.2:0/4189093985 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed40052e0 con 0x7f4ee815d520 2023-08-25T03:35:20.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.366+0000 7f4ee57fa700 1 -- 172.21.15.2:0/4189093985 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed4005460 con 0x7f4ee815d520 2023-08-25T03:35:20.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.366+0000 7f4eeefc2700 1 -- 172.21.15.2:0/4189093985 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee81f22f0 con 0x7f4ee815d520 2023-08-25T03:35:20.386 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.366+0000 7f4ee57fa700 1 -- 172.21.15.2:0/4189093985 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed40038c0 con 0x7f4ee815d520 2023-08-25T03:35:20.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.366+0000 7f4eeefc2700 1 -- 172.21.15.2:0/4189093985 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee80c9ca0 con 0x7f4ee815d520 2023-08-25T03:35:20.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.367+0000 7f4eeefc2700 1 -- 172.21.15.2:0/4189093985 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee80746d0 con 0x7f4ee815d520 2023-08-25T03:35:20.387 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.367+0000 7f4ee57fa700 1 -- 172.21.15.2:0/4189093985 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f4ed4003a20 con 0x7f4ee815d520 2023-08-25T03:35:20.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.368+0000 7f4ee57fa700 1 --2- 172.21.15.2:0/4189093985 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f4ecc06a110 0x7f4ecc06c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:20.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.368+0000 7f4eecd5e700 1 --2- 172.21.15.2:0/4189093985 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f4ecc06a110 0x7f4ecc06c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:20.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.368+0000 7f4ee57fa700 1 -- 172.21.15.2:0/4189093985 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f4ed4088360 con 0x7f4ee815d520 2023-08-25T03:35:20.388 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.368+0000 7f4eecd5e700 1 --2- 172.21.15.2:0/4189093985 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f4ecc06a110 0x7f4ecc06c5d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009b50 tx=0x7f4edc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:20.389 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.373+0000 7f4ee57fa700 1 -- 172.21.15.2:0/4189093985 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f4ed4053ae0 con 0x7f4ee815d520 2023-08-25T03:35:20.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.571+0000 7f4eeefc2700 1 -- 172.21.15.2:0/4189093985 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4ee806dbe0 con 0x7f4ee815d520 2023-08-25T03:35:20.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.571+0000 7f4ee57fa700 1 -- 172.21.15.2:0/4189093985 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4ed4057100 con 0x7f4ee815d520 2023-08-25T03:35:20.574 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-08-25T03:35:20.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.573+0000 7f4eeefc2700 1 -- 172.21.15.2:0/4189093985 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f4ecc06a110 msgr2=0x7f4ecc06c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:20.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.573+0000 7f4eeefc2700 1 --2- 172.21.15.2:0/4189093985 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f4ecc06a110 0x7f4ecc06c5d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4edc009b50 tx=0x7f4edc008040 comp rx=0 tx=0).stop 2023-08-25T03:35:20.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.573+0000 7f4eeefc2700 1 -- 172.21.15.2:0/4189093985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ee815d520 msgr2=0x7f4ee80b1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:20.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.573+0000 7f4eeefc2700 1 --2- 172.21.15.2:0/4189093985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ee815d520 0x7f4ee80b1160 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400f770 tx=0x7f4ed400fb30 comp rx=0 tx=0).stop 2023-08-25T03:35:20.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.573+0000 7f4eeefc2700 1 -- 172.21.15.2:0/4189093985 shutdown_connections 2023-08-25T03:35:20.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.573+0000 7f4eeefc2700 1 --2- 172.21.15.2:0/4189093985 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4ee8071ab0 0x7f4ee806a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.573+0000 7f4eeefc2700 1 --2- 172.21.15.2:0/4189093985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ee8156200 0x7f4ee80b0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.573+0000 7f4eeefc2700 1 --2- 172.21.15.2:0/4189093985 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f4ecc06a110 0x7f4ecc06c5d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.573+0000 7f4eeefc2700 1 --2- 172.21.15.2:0/4189093985 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f4ee815d520 0x7f4ee80b1160 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.573+0000 7f4eeefc2700 1 -- 172.21.15.2:0/4189093985 >> 172.21.15.2:0/4189093985 conn(0x7f4ee80cfd90 msgr2=0x7f4ee80b4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:20.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.573+0000 7f4eeefc2700 1 -- 172.21.15.2:0/4189093985 shutdown_connections 2023-08-25T03:35:20.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.574+0000 7f4eeefc2700 1 -- 172.21.15.2:0/4189093985 wait complete. 2023-08-25T03:35:20.588 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T03:35:20.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.714+0000 7fdf59678700 1 Processor -- start 2023-08-25T03:35:20.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.714+0000 7fdf59678700 1 -- start start 2023-08-25T03:35:20.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.714+0000 7fdf59678700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fdf5406a8f0 0x7fdf5406cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:20.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.714+0000 7fdf59678700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fdf5406e710 0x7fdf540ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:20.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.714+0000 7fdf59678700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf5406e280 con 0x7fdf5415d5e0 2023-08-25T03:35:20.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.715+0000 7fdf59678700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf5406d480 con 0x7fdf5406a8f0 2023-08-25T03:35:20.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.715+0000 7fdf59678700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf5406d670 con 0x7fdf5406e710 2023-08-25T03:35:20.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.715+0000 7fdf527fc700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fdf5406a8f0 0x7fdf5406cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:20.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.715+0000 7fdf527fc700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fdf5406a8f0 0x7fdf5406cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33490/0 (socket says 172.21.15.2:33490) 2023-08-25T03:35:20.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.715+0000 7fdf527fc700 1 -- 172.21.15.2:0/984011122 learned_addr learned my addr 172.21.15.2:0/984011122 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:20.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.715+0000 7fdf537fe700 1 --2- 172.21.15.2:0/984011122 >> v2:172.21.15.31:3300/0 conn(0x7fdf5406e710 0x7fdf540ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:20.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.715+0000 7fdf527fc700 1 -- 172.21.15.2:0/984011122 >> v2:172.21.15.31:3300/0 conn(0x7fdf5406e710 msgr2=0x7fdf540ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:20.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.715+0000 7fdf527fc700 1 --2- 172.21.15.2:0/984011122 >> v2:172.21.15.31:3300/0 conn(0x7fdf5406e710 0x7fdf540ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.715+0000 7fdf527fc700 1 -- 172.21.15.2:0/984011122 >> v1:172.21.15.31:6789/0 conn(0x7fdf5415d5e0 legacy=0x7fdf5406a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:20.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.715+0000 7fdf527fc700 1 -- 172.21.15.2:0/984011122 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf5406d950 con 0x7fdf5406a8f0 2023-08-25T03:35:20.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.715+0000 7fdf527fc700 1 --2- 172.21.15.2:0/984011122 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf5406a8f0 0x7fdf5406cf40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fdf440098f0 tx=0x7fdf44009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52b4dd3e75b76d7 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:20.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.715+0000 7fdf51ffb700 1 -- 172.21.15.2:0/984011122 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4400b860 con 0x7fdf5406a8f0 2023-08-25T03:35:20.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.715+0000 7fdf51ffb700 1 -- 172.21.15.2:0/984011122 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4400b9c0 con 0x7fdf5406a8f0 2023-08-25T03:35:20.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.715+0000 7fdf51ffb700 1 -- 172.21.15.2:0/984011122 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4400bd50 con 0x7fdf5406a8f0 2023-08-25T03:35:20.727 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.716+0000 7fdf59678700 1 -- 172.21.15.2:0/984011122 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf5406a8f0 msgr2=0x7fdf5406cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:20.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.716+0000 7fdf59678700 1 --2- 172.21.15.2:0/984011122 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf5406a8f0 0x7fdf5406cf40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fdf440098f0 tx=0x7fdf44009c00 comp rx=0 tx=0).stop 2023-08-25T03:35:20.728 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.716+0000 7fdf59678700 1 -- 172.21.15.2:0/984011122 shutdown_connections 2023-08-25T03:35:20.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.716+0000 7fdf59678700 1 --2- 172.21.15.2:0/984011122 >> v2:172.21.15.31:3300/0 conn(0x7fdf5406e710 0x7fdf540ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.716+0000 7fdf59678700 1 --2- 172.21.15.2:0/984011122 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf5406a8f0 0x7fdf5406cf40 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.716+0000 7fdf59678700 1 -- 172.21.15.2:0/984011122 >> 172.21.15.2:0/984011122 conn(0x7fdf540b1a10 msgr2=0x7fdf540b1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:20.729 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.716+0000 7fdf59678700 1 -- 172.21.15.2:0/984011122 shutdown_connections 2023-08-25T03:35:20.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.716+0000 7fdf59678700 1 -- 172.21.15.2:0/984011122 wait complete. 2023-08-25T03:35:20.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.717+0000 7fdf59678700 1 Processor -- start 2023-08-25T03:35:20.730 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.717+0000 7fdf59678700 1 -- start start 2023-08-25T03:35:20.731 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.718+0000 7fdf59678700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdf5406a8f0 0x7fdf54157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:20.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.718+0000 7fdf59678700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf5406e710 0x7fdf540c96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:20.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.718+0000 7fdf59678700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf5415d5e0 0x7fdf540ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:20.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.718+0000 7fdf59678700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf54157e80 con 0x7fdf5415d5e0 2023-08-25T03:35:20.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.718+0000 7fdf59678700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf540b3030 con 0x7fdf5406e710 2023-08-25T03:35:20.732 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.718+0000 7fdf59678700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf540b3310 con 0x7fdf5406a8f0 2023-08-25T03:35:20.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.718+0000 7fdf537fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf5415d5e0 0x7fdf540ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:20.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.718+0000 7fdf537fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf5415d5e0 0x7fdf540ceb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33498/0 (socket says 172.21.15.2:33498) 2023-08-25T03:35:20.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.718+0000 7fdf537fe700 1 -- 172.21.15.2:0/796921529 learned_addr learned my addr 172.21.15.2:0/796921529 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:20.733 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.718+0000 7fdf52ffd700 1 --2- 172.21.15.2:0/796921529 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdf5406a8f0 0x7fdf54157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:20.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.718+0000 7fdf527fc700 1 --2- 172.21.15.2:0/796921529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf5406e710 0x7fdf540c96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:20.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.718+0000 7fdf537fe700 1 -- 172.21.15.2:0/796921529 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdf5406a8f0 msgr2=0x7fdf54157940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:20.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.718+0000 7fdf537fe700 1 --2- 172.21.15.2:0/796921529 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdf5406a8f0 0x7fdf54157940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.718+0000 7fdf537fe700 1 -- 172.21.15.2:0/796921529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf5406e710 msgr2=0x7fdf540c96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:20.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.718+0000 7fdf537fe700 1 --2- 172.21.15.2:0/796921529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf5406e710 0x7fdf540c96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.718+0000 7fdf537fe700 1 -- 172.21.15.2:0/796921529 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf44009550 con 0x7fdf5415d5e0 2023-08-25T03:35:20.738 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.719+0000 7fdf537fe700 1 --2- 172.21.15.2:0/796921529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf5415d5e0 0x7fdf540ceb20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c007e40 tx=0x7fdf4c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:20.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.719+0000 7fdf3b7fe700 1 -- 172.21.15.2:0/796921529 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c00ed30 con 0x7fdf5415d5e0 2023-08-25T03:35:20.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.719+0000 7fdf3b7fe700 1 -- 172.21.15.2:0/796921529 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4c014420 con 0x7fdf5415d5e0 2023-08-25T03:35:20.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.719+0000 7fdf3b7fe700 1 -- 172.21.15.2:0/796921529 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c0147d0 con 0x7fdf5415d5e0 2023-08-25T03:35:20.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.719+0000 7fdf59678700 1 -- 172.21.15.2:0/796921529 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf540b35f0 con 0x7fdf5415d5e0 2023-08-25T03:35:20.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.719+0000 7fdf59678700 1 -- 172.21.15.2:0/796921529 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf540b3bb0 con 0x7fdf5415d5e0 2023-08-25T03:35:20.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.720+0000 7fdf59678700 1 -- 172.21.15.2:0/796921529 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf540746d0 con 0x7fdf5415d5e0 2023-08-25T03:35:20.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.720+0000 7fdf3b7fe700 1 -- 172.21.15.2:0/796921529 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fdf4c014930 con 0x7fdf5415d5e0 2023-08-25T03:35:20.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.721+0000 7fdf3b7fe700 1 --2- 172.21.15.2:0/796921529 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fdf4006a0e0 0x7fdf4006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:20.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.721+0000 7fdf3b7fe700 1 -- 172.21.15.2:0/796921529 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fdf4c088b70 con 0x7fdf5415d5e0 2023-08-25T03:35:20.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.721+0000 7fdf52ffd700 1 --2- 172.21.15.2:0/796921529 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fdf4006a0e0 0x7fdf4006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:20.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.721+0000 7fdf52ffd700 1 --2- 172.21.15.2:0/796921529 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fdf4006a0e0 0x7fdf4006c5a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdf48007fd0 tx=0x7fdf48009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:20.740 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.725+0000 7fdf3b7fe700 1 -- 172.21.15.2:0/796921529 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fdf4c0542f0 con 0x7fdf5415d5e0 2023-08-25T03:35:20.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.921+0000 7fdf59678700 1 -- 172.21.15.2:0/796921529 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdf5406dc00 con 0x7fdf5415d5e0 2023-08-25T03:35:20.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.922+0000 7fdf3b7fe700 1 -- 172.21.15.2:0/796921529 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdf4c057910 con 0x7fdf5415d5e0 2023-08-25T03:35:20.924 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-08-25T03:35:20.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.924+0000 7fdf59678700 1 -- 172.21.15.2:0/796921529 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fdf4006a0e0 msgr2=0x7fdf4006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:20.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.924+0000 7fdf59678700 1 --2- 172.21.15.2:0/796921529 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fdf4006a0e0 0x7fdf4006c5a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdf48007fd0 tx=0x7fdf48009040 comp rx=0 tx=0).stop 2023-08-25T03:35:20.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.924+0000 7fdf59678700 1 -- 172.21.15.2:0/796921529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf5415d5e0 msgr2=0x7fdf540ceb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:20.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.924+0000 7fdf59678700 1 --2- 172.21.15.2:0/796921529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf5415d5e0 0x7fdf540ceb20 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c007e40 tx=0x7fdf4c00e3c0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.924+0000 7fdf59678700 1 -- 172.21.15.2:0/796921529 shutdown_connections 2023-08-25T03:35:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.924+0000 7fdf59678700 1 --2- 172.21.15.2:0/796921529 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fdf4006a0e0 0x7fdf4006c5a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.924+0000 7fdf59678700 1 --2- 172.21.15.2:0/796921529 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fdf5415d5e0 0x7fdf540ceb20 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.924+0000 7fdf59678700 1 --2- 172.21.15.2:0/796921529 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdf5406a8f0 0x7fdf54157940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.924+0000 7fdf59678700 1 --2- 172.21.15.2:0/796921529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf5406e710 0x7fdf540c96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.924+0000 7fdf59678700 1 -- 172.21.15.2:0/796921529 >> 172.21.15.2:0/796921529 conn(0x7fdf540b1a10 msgr2=0x7fdf5415b430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:20.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.924+0000 7fdf59678700 1 -- 172.21.15.2:0/796921529 shutdown_connections 2023-08-25T03:35:20.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:20.924+0000 7fdf59678700 1 -- 172.21.15.2:0/796921529 wait complete. 2023-08-25T03:35:20.938 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-08-25T03:35:21.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.062+0000 7f2073ce3700 1 Processor -- start 2023-08-25T03:35:21.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.063+0000 7f2073ce3700 1 -- start start 2023-08-25T03:35:21.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.063+0000 7f2073ce3700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f206c15a030 0x7f206c0cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:21.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.063+0000 7f2073ce3700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f206c15a410 0x7f206c0ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:21.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.063+0000 7f2073ce3700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f206c071dc0 con 0x7f206c0cc1f0 2023-08-25T03:35:21.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.063+0000 7f2073ce3700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f206c0720a0 con 0x7f206c15a030 2023-08-25T03:35:21.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.063+0000 7f2073ce3700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206c0cb4b0 con 0x7f206c15a410 2023-08-25T03:35:21.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.063+0000 7f2071a7f700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f206c15a030 0x7f206c0cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:21.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.064+0000 7f2071a7f700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f206c15a030 0x7f206c0cbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33502/0 (socket says 172.21.15.2:33502) 2023-08-25T03:35:21.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.064+0000 7f2071a7f700 1 -- 172.21.15.2:0/3929493002 learned_addr learned my addr 172.21.15.2:0/3929493002 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:21.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.064+0000 7f2072280700 1 --2- 172.21.15.2:0/3929493002 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f206c15a410 0x7f206c0ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:21.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.064+0000 7f2071a7f700 1 -- 172.21.15.2:0/3929493002 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f206c15a410 msgr2=0x7f206c0ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:21.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.064+0000 7f2071a7f700 1 --2- 172.21.15.2:0/3929493002 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f206c15a410 0x7f206c0ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:21.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.064+0000 7f2071a7f700 1 -- 172.21.15.2:0/3929493002 >> v1:172.21.15.2:6789/0 conn(0x7f206c0cc1f0 legacy=0x7f206c0ce170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:21.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.064+0000 7f2071a7f700 1 -- 172.21.15.2:0/3929493002 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f206c0cb790 con 0x7f206c15a030 2023-08-25T03:35:21.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.064+0000 7f2072280700 1 --2- 172.21.15.2:0/3929493002 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f206c15a410 0x7f206c0ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:21.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.064+0000 7f2071a7f700 1 --2- 172.21.15.2:0/3929493002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f206c15a030 0x7f206c0cbcb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f20680098f0 tx=0x7f2068009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de020e553a16053 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:21.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.064+0000 7f2070a7d700 1 -- 172.21.15.2:0/3929493002 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206800b8a0 con 0x7f206c15a030 2023-08-25T03:35:21.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.064+0000 7f2070a7d700 1 -- 172.21.15.2:0/3929493002 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f206800ba00 con 0x7f206c15a030 2023-08-25T03:35:21.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.064+0000 7f2070a7d700 1 -- 172.21.15.2:0/3929493002 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206800bd90 con 0x7f206c15a030 2023-08-25T03:35:21.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.065+0000 7f2073ce3700 1 -- 172.21.15.2:0/3929493002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f206c15a030 msgr2=0x7f206c0cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:21.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.065+0000 7f2073ce3700 1 --2- 172.21.15.2:0/3929493002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f206c15a030 0x7f206c0cbcb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f20680098f0 tx=0x7f2068009c00 comp rx=0 tx=0).stop 2023-08-25T03:35:21.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.065+0000 7f2073ce3700 1 -- 172.21.15.2:0/3929493002 shutdown_connections 2023-08-25T03:35:21.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.065+0000 7f2073ce3700 1 --2- 172.21.15.2:0/3929493002 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f206c15a410 0x7f206c0ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:21.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.065+0000 7f2073ce3700 1 --2- 172.21.15.2:0/3929493002 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f206c15a030 0x7f206c0cbcb0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:21.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.065+0000 7f2073ce3700 1 -- 172.21.15.2:0/3929493002 >> 172.21.15.2:0/3929493002 conn(0x7f206c0b1cf0 msgr2=0x7f206c0b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:21.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.065+0000 7f2073ce3700 1 -- 172.21.15.2:0/3929493002 shutdown_connections 2023-08-25T03:35:21.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.065+0000 7f2073ce3700 1 -- 172.21.15.2:0/3929493002 wait complete. 2023-08-25T03:35:21.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.066+0000 7f2073ce3700 1 Processor -- start 2023-08-25T03:35:21.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.066+0000 7f2073ce3700 1 -- start start 2023-08-25T03:35:21.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.067+0000 7f2073ce3700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f206c0cc1f0 0x7f206c0b44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:21.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.067+0000 7f2073ce3700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f206c15a030 0x7f206c15c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:21.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.067+0000 7f2073ce3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f206c15a410 0x7f206c154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:21.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.067+0000 7f2073ce3700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c0c4ab0 con 0x7f206c15a410 2023-08-25T03:35:21.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.067+0000 7f2073ce3700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f206c0b4aa0 con 0x7f206c15a030 2023-08-25T03:35:21.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.067+0000 7f2073ce3700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206c0b4d80 con 0x7f206c0cc1f0 2023-08-25T03:35:21.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.067+0000 7f2072280700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f206c15a410 0x7f206c154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:21.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.067+0000 7f2072280700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f206c15a410 0x7f206c154280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33512/0 (socket says 172.21.15.2:33512) 2023-08-25T03:35:21.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.067+0000 7f2072280700 1 -- 172.21.15.2:0/1085034404 learned_addr learned my addr 172.21.15.2:0/1085034404 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:21.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.067+0000 7f2071a7f700 1 --2- 172.21.15.2:0/1085034404 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f206c0cc1f0 0x7f206c0b44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:21.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.067+0000 7f207127e700 1 --2- 172.21.15.2:0/1085034404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f206c15a030 0x7f206c15c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:21.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.067+0000 7f2072280700 1 -- 172.21.15.2:0/1085034404 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f206c0cc1f0 msgr2=0x7f206c0b44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:21.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.067+0000 7f2072280700 1 --2- 172.21.15.2:0/1085034404 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f206c0cc1f0 0x7f206c0b44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:21.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.067+0000 7f2072280700 1 -- 172.21.15.2:0/1085034404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f206c15a030 msgr2=0x7f206c15c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:21.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.067+0000 7f2072280700 1 --2- 172.21.15.2:0/1085034404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f206c15a030 0x7f206c15c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:21.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.067+0000 7f2072280700 1 -- 172.21.15.2:0/1085034404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2068009550 con 0x7f206c15a410 2023-08-25T03:35:21.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.067+0000 7f207127e700 1 --2- 172.21.15.2:0/1085034404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f206c15a030 0x7f206c15c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:21.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.067+0000 7f2072280700 1 --2- 172.21.15.2:0/1085034404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f206c15a410 0x7f206c154280 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2064009fd0 tx=0x7f206400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:21.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.068+0000 7f205e7fc700 1 -- 172.21.15.2:0/1085034404 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f206400ddd0 con 0x7f206c15a410 2023-08-25T03:35:21.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.068+0000 7f205e7fc700 1 -- 172.21.15.2:0/1085034404 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2064003a20 con 0x7f206c15a410 2023-08-25T03:35:21.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.068+0000 7f2073ce3700 1 -- 172.21.15.2:0/1085034404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f206c0b5060 con 0x7f206c15a410 2023-08-25T03:35:21.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.068+0000 7f205e7fc700 1 -- 172.21.15.2:0/1085034404 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2064003db0 con 0x7f206c15a410 2023-08-25T03:35:21.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.068+0000 7f2073ce3700 1 -- 172.21.15.2:0/1085034404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f206c0b5620 con 0x7f206c15a410 2023-08-25T03:35:21.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.069+0000 7f2073ce3700 1 -- 172.21.15.2:0/1085034404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f206c066f70 con 0x7f206c15a410 2023-08-25T03:35:21.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.069+0000 7f205e7fc700 1 -- 172.21.15.2:0/1085034404 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2064004030 con 0x7f206c15a410 2023-08-25T03:35:21.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.069+0000 7f205e7fc700 1 --2- 172.21.15.2:0/1085034404 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f205006a0e0 0x7f205006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:21.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.070+0000 7f205e7fc700 1 -- 172.21.15.2:0/1085034404 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f206408a7e0 con 0x7f206c15a410 2023-08-25T03:35:21.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.070+0000 7f2071a7f700 1 --2- 172.21.15.2:0/1085034404 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f205006a0e0 0x7f205006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:21.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.070+0000 7f2071a7f700 1 --2- 172.21.15.2:0/1085034404 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f205006a0e0 0x7f205006c5a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2068009c40 tx=0x7f2068008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:21.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.074+0000 7f205e7fc700 1 -- 172.21.15.2:0/1085034404 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2064055f60 con 0x7f206c15a410 2023-08-25T03:35:21.293 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.292+0000 7f2073ce3700 1 -- 172.21.15.2:0/1085034404 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f206c0b2c00 con 0x7f206c15a410 2023-08-25T03:35:21.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.292+0000 7f205e7fc700 1 -- 172.21.15.2:0/1085034404 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v37) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2064059580 con 0x7f206c15a410 2023-08-25T03:35:21.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.294+0000 7f2073ce3700 1 -- 172.21.15.2:0/1085034404 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f205006a0e0 msgr2=0x7f205006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:21.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.294+0000 7f2073ce3700 1 --2- 172.21.15.2:0/1085034404 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f205006a0e0 0x7f205006c5a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2068009c40 tx=0x7f2068008030 comp rx=0 tx=0).stop 2023-08-25T03:35:21.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.294+0000 7f2073ce3700 1 -- 172.21.15.2:0/1085034404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f206c15a410 msgr2=0x7f206c154280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:21.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.294+0000 7f2073ce3700 1 --2- 172.21.15.2:0/1085034404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f206c15a410 0x7f206c154280 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f2064009fd0 tx=0x7f206400d3b0 comp rx=0 tx=0).stop 2023-08-25T03:35:21.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.294+0000 7f2073ce3700 1 -- 172.21.15.2:0/1085034404 shutdown_connections 2023-08-25T03:35:21.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.294+0000 7f2073ce3700 1 --2- 172.21.15.2:0/1085034404 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f205006a0e0 0x7f205006c5a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:21.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.294+0000 7f2073ce3700 1 --2- 172.21.15.2:0/1085034404 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f206c15a410 0x7f206c154280 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:21.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.294+0000 7f2073ce3700 1 --2- 172.21.15.2:0/1085034404 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f206c0cc1f0 0x7f206c0b44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:21.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.294+0000 7f2073ce3700 1 --2- 172.21.15.2:0/1085034404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f206c15a030 0x7f206c15c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:21.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.294+0000 7f2073ce3700 1 -- 172.21.15.2:0/1085034404 >> 172.21.15.2:0/1085034404 conn(0x7f206c0b1cf0 msgr2=0x7f206c0aff70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:21.297 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.294+0000 7f2073ce3700 1 -- 172.21.15.2:0/1085034404 shutdown_connections 2023-08-25T03:35:21.298 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.294+0000 7f2073ce3700 1 -- 172.21.15.2:0/1085034404 wait complete. 2023-08-25T03:35:21.298 INFO:teuthology.orchestra.run.smithi002.stderr:listed 0 entries 2023-08-25T03:35:21.308 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-08-25T03:35:21.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:21.309 DEBUG:teuthology.orchestra.run.smithi031:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-08-25T03:35:21.368 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi031.front.sepia.ceph.com 2023-08-25T03:35:21.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:21.369 DEBUG:teuthology.orchestra.run.smithi031:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-08-25T03:35:21.390 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:35:21.391 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:35:21.409 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18212 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-25T03:35:21.410 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:35:21.410 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:35:21.449 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15003 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-25T03:35:21.450 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T03:35:21.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.574+0000 7f249d696700 1 Processor -- start 2023-08-25T03:35:21.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.574+0000 7f249d696700 1 -- start start 2023-08-25T03:35:21.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.575+0000 7f249d696700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f249815d370 0x7f24980d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:21.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.575+0000 7f249d696700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f24980b8aa0 0x7f24980cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:21.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.575+0000 7f249d696700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f249815d750 con 0x7f24980d2b40 2023-08-25T03:35:21.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.575+0000 7f249d696700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24980d1aa0 con 0x7f24980b8aa0 2023-08-25T03:35:21.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.575+0000 7f249d696700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24980d1d80 con 0x7f249815d370 2023-08-25T03:35:21.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.575+0000 7f24967fc700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f24980b8aa0 0x7f24980cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:21.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.575+0000 7f24977fe700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f24980d2b40 0x7f24980b4b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48416/0 (socket says 172.21.15.2:48416) 2023-08-25T03:35:21.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.575+0000 7f24977fe700 1 -- 172.21.15.2:0/194520634 learned_addr learned my addr 172.21.15.2:0/194520634 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:21.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.575+0000 7f2496ffd700 1 --2- 172.21.15.2:0/194520634 >> v2:172.21.15.31:3300/0 conn(0x7f249815d370 0x7f24980d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:21.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.575+0000 7f24967fc700 1 -- 172.21.15.2:0/194520634 >> v2:172.21.15.31:3300/0 conn(0x7f249815d370 msgr2=0x7f24980d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:21.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.575+0000 7f24967fc700 1 --2- 172.21.15.2:0/194520634 >> v2:172.21.15.31:3300/0 conn(0x7f249815d370 0x7f24980d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:21.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.575+0000 7f24967fc700 1 -- 172.21.15.2:0/194520634 >> v1:172.21.15.2:6789/0 conn(0x7f24980d2b40 legacy=0x7f24980b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:21.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.575+0000 7f2495ffb700 1 -- 172.21.15.2:0/194520634 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4129082996 0 0) 0x7f249815d750 con 0x7f24980d2b40 2023-08-25T03:35:21.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.575+0000 7f24967fc700 1 -- 172.21.15.2:0/194520634 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24980d2060 con 0x7f24980b8aa0 2023-08-25T03:35:21.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.575+0000 7f24967fc700 1 --2- 172.21.15.2:0/194520634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24980b8aa0 0x7f24980cdde0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f24980b8860 tx=0x7f2488009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f330ffb85f2afb server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:21.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.576+0000 7f2495ffb700 1 -- 172.21.15.2:0/194520634 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2488011830 con 0x7f24980b8aa0 2023-08-25T03:35:21.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.576+0000 7f2495ffb700 1 -- 172.21.15.2:0/194520634 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2488011990 con 0x7f24980b8aa0 2023-08-25T03:35:21.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.576+0000 7f2495ffb700 1 -- 172.21.15.2:0/194520634 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2488011d20 con 0x7f24980b8aa0 2023-08-25T03:35:21.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.576+0000 7f249d696700 1 -- 172.21.15.2:0/194520634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24980b8aa0 msgr2=0x7f24980cdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:21.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.576+0000 7f249d696700 1 --2- 172.21.15.2:0/194520634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24980b8aa0 0x7f24980cdde0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f24980b8860 tx=0x7f2488009b20 comp rx=0 tx=0).stop 2023-08-25T03:35:21.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.576+0000 7f249d696700 1 -- 172.21.15.2:0/194520634 shutdown_connections 2023-08-25T03:35:21.581 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.576+0000 7f249d696700 1 --2- 172.21.15.2:0/194520634 >> v2:172.21.15.31:3300/0 conn(0x7f249815d370 0x7f24980d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:21.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.576+0000 7f249d696700 1 --2- 172.21.15.2:0/194520634 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24980b8aa0 0x7f24980cdde0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:21.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.576+0000 7f249d696700 1 -- 172.21.15.2:0/194520634 >> 172.21.15.2:0/194520634 conn(0x7f249806ab50 msgr2=0x7f249806af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:21.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.576+0000 7f249d696700 1 -- 172.21.15.2:0/194520634 shutdown_connections 2023-08-25T03:35:21.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.576+0000 7f249d696700 1 -- 172.21.15.2:0/194520634 wait complete. 2023-08-25T03:35:21.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.578+0000 7f249d696700 1 Processor -- start 2023-08-25T03:35:21.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.578+0000 7f249d696700 1 -- start start 2023-08-25T03:35:21.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.578+0000 7f249d696700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24980b8aa0 0x7f249815eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:21.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.578+0000 7f249d696700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f24980d2b40 0x7f249814a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:21.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.578+0000 7f249d696700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f249815d370 0x7f2498157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:21.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.578+0000 7f249d696700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24981f5ed0 con 0x7f24980b8aa0 2023-08-25T03:35:21.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.578+0000 7f249d696700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24981f6120 con 0x7f249815d370 2023-08-25T03:35:21.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.578+0000 7f249d696700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24981f6400 con 0x7f24980d2b40 2023-08-25T03:35:21.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.579+0000 7f2496ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24980b8aa0 0x7f249815eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:21.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.579+0000 7f2496ffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24980b8aa0 0x7f249815eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33522/0 (socket says 172.21.15.2:33522) 2023-08-25T03:35:21.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.579+0000 7f2496ffd700 1 -- 172.21.15.2:0/2871405451 learned_addr learned my addr 172.21.15.2:0/2871405451 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:21.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.579+0000 7f24977fe700 1 --2- 172.21.15.2:0/2871405451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f249815d370 0x7f2498157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:21.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.579+0000 7f24967fc700 1 --2- 172.21.15.2:0/2871405451 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f24980d2b40 0x7f249814a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:21.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.579+0000 7f2496ffd700 1 -- 172.21.15.2:0/2871405451 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f24980d2b40 msgr2=0x7f249814a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:21.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.579+0000 7f2496ffd700 1 --2- 172.21.15.2:0/2871405451 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f24980d2b40 0x7f249814a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:21.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.579+0000 7f2496ffd700 1 -- 172.21.15.2:0/2871405451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f249815d370 msgr2=0x7f2498157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:21.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.579+0000 7f2496ffd700 1 --2- 172.21.15.2:0/2871405451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f249815d370 0x7f2498157600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:21.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.579+0000 7f2496ffd700 1 -- 172.21.15.2:0/2871405451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24880094a0 con 0x7f24980b8aa0 2023-08-25T03:35:21.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.579+0000 7f24967fc700 1 --2- 172.21.15.2:0/2871405451 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f24980d2b40 0x7f249814a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:35:21.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.579+0000 7f24977fe700 1 --2- 172.21.15.2:0/2871405451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f249815d370 0x7f2498157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:21.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.579+0000 7f2496ffd700 1 --2- 172.21.15.2:0/2871405451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24980b8aa0 0x7f249815eec0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f248c00e9d0 tx=0x7f248c00ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:21.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.579+0000 7f247f7fe700 1 -- 172.21.15.2:0/2871405451 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c009730 con 0x7f24980b8aa0 2023-08-25T03:35:21.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.579+0000 7f247f7fe700 1 -- 172.21.15.2:0/2871405451 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f248c00c850 con 0x7f24980b8aa0 2023-08-25T03:35:21.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.579+0000 7f247f7fe700 1 -- 172.21.15.2:0/2871405451 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f248c00cbe0 con 0x7f24980b8aa0 2023-08-25T03:35:21.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.579+0000 7f249d696700 1 -- 172.21.15.2:0/2871405451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24981f66e0 con 0x7f24980b8aa0 2023-08-25T03:35:21.590 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.579+0000 7f249d696700 1 -- 172.21.15.2:0/2871405451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f249814aa60 con 0x7f24980b8aa0 2023-08-25T03:35:21.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.580+0000 7f249d696700 1 -- 172.21.15.2:0/2871405451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f249815bc50 con 0x7f24980b8aa0 2023-08-25T03:35:21.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.582+0000 7f247f7fe700 1 -- 172.21.15.2:0/2871405451 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f248c00cd40 con 0x7f24980b8aa0 2023-08-25T03:35:21.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.583+0000 7f247f7fe700 1 --2- 172.21.15.2:0/2871405451 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f248406a0e0 0x7f248406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:21.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.583+0000 7f247f7fe700 1 -- 172.21.15.2:0/2871405451 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f248c088e50 con 0x7f24980b8aa0 2023-08-25T03:35:21.591 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.583+0000 7f24967fc700 1 --2- 172.21.15.2:0/2871405451 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f248406a0e0 0x7f248406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:21.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.583+0000 7f24967fc700 1 --2- 172.21.15.2:0/2871405451 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f248406a0e0 0x7f248406c5a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f24980b8860 tx=0x7f2488008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:21.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.587+0000 7f247f7fe700 1 -- 172.21.15.2:0/2871405451 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f248c0580b0 con 0x7f24980b8aa0 2023-08-25T03:35:21.832 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.830+0000 7f249d696700 1 -- 172.21.15.2:0/2871405451 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24980d2310 con 0x7f24980b8aa0 2023-08-25T03:35:21.833 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.831+0000 7f247f7fe700 1 -- 172.21.15.2:0/2871405451 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4869 (secure 0 0 0) 0x7f248c017090 con 0x7f24980b8aa0 2023-08-25T03:35:21.833 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:35:21.833 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4751,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/3802947908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3802947908},{"type":"v1","addr":"172.21.15.31:6834","nonce":3802947908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1470640897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6835","nonce":1470640897},{"type":"v1","addr":"172.21.15.2:6837","nonce":1470640897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6836/3706466654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3706466654},{"type":"v1","addr":"172.21.15.2:6836","nonce":3706466654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T03:35:07.333851+0000","modified":"2023-08-25T03:35:10.931788+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5229":{"gid":5229,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1140461626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1140461626},{"type":"v1","addr":"172.21.15.31:6835","nonce":1140461626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-25T03:35:21.834 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.833+0000 7f249d696700 1 -- 172.21.15.2:0/2871405451 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f248406a0e0 msgr2=0x7f248406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:21.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.833+0000 7f249d696700 1 --2- 172.21.15.2:0/2871405451 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f248406a0e0 0x7f248406c5a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f24980b8860 tx=0x7f2488008030 comp rx=0 tx=0).stop 2023-08-25T03:35:21.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.833+0000 7f249d696700 1 -- 172.21.15.2:0/2871405451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24980b8aa0 msgr2=0x7f249815eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:21.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.833+0000 7f249d696700 1 --2- 172.21.15.2:0/2871405451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24980b8aa0 0x7f249815eec0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f248c00e9d0 tx=0x7f248c00ed90 comp rx=0 tx=0).stop 2023-08-25T03:35:21.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.833+0000 7f249d696700 1 -- 172.21.15.2:0/2871405451 shutdown_connections 2023-08-25T03:35:21.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.833+0000 7f249d696700 1 --2- 172.21.15.2:0/2871405451 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f248406a0e0 0x7f248406c5a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:21.835 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.833+0000 7f249d696700 1 --2- 172.21.15.2:0/2871405451 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f24980b8aa0 0x7f249815eec0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:21.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.833+0000 7f249d696700 1 --2- 172.21.15.2:0/2871405451 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f249815d370 0x7f2498157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:21.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.833+0000 7f249d696700 1 --2- 172.21.15.2:0/2871405451 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f24980d2b40 0x7f249814a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:21.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.833+0000 7f249d696700 1 -- 172.21.15.2:0/2871405451 >> 172.21.15.2:0/2871405451 conn(0x7f249806ab50 msgr2=0x7f24980b6270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:21.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.833+0000 7f249d696700 1 -- 172.21.15.2:0/2871405451 shutdown_connections 2023-08-25T03:35:21.836 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.833+0000 7f249d696700 1 -- 172.21.15.2:0/2871405451 wait complete. 2023-08-25T03:35:21.836 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2023-08-25T03:35:21.850 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:35:21.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.975+0000 7fe7d9de7700 1 Processor -- start 2023-08-25T03:35:21.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.975+0000 7fe7d9de7700 1 -- start start 2023-08-25T03:35:21.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.976+0000 7fe7d9de7700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe7d406b7e0 0x7fe7d40cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:21.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.976+0000 7fe7d9de7700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe7d40cd730 0x7fe7d40712f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:21.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.976+0000 7fe7d9de7700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7d40cb8b0 con 0x7fe7d415d220 2023-08-25T03:35:21.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.976+0000 7fe7d9de7700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7d4070810 con 0x7fe7d40cd730 2023-08-25T03:35:21.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.976+0000 7fe7d9de7700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7d4070af0 con 0x7fe7d406b7e0 2023-08-25T03:35:21.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.976+0000 7fe7d37fe700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fe7d415d220 0x7fe7d40cb0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48428/0 (socket says 172.21.15.2:48428) 2023-08-25T03:35:21.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.976+0000 7fe7d37fe700 1 -- 172.21.15.2:0/3380945105 learned_addr learned my addr 172.21.15.2:0/3380945105 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:21.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.976+0000 7fe7d3fff700 1 --2- 172.21.15.2:0/3380945105 >> v2:172.21.15.2:3300/0 conn(0x7fe7d40cd730 0x7fe7d40712f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:21.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.976+0000 7fe7d2ffd700 1 --2- 172.21.15.2:0/3380945105 >> v2:172.21.15.31:3300/0 conn(0x7fe7d406b7e0 0x7fe7d40cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:21.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.976+0000 7fe7d3fff700 1 -- 172.21.15.2:0/3380945105 >> v2:172.21.15.31:3300/0 conn(0x7fe7d406b7e0 msgr2=0x7fe7d40cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:21.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.976+0000 7fe7d3fff700 1 --2- 172.21.15.2:0/3380945105 >> v2:172.21.15.31:3300/0 conn(0x7fe7d406b7e0 0x7fe7d40cd1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:21.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.976+0000 7fe7d3fff700 1 -- 172.21.15.2:0/3380945105 >> v1:172.21.15.2:6789/0 conn(0x7fe7d415d220 legacy=0x7fe7d40cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:21.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.976+0000 7fe7d3fff700 1 -- 172.21.15.2:0/3380945105 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7d4070dd0 con 0x7fe7d40cd730 2023-08-25T03:35:21.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.976+0000 7fe7d27fc700 1 -- 172.21.15.2:0/3380945105 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 391274053 0 0) 0x7fe7d40cb8b0 con 0x7fe7d415d220 2023-08-25T03:35:21.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.976+0000 7fe7d3fff700 1 --2- 172.21.15.2:0/3380945105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7d40cd730 0x7fe7d40712f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe7d40b2120 tx=0x7fe7cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e9f26e56afff31 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:21.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.977+0000 7fe7d27fc700 1 -- 172.21.15.2:0/3380945105 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc00e070 con 0x7fe7d40cd730 2023-08-25T03:35:21.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.977+0000 7fe7d27fc700 1 -- 172.21.15.2:0/3380945105 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7cc00bbd0 con 0x7fe7d40cd730 2023-08-25T03:35:21.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.977+0000 7fe7d27fc700 1 -- 172.21.15.2:0/3380945105 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7cc012430 con 0x7fe7d40cd730 2023-08-25T03:35:21.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.977+0000 7fe7d9de7700 1 -- 172.21.15.2:0/3380945105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7d40cd730 msgr2=0x7fe7d40712f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:21.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.977+0000 7fe7d9de7700 1 --2- 172.21.15.2:0/3380945105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7d40cd730 0x7fe7d40712f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe7d40b2120 tx=0x7fe7cc00b330 comp rx=0 tx=0).stop 2023-08-25T03:35:21.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.977+0000 7fe7d9de7700 1 -- 172.21.15.2:0/3380945105 shutdown_connections 2023-08-25T03:35:21.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.977+0000 7fe7d9de7700 1 --2- 172.21.15.2:0/3380945105 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7d40cd730 0x7fe7d40712f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:21.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.977+0000 7fe7d9de7700 1 --2- 172.21.15.2:0/3380945105 >> v2:172.21.15.31:3300/0 conn(0x7fe7d406b7e0 0x7fe7d40cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:21.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.977+0000 7fe7d9de7700 1 -- 172.21.15.2:0/3380945105 >> 172.21.15.2:0/3380945105 conn(0x7fe7d40cc0a0 msgr2=0x7fe7d40cc4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:21.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.977+0000 7fe7d9de7700 1 -- 172.21.15.2:0/3380945105 shutdown_connections 2023-08-25T03:35:21.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.977+0000 7fe7d9de7700 1 -- 172.21.15.2:0/3380945105 wait complete. 2023-08-25T03:35:21.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.979+0000 7fe7d9de7700 1 Processor -- start 2023-08-25T03:35:21.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.979+0000 7fe7d9de7700 1 -- start start 2023-08-25T03:35:21.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.980+0000 7fe7d9de7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7d406b7e0 0x7fe7d40b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:21.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.980+0000 7fe7d9de7700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7d40cd730 0x7fe7d4161030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:21.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.980+0000 7fe7d9de7700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7d415d220 0x7fe7d415e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:21.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.980+0000 7fe7d9de7700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d4160c10 con 0x7fe7d406b7e0 2023-08-25T03:35:21.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.980+0000 7fe7d9de7700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d415ff90 con 0x7fe7d40cd730 2023-08-25T03:35:21.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.980+0000 7fe7d9de7700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d4160270 con 0x7fe7d415d220 2023-08-25T03:35:21.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.980+0000 7fe7d37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7d406b7e0 0x7fe7d40b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:21.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.980+0000 7fe7d37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7d406b7e0 0x7fe7d40b3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33538/0 (socket says 172.21.15.2:33538) 2023-08-25T03:35:21.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.980+0000 7fe7d37fe700 1 -- 172.21.15.2:0/3884306017 learned_addr learned my addr 172.21.15.2:0/3884306017 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:21.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.980+0000 7fe7d2ffd700 1 --2- 172.21.15.2:0/3884306017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7d40cd730 0x7fe7d4161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:21.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.980+0000 7fe7d3fff700 1 --2- 172.21.15.2:0/3884306017 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7d415d220 0x7fe7d415e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:21.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.980+0000 7fe7d37fe700 1 -- 172.21.15.2:0/3884306017 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7d415d220 msgr2=0x7fe7d415e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:21.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.980+0000 7fe7d37fe700 1 --2- 172.21.15.2:0/3884306017 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7d415d220 0x7fe7d415e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:21.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.980+0000 7fe7d37fe700 1 -- 172.21.15.2:0/3884306017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7d40cd730 msgr2=0x7fe7d4161030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:21.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.980+0000 7fe7d37fe700 1 --2- 172.21.15.2:0/3884306017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7d40cd730 0x7fe7d4161030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:21.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.980+0000 7fe7d37fe700 1 -- 172.21.15.2:0/3884306017 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7cc009a70 con 0x7fe7d406b7e0 2023-08-25T03:35:21.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.980+0000 7fe7d2ffd700 1 --2- 172.21.15.2:0/3884306017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7d40cd730 0x7fe7d4161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:21.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.980+0000 7fe7d37fe700 1 --2- 172.21.15.2:0/3884306017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7d406b7e0 0x7fe7d40b3f30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe7c400c550 tx=0x7fe7c400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:21.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.981+0000 7fe7bbfff700 1 -- 172.21.15.2:0/3884306017 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4004c80 con 0x7fe7d406b7e0 2023-08-25T03:35:21.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.981+0000 7fe7bbfff700 1 -- 172.21.15.2:0/3884306017 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c4004e00 con 0x7fe7d406b7e0 2023-08-25T03:35:21.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.981+0000 7fe7bbfff700 1 -- 172.21.15.2:0/3884306017 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c4007310 con 0x7fe7d406b7e0 2023-08-25T03:35:21.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.981+0000 7fe7d9de7700 1 -- 172.21.15.2:0/3884306017 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7d415d780 con 0x7fe7d406b7e0 2023-08-25T03:35:21.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.981+0000 7fe7d9de7700 1 -- 172.21.15.2:0/3884306017 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7d4160830 con 0x7fe7d406b7e0 2023-08-25T03:35:21.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.982+0000 7fe7d9de7700 1 -- 172.21.15.2:0/3884306017 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7d40746d0 con 0x7fe7d406b7e0 2023-08-25T03:35:21.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.982+0000 7fe7bbfff700 1 -- 172.21.15.2:0/3884306017 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe7c4007470 con 0x7fe7d406b7e0 2023-08-25T03:35:21.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.983+0000 7fe7bbfff700 1 --2- 172.21.15.2:0/3884306017 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe7c006a0e0 0x7fe7c006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:21.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.983+0000 7fe7bbfff700 1 -- 172.21.15.2:0/3884306017 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fe7c4088870 con 0x7fe7d406b7e0 2023-08-25T03:35:21.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.983+0000 7fe7d2ffd700 1 --2- 172.21.15.2:0/3884306017 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe7c006a0e0 0x7fe7c006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:21.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.983+0000 7fe7d2ffd700 1 --2- 172.21.15.2:0/3884306017 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe7c006a0e0 0x7fe7c006c5a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe7d415ca70 tx=0x7fe7c8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:21.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:21.987+0000 7fe7bbfff700 1 -- 172.21.15.2:0/3884306017 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe7c4054040 con 0x7fe7d406b7e0 2023-08-25T03:35:22.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.188+0000 7fe7d9de7700 1 -- 172.21.15.2:0/3884306017 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe7d40665a0 con 0x7fe7d406b7e0 2023-08-25T03:35:22.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.189+0000 7fe7bbfff700 1 -- 172.21.15.2:0/3884306017 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18207 (secure 0 0 0) 0x7fe7c4080e60 con 0x7fe7d406b7e0 2023-08-25T03:35:22.191 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:35:22.191 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":37,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:35:08.407381+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:35:05.870790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-08-25T03:35:06.296959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:35:22.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.191+0000 7fe7d9de7700 1 -- 172.21.15.2:0/3884306017 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe7c006a0e0 msgr2=0x7fe7c006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:22.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.191+0000 7fe7d9de7700 1 --2- 172.21.15.2:0/3884306017 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe7c006a0e0 0x7fe7c006c5a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe7d415ca70 tx=0x7fe7c8008040 comp rx=0 tx=0).stop 2023-08-25T03:35:22.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.191+0000 7fe7d9de7700 1 -- 172.21.15.2:0/3884306017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7d406b7e0 msgr2=0x7fe7d40b3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:22.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.191+0000 7fe7d9de7700 1 --2- 172.21.15.2:0/3884306017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7d406b7e0 0x7fe7d40b3f30 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fe7c400c550 tx=0x7fe7c400c910 comp rx=0 tx=0).stop 2023-08-25T03:35:22.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.191+0000 7fe7d9de7700 1 -- 172.21.15.2:0/3884306017 shutdown_connections 2023-08-25T03:35:22.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.191+0000 7fe7d9de7700 1 --2- 172.21.15.2:0/3884306017 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe7c006a0e0 0x7fe7c006c5a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:22.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.191+0000 7fe7d9de7700 1 --2- 172.21.15.2:0/3884306017 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe7d406b7e0 0x7fe7d40b3f30 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:22.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.191+0000 7fe7d9de7700 1 --2- 172.21.15.2:0/3884306017 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7d415d220 0x7fe7d415e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:22.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.191+0000 7fe7d9de7700 1 --2- 172.21.15.2:0/3884306017 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7d40cd730 0x7fe7d4161030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:22.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.191+0000 7fe7d9de7700 1 -- 172.21.15.2:0/3884306017 >> 172.21.15.2:0/3884306017 conn(0x7fe7d40cc0a0 msgr2=0x7fe7d40b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:22.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.191+0000 7fe7d9de7700 1 -- 172.21.15.2:0/3884306017 shutdown_connections 2023-08-25T03:35:22.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.191+0000 7fe7d9de7700 1 -- 172.21.15.2:0/3884306017 wait complete. 2023-08-25T03:35:22.205 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-08-25T03:33:49.560643+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-08-25T03:33:56.265184+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-08-25T03:35:05.870790+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-08-25T03:35:06.296959+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-08-25T03:35:22.206 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-08-25T03:35:22.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.332+0000 7f2f40f17700 1 Processor -- start 2023-08-25T03:35:22.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.332+0000 7f2f40f17700 1 -- start start 2023-08-25T03:35:22.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.333+0000 7f2f40f17700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2f3c06f580 0x7f2f3c1681f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:22.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.333+0000 7f2f40f17700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f3c0d1b40 con 0x7f2f3c15d360 2023-08-25T03:35:22.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.333+0000 7f2f40f17700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f3c06e0e0 con 0x7f2f3c0cfd80 2023-08-25T03:35:22.335 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.333+0000 7f2f40f17700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f3c06c320 con 0x7f2f3c06f580 2023-08-25T03:35:22.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.333+0000 7f2f3ad9d700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f2f3c06f580 0x7f2f3c1681f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:22.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.333+0000 7f2f3a59c700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f2f3c15d360 0x7f2f3c06b500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48442/0 (socket says 172.21.15.2:48442) 2023-08-25T03:35:22.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.333+0000 7f2f3a59c700 1 -- 172.21.15.2:0/1645916094 learned_addr learned my addr 172.21.15.2:0/1645916094 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:22.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.333+0000 7f2f3ad9d700 1 -- 172.21.15.2:0/1645916094 >> v1:172.21.15.31:6789/0 conn(0x7f2f3c0cfd80 legacy=0x7f2f3c06bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:22.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.333+0000 7f2f3ad9d700 1 -- 172.21.15.2:0/1645916094 >> v1:172.21.15.2:6789/0 conn(0x7f2f3c15d360 legacy=0x7f2f3c06b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:22.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.333+0000 7f2f3ad9d700 1 -- 172.21.15.2:0/1645916094 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3c0d0620 con 0x7f2f3c06f580 2023-08-25T03:35:22.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.333+0000 7f2f3959a700 1 -- 172.21.15.2:0/1645916094 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1070913279 0 0) 0x7f2f3c0d1b40 con 0x7f2f3c15d360 2023-08-25T03:35:22.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.333+0000 7f2f3ad9d700 1 --2- 172.21.15.2:0/1645916094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f3c06f580 0x7f2f3c1681f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009d90 tx=0x7f2f3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c29253c71f48097 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:22.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.334+0000 7f2f3959a700 1 -- 172.21.15.2:0/1645916094 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3400e070 con 0x7f2f3c06f580 2023-08-25T03:35:22.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.334+0000 7f2f3959a700 1 -- 172.21.15.2:0/1645916094 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3400bb90 con 0x7f2f3c06f580 2023-08-25T03:35:22.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.334+0000 7f2f3959a700 1 -- 172.21.15.2:0/1645916094 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34003a20 con 0x7f2f3c06f580 2023-08-25T03:35:22.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.334+0000 7f2f40f17700 1 -- 172.21.15.2:0/1645916094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f3c06f580 msgr2=0x7f2f3c1681f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:22.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.334+0000 7f2f40f17700 1 --2- 172.21.15.2:0/1645916094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f3c06f580 0x7f2f3c1681f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009d90 tx=0x7f2f3400b330 comp rx=0 tx=0).stop 2023-08-25T03:35:22.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.334+0000 7f2f40f17700 1 -- 172.21.15.2:0/1645916094 shutdown_connections 2023-08-25T03:35:22.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.334+0000 7f2f40f17700 1 --2- 172.21.15.2:0/1645916094 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f3c06f580 0x7f2f3c1681f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:22.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.334+0000 7f2f40f17700 1 -- 172.21.15.2:0/1645916094 >> 172.21.15.2:0/1645916094 conn(0x7f2f3c09a200 msgr2=0x7f2f3c0d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:22.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.334+0000 7f2f40f17700 1 -- 172.21.15.2:0/1645916094 shutdown_connections 2023-08-25T03:35:22.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.334+0000 7f2f40f17700 1 -- 172.21.15.2:0/1645916094 wait complete. 2023-08-25T03:35:22.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.336+0000 7f2f40f17700 1 Processor -- start 2023-08-25T03:35:22.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.336+0000 7f2f40f17700 1 -- start start 2023-08-25T03:35:22.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.336+0000 7f2f40f17700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f3c06f580 0x7f2f3c152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:22.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.336+0000 7f2f40f17700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2f3c0cfd80 0x7f2f3c155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:22.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.336+0000 7f2f40f17700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f3c15d360 0x7f2f3c0b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:22.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.336+0000 7f2f40f17700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c1549b0 con 0x7f2f3c15d360 2023-08-25T03:35:22.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.336+0000 7f2f40f17700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c154c90 con 0x7f2f3c06f580 2023-08-25T03:35:22.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.336+0000 7f2f40f17700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c154f70 con 0x7f2f3c0cfd80 2023-08-25T03:35:22.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.336+0000 7f2f3ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f3c15d360 0x7f2f3c0b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:22.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.336+0000 7f2f3ad9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f3c15d360 0x7f2f3c0b6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33554/0 (socket says 172.21.15.2:33554) 2023-08-25T03:35:22.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.336+0000 7f2f3ad9d700 1 -- 172.21.15.2:0/2627711124 learned_addr learned my addr 172.21.15.2:0/2627711124 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:22.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.337+0000 7f2f3a59c700 1 --2- 172.21.15.2:0/2627711124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f3c06f580 0x7f2f3c152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:22.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.337+0000 7f2f3ad9d700 1 -- 172.21.15.2:0/2627711124 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2f3c0cfd80 msgr2=0x7f2f3c155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:22.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.337+0000 7f2f39d9b700 1 --2- 172.21.15.2:0/2627711124 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2f3c0cfd80 0x7f2f3c155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:22.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.337+0000 7f2f3ad9d700 1 --2- 172.21.15.2:0/2627711124 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2f3c0cfd80 0x7f2f3c155a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:22.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.337+0000 7f2f3ad9d700 1 -- 172.21.15.2:0/2627711124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f3c06f580 msgr2=0x7f2f3c152390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:22.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.337+0000 7f2f3ad9d700 1 --2- 172.21.15.2:0/2627711124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f3c06f580 0x7f2f3c152390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:22.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.337+0000 7f2f3ad9d700 1 -- 172.21.15.2:0/2627711124 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f34009a70 con 0x7f2f3c15d360 2023-08-25T03:35:22.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.337+0000 7f2f3a59c700 1 --2- 172.21.15.2:0/2627711124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f3c06f580 0x7f2f3c152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:35:22.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.337+0000 7f2f3ad9d700 1 --2- 172.21.15.2:0/2627711124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f3c15d360 0x7f2f3c0b6310 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2f34004000 tx=0x7f2f34004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:22.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.337+0000 7f2f2affd700 1 -- 172.21.15.2:0/2627711124 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34012630 con 0x7f2f3c15d360 2023-08-25T03:35:22.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.337+0000 7f2f2affd700 1 -- 172.21.15.2:0/2627711124 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f340127b0 con 0x7f2f3c15d360 2023-08-25T03:35:22.343 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.337+0000 7f2f40f17700 1 -- 172.21.15.2:0/2627711124 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f3c155250 con 0x7f2f3c15d360 2023-08-25T03:35:22.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.337+0000 7f2f2affd700 1 -- 172.21.15.2:0/2627711124 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34012b60 con 0x7f2f3c15d360 2023-08-25T03:35:22.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.337+0000 7f2f40f17700 1 -- 172.21.15.2:0/2627711124 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3c1528d0 con 0x7f2f3c15d360 2023-08-25T03:35:22.344 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.338+0000 7f2f40f17700 1 -- 172.21.15.2:0/2627711124 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f3c0d1790 con 0x7f2f3c15d360 2023-08-25T03:35:22.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.339+0000 7f2f2affd700 1 -- 172.21.15.2:0/2627711124 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2f34012cc0 con 0x7f2f3c15d360 2023-08-25T03:35:22.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.339+0000 7f2f2affd700 1 --2- 172.21.15.2:0/2627711124 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2f1c06a0e0 0x7f2f1c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:22.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.339+0000 7f2f2affd700 1 -- 172.21.15.2:0/2627711124 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f2f3408a5c0 con 0x7f2f3c15d360 2023-08-25T03:35:22.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.339+0000 7f2f3a59c700 1 --2- 172.21.15.2:0/2627711124 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2f1c06a0e0 0x7f2f1c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:22.345 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.339+0000 7f2f3a59c700 1 --2- 172.21.15.2:0/2627711124 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2f1c06a0e0 0x7f2f1c06c5a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c007fd0 tx=0x7f2f2c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:22.346 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.343+0000 7f2f2affd700 1 -- 172.21.15.2:0/2627711124 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f2f340597d0 con 0x7f2f3c15d360 2023-08-25T03:35:22.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.542+0000 7f2f40f17700 1 -- 172.21.15.2:0/2627711124 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2f3c0746d0 con 0x7f2f3c15d360 2023-08-25T03:35:22.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.542+0000 7f2f2affd700 1 -- 172.21.15.2:0/2627711124 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2f34059360 con 0x7f2f3c15d360 2023-08-25T03:35:22.545 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 1 2023-08-25T03:35:22.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.544+0000 7f2f40f17700 1 -- 172.21.15.2:0/2627711124 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2f1c06a0e0 msgr2=0x7f2f1c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:22.546 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.544+0000 7f2f40f17700 1 --2- 172.21.15.2:0/2627711124 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2f1c06a0e0 0x7f2f1c06c5a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c007fd0 tx=0x7f2f2c009040 comp rx=0 tx=0).stop 2023-08-25T03:35:22.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.544+0000 7f2f40f17700 1 -- 172.21.15.2:0/2627711124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f3c15d360 msgr2=0x7f2f3c0b6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:22.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.544+0000 7f2f40f17700 1 --2- 172.21.15.2:0/2627711124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f3c15d360 0x7f2f3c0b6310 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2f34004000 tx=0x7f2f34004ad0 comp rx=0 tx=0).stop 2023-08-25T03:35:22.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.544+0000 7f2f40f17700 1 -- 172.21.15.2:0/2627711124 shutdown_connections 2023-08-25T03:35:22.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.544+0000 7f2f40f17700 1 --2- 172.21.15.2:0/2627711124 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2f1c06a0e0 0x7f2f1c06c5a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:22.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.544+0000 7f2f40f17700 1 --2- 172.21.15.2:0/2627711124 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2f3c15d360 0x7f2f3c0b6310 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:22.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.544+0000 7f2f40f17700 1 --2- 172.21.15.2:0/2627711124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f3c06f580 0x7f2f3c152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:22.547 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.544+0000 7f2f40f17700 1 --2- 172.21.15.2:0/2627711124 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2f3c0cfd80 0x7f2f3c155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:22.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.544+0000 7f2f40f17700 1 -- 172.21.15.2:0/2627711124 >> 172.21.15.2:0/2627711124 conn(0x7f2f3c09a200 msgr2=0x7f2f3c06bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:22.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.545+0000 7f2f40f17700 1 -- 172.21.15.2:0/2627711124 shutdown_connections 2023-08-25T03:35:22.548 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.545+0000 7f2f40f17700 1 -- 172.21.15.2:0/2627711124 wait complete. 2023-08-25T03:35:22.559 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-08-25T03:35:22.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.686+0000 7fc65b297700 1 Processor -- start 2023-08-25T03:35:22.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.686+0000 7fc65b297700 1 -- start start 2023-08-25T03:35:22.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.687+0000 7fc65b297700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fc6540af180 0x7fc654070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:22.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.687+0000 7fc65b297700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fc65406a1e0 0x7fc65406a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:22.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.687+0000 7fc65b297700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc654160bd0 con 0x7fc6541607f0 2023-08-25T03:35:22.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.687+0000 7fc65b297700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc65406abd0 con 0x7fc65406a1e0 2023-08-25T03:35:22.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.687+0000 7fc65b297700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6540713a0 con 0x7fc6540af180 2023-08-25T03:35:22.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.687+0000 7fc659834700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fc65406a1e0 0x7fc65406a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:22.692 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.687+0000 7fc659834700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fc65406a1e0 0x7fc65406a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33562/0 (socket says 172.21.15.2:33562) 2023-08-25T03:35:22.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.687+0000 7fc659834700 1 -- 172.21.15.2:0/2138033918 learned_addr learned my addr 172.21.15.2:0/2138033918 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:22.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.687+0000 7fc658832700 1 --2- 172.21.15.2:0/2138033918 >> v2:172.21.15.31:3300/0 conn(0x7fc6540af180 0x7fc654070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:22.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.687+0000 7fc659834700 1 -- 172.21.15.2:0/2138033918 >> v2:172.21.15.31:3300/0 conn(0x7fc6540af180 msgr2=0x7fc654070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:22.693 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.687+0000 7fc659834700 1 --2- 172.21.15.2:0/2138033918 >> v2:172.21.15.31:3300/0 conn(0x7fc6540af180 0x7fc654070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:22.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.687+0000 7fc659834700 1 -- 172.21.15.2:0/2138033918 >> v1:172.21.15.31:6789/0 conn(0x7fc6541607f0 legacy=0x7fc6540ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:22.694 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.687+0000 7fc659834700 1 -- 172.21.15.2:0/2138033918 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6540cdd00 con 0x7fc65406a1e0 2023-08-25T03:35:22.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.688+0000 7fc659834700 1 --2- 172.21.15.2:0/2138033918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc65406a1e0 0x7fc65406a690 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc654071f20 tx=0x7fc64c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660d9f76788f03d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:22.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.688+0000 7fc64bfff700 1 -- 172.21.15.2:0/2138033918 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64c00e070 con 0x7fc65406a1e0 2023-08-25T03:35:22.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.688+0000 7fc64bfff700 1 -- 172.21.15.2:0/2138033918 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64c003a70 con 0x7fc65406a1e0 2023-08-25T03:35:22.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.688+0000 7fc64bfff700 1 -- 172.21.15.2:0/2138033918 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64c003e20 con 0x7fc65406a1e0 2023-08-25T03:35:22.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.688+0000 7fc65b297700 1 -- 172.21.15.2:0/2138033918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc65406a1e0 msgr2=0x7fc65406a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:22.695 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.688+0000 7fc65b297700 1 --2- 172.21.15.2:0/2138033918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc65406a1e0 0x7fc65406a690 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc654071f20 tx=0x7fc64c00b330 comp rx=0 tx=0).stop 2023-08-25T03:35:22.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.688+0000 7fc65b297700 1 -- 172.21.15.2:0/2138033918 shutdown_connections 2023-08-25T03:35:22.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.688+0000 7fc65b297700 1 --2- 172.21.15.2:0/2138033918 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc65406a1e0 0x7fc65406a690 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:22.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.688+0000 7fc65b297700 1 --2- 172.21.15.2:0/2138033918 >> v2:172.21.15.31:3300/0 conn(0x7fc6540af180 0x7fc654070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:22.696 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.688+0000 7fc65b297700 1 -- 172.21.15.2:0/2138033918 >> 172.21.15.2:0/2138033918 conn(0x7fc6540cfd90 msgr2=0x7fc6540d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:22.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.688+0000 7fc65b297700 1 -- 172.21.15.2:0/2138033918 shutdown_connections 2023-08-25T03:35:22.697 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.688+0000 7fc65b297700 1 -- 172.21.15.2:0/2138033918 wait complete. 2023-08-25T03:35:22.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.689+0000 7fc65b297700 1 Processor -- start 2023-08-25T03:35:22.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.690+0000 7fc65b297700 1 -- start start 2023-08-25T03:35:22.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.690+0000 7fc65b297700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc65406a1e0 0x7fc65415c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:22.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.690+0000 7fc65b297700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6540af180 0x7fc6540c5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:22.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.690+0000 7fc65b297700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6541607f0 0x7fc6540b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:22.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.690+0000 7fc65b297700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540c6e60 con 0x7fc6540af180 2023-08-25T03:35:22.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.690+0000 7fc65b297700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540c7420 con 0x7fc6541607f0 2023-08-25T03:35:22.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.690+0000 7fc65b297700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540c6120 con 0x7fc65406a1e0 2023-08-25T03:35:22.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.690+0000 7fc658832700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6540af180 0x7fc6540c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:22.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.690+0000 7fc658832700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6540af180 0x7fc6540c5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33566/0 (socket says 172.21.15.2:33566) 2023-08-25T03:35:22.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.690+0000 7fc658832700 1 -- 172.21.15.2:0/1923768991 learned_addr learned my addr 172.21.15.2:0/1923768991 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:22.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.691+0000 7fc659834700 1 --2- 172.21.15.2:0/1923768991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6541607f0 0x7fc6540b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:22.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.691+0000 7fc659033700 1 --2- 172.21.15.2:0/1923768991 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc65406a1e0 0x7fc65415c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:22.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.691+0000 7fc658832700 1 -- 172.21.15.2:0/1923768991 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc65406a1e0 msgr2=0x7fc65415c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:22.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.691+0000 7fc658832700 1 --2- 172.21.15.2:0/1923768991 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc65406a1e0 0x7fc65415c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:22.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.691+0000 7fc658832700 1 -- 172.21.15.2:0/1923768991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6541607f0 msgr2=0x7fc6540b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:22.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.691+0000 7fc658832700 1 --2- 172.21.15.2:0/1923768991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6541607f0 0x7fc6540b0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:22.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.691+0000 7fc658832700 1 -- 172.21.15.2:0/1923768991 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64c009b30 con 0x7fc6540af180 2023-08-25T03:35:22.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.691+0000 7fc658832700 1 --2- 172.21.15.2:0/1923768991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6540af180 0x7fc6540c5660 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc64400e840 tx=0x7fc64400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:22.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.691+0000 7fc649ffb700 1 -- 172.21.15.2:0/1923768991 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644013610 con 0x7fc6540af180 2023-08-25T03:35:22.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.691+0000 7fc649ffb700 1 -- 172.21.15.2:0/1923768991 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc644013770 con 0x7fc6540af180 2023-08-25T03:35:22.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.691+0000 7fc65b297700 1 -- 172.21.15.2:0/1923768991 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6540c6400 con 0x7fc6540af180 2023-08-25T03:35:22.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.691+0000 7fc649ffb700 1 -- 172.21.15.2:0/1923768991 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644013b20 con 0x7fc6540af180 2023-08-25T03:35:22.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.691+0000 7fc65b297700 1 -- 172.21.15.2:0/1923768991 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6540c4970 con 0x7fc6540af180 2023-08-25T03:35:22.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.692+0000 7fc65b297700 1 -- 172.21.15.2:0/1923768991 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6540c75b0 con 0x7fc6540af180 2023-08-25T03:35:22.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.693+0000 7fc649ffb700 1 -- 172.21.15.2:0/1923768991 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fc644013c80 con 0x7fc6540af180 2023-08-25T03:35:22.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.693+0000 7fc649ffb700 1 --2- 172.21.15.2:0/1923768991 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc63806a0e0 0x7fc63806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:22.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.693+0000 7fc649ffb700 1 -- 172.21.15.2:0/1923768991 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fc6440880b0 con 0x7fc6540af180 2023-08-25T03:35:22.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.693+0000 7fc659033700 1 --2- 172.21.15.2:0/1923768991 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc63806a0e0 0x7fc63806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:22.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.694+0000 7fc659033700 1 --2- 172.21.15.2:0/1923768991 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc63806a0e0 0x7fc63806c5a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc650007fd0 tx=0x7fc650007ec0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:22.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.698+0000 7fc649ffb700 1 -- 172.21.15.2:0/1923768991 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fc644057240 con 0x7fc6540af180 2023-08-25T03:35:22.896 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.894+0000 7fc65b297700 1 -- 172.21.15.2:0/1923768991 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc6540746d0 con 0x7fc6540af180 2023-08-25T03:35:22.897 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.895+0000 7fc649ffb700 1 -- 172.21.15.2:0/1923768991 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v37) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc644056dd0 con 0x7fc6540af180 2023-08-25T03:35:22.897 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 8 2023-08-25T03:35:22.899 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.897+0000 7fc65b297700 1 -- 172.21.15.2:0/1923768991 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc63806a0e0 msgr2=0x7fc63806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:22.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.897+0000 7fc65b297700 1 --2- 172.21.15.2:0/1923768991 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc63806a0e0 0x7fc63806c5a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc650007fd0 tx=0x7fc650007ec0 comp rx=0 tx=0).stop 2023-08-25T03:35:22.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.897+0000 7fc65b297700 1 -- 172.21.15.2:0/1923768991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6540af180 msgr2=0x7fc6540c5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:22.900 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.897+0000 7fc65b297700 1 --2- 172.21.15.2:0/1923768991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6540af180 0x7fc6540c5660 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc64400e840 tx=0x7fc64400eb50 comp rx=0 tx=0).stop 2023-08-25T03:35:22.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.897+0000 7fc65b297700 1 -- 172.21.15.2:0/1923768991 shutdown_connections 2023-08-25T03:35:22.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.897+0000 7fc65b297700 1 --2- 172.21.15.2:0/1923768991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6541607f0 0x7fc6540b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:22.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.897+0000 7fc65b297700 1 --2- 172.21.15.2:0/1923768991 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc65406a1e0 0x7fc65415c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:22.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.897+0000 7fc65b297700 1 --2- 172.21.15.2:0/1923768991 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fc63806a0e0 0x7fc63806c5a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:22.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.897+0000 7fc65b297700 1 --2- 172.21.15.2:0/1923768991 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fc6540af180 0x7fc6540c5660 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:22.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.897+0000 7fc65b297700 1 -- 172.21.15.2:0/1923768991 >> 172.21.15.2:0/1923768991 conn(0x7fc6540cfd90 msgr2=0x7fc6540cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:22.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.897+0000 7fc65b297700 1 -- 172.21.15.2:0/1923768991 shutdown_connections 2023-08-25T03:35:22.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:22.897+0000 7fc65b297700 1 -- 172.21.15.2:0/1923768991 wait complete. 2023-08-25T03:35:22.912 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-08-25T03:35:23.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.039+0000 7f9431737700 1 Processor -- start 2023-08-25T03:35:23.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.039+0000 7f9431737700 1 -- start start 2023-08-25T03:35:23.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.040+0000 7f9431737700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f942c1607b0 0x7f942c0cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:23.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.040+0000 7f9431737700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f942c0d2b20 0x7f942c0ce300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:23.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.040+0000 7f9431737700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f942c160b90 con 0x7f942c0d1510 2023-08-25T03:35:23.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.040+0000 7f9431737700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f942c0d1a80 con 0x7f942c1607b0 2023-08-25T03:35:23.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.040+0000 7f9431737700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942c0d1d60 con 0x7f942c0d2b20 2023-08-25T03:35:23.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.040+0000 7f942affd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f942c1607b0 0x7f942c0cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:23.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.040+0000 7f942a7fc700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f942c0d1510 0x7f942c06c260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48470/0 (socket says 172.21.15.2:48470) 2023-08-25T03:35:23.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.040+0000 7f942affd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f942c1607b0 0x7f942c0cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33572/0 (socket says 172.21.15.2:33572) 2023-08-25T03:35:23.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.040+0000 7f942a7fc700 1 -- 172.21.15.2:0/710805776 learned_addr learned my addr 172.21.15.2:0/710805776 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:23.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.040+0000 7f942b7fe700 1 --2- 172.21.15.2:0/710805776 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f942c0d2b20 0x7f942c0ce300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:23.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.040+0000 7f942affd700 1 -- 172.21.15.2:0/710805776 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f942c0d2b20 msgr2=0x7f942c0ce300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:23.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.040+0000 7f942affd700 1 --2- 172.21.15.2:0/710805776 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f942c0d2b20 0x7f942c0ce300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:23.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.040+0000 7f942affd700 1 -- 172.21.15.2:0/710805776 >> v1:172.21.15.2:6789/0 conn(0x7f942c0d1510 legacy=0x7f942c06c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:23.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.041+0000 7f942affd700 1 -- 172.21.15.2:0/710805776 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c0d2040 con 0x7f942c1607b0 2023-08-25T03:35:23.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.041+0000 7f9429ffb700 1 -- 172.21.15.2:0/710805776 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3506067263 0 0) 0x7f942c160b90 con 0x7f942c0d1510 2023-08-25T03:35:23.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.041+0000 7f942b7fe700 1 --2- 172.21.15.2:0/710805776 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f942c0d2b20 0x7f942c0ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:35:23.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.041+0000 7f942affd700 1 --2- 172.21.15.2:0/710805776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f942c1607b0 0x7f942c0cddc0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9420009840 tx=0x7f9420009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abbeb806bd4f42bc server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:23.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.041+0000 7f9429ffb700 1 -- 172.21.15.2:0/710805776 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9420011830 con 0x7f942c1607b0 2023-08-25T03:35:23.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.041+0000 7f9429ffb700 1 -- 172.21.15.2:0/710805776 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9420011990 con 0x7f942c1607b0 2023-08-25T03:35:23.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.041+0000 7f9429ffb700 1 -- 172.21.15.2:0/710805776 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9420011d20 con 0x7f942c1607b0 2023-08-25T03:35:23.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.041+0000 7f9431737700 1 -- 172.21.15.2:0/710805776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f942c1607b0 msgr2=0x7f942c0cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:23.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.041+0000 7f9431737700 1 --2- 172.21.15.2:0/710805776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f942c1607b0 0x7f942c0cddc0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9420009840 tx=0x7f9420009b50 comp rx=0 tx=0).stop 2023-08-25T03:35:23.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.041+0000 7f9431737700 1 -- 172.21.15.2:0/710805776 shutdown_connections 2023-08-25T03:35:23.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.041+0000 7f9431737700 1 --2- 172.21.15.2:0/710805776 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f942c0d2b20 0x7f942c0ce300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:23.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.041+0000 7f9431737700 1 --2- 172.21.15.2:0/710805776 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f942c1607b0 0x7f942c0cddc0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:23.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.041+0000 7f9431737700 1 -- 172.21.15.2:0/710805776 >> 172.21.15.2:0/710805776 conn(0x7f942c0cfb70 msgr2=0x7f942c0cff80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:23.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.041+0000 7f9431737700 1 -- 172.21.15.2:0/710805776 shutdown_connections 2023-08-25T03:35:23.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.041+0000 7f9431737700 1 -- 172.21.15.2:0/710805776 wait complete. 2023-08-25T03:35:23.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.043+0000 7f9431737700 1 Processor -- start 2023-08-25T03:35:23.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.043+0000 7f9431737700 1 -- start start 2023-08-25T03:35:23.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.044+0000 7f9431737700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f942c0d1510 0x7f942c1f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:23.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.044+0000 7f9431737700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f942c0d2b20 0x7f942c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:23.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.044+0000 7f9431737700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f942c1607b0 0x7f942c0b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:23.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.044+0000 7f9431737700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c0c9380 con 0x7f942c0d2b20 2023-08-25T03:35:23.050 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.044+0000 7f9431737700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f942c0c87b0 con 0x7f942c0d1510 2023-08-25T03:35:23.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.044+0000 7f9431737700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f942c0c8a50 con 0x7f942c1607b0 2023-08-25T03:35:23.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.044+0000 7f942a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f942c0d2b20 0x7f942c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:23.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.044+0000 7f942a7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f942c0d2b20 0x7f942c0b4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33580/0 (socket says 172.21.15.2:33580) 2023-08-25T03:35:23.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.044+0000 7f942a7fc700 1 -- 172.21.15.2:0/1748258080 learned_addr learned my addr 172.21.15.2:0/1748258080 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:23.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.044+0000 7f942a7fc700 1 -- 172.21.15.2:0/1748258080 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f942c1607b0 msgr2=0x7f942c0b52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:23.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.044+0000 7f942b7fe700 1 --2- 172.21.15.2:0/1748258080 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f942c1607b0 0x7f942c0b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:23.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.044+0000 7f942affd700 1 --2- 172.21.15.2:0/1748258080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f942c0d1510 0x7f942c1f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:23.051 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.044+0000 7f942a7fc700 1 --2- 172.21.15.2:0/1748258080 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f942c1607b0 0x7f942c0b52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:23.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.044+0000 7f942a7fc700 1 -- 172.21.15.2:0/1748258080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f942c0d1510 msgr2=0x7f942c1f1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:23.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.044+0000 7f942a7fc700 1 --2- 172.21.15.2:0/1748258080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f942c0d1510 0x7f942c1f1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:23.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.044+0000 7f942a7fc700 1 -- 172.21.15.2:0/1748258080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94200094a0 con 0x7f942c0d2b20 2023-08-25T03:35:23.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.044+0000 7f942a7fc700 1 --2- 172.21.15.2:0/1748258080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f942c0d2b20 0x7f942c0b4d90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f941c00c550 tx=0x7f941c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:23.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.044+0000 7f94137fe700 1 -- 172.21.15.2:0/1748258080 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c004c80 con 0x7f942c0d2b20 2023-08-25T03:35:23.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.044+0000 7f94137fe700 1 -- 172.21.15.2:0/1748258080 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941c004e00 con 0x7f942c0d2b20 2023-08-25T03:35:23.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.044+0000 7f9431737700 1 -- 172.21.15.2:0/1748258080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f942c0c8d30 con 0x7f942c0d2b20 2023-08-25T03:35:23.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.044+0000 7f94137fe700 1 -- 172.21.15.2:0/1748258080 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c007310 con 0x7f942c0d2b20 2023-08-25T03:35:23.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.044+0000 7f9431737700 1 -- 172.21.15.2:0/1748258080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942c1f2340 con 0x7f942c0d2b20 2023-08-25T03:35:23.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.045+0000 7f9431737700 1 -- 172.21.15.2:0/1748258080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f942c0c9530 con 0x7f942c0d2b20 2023-08-25T03:35:23.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.046+0000 7f94137fe700 1 -- 172.21.15.2:0/1748258080 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f941c007470 con 0x7f942c0d2b20 2023-08-25T03:35:23.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.046+0000 7f94137fe700 1 --2- 172.21.15.2:0/1748258080 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f941806a0e0 0x7f941806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:23.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.046+0000 7f94137fe700 1 -- 172.21.15.2:0/1748258080 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f941c0881c0 con 0x7f942c0d2b20 2023-08-25T03:35:23.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.046+0000 7f942affd700 1 --2- 172.21.15.2:0/1748258080 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f941806a0e0 0x7f941806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:23.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.047+0000 7f942affd700 1 --2- 172.21.15.2:0/1748258080 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f941806a0e0 0x7f941806c5a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f94200090b0 tx=0x7f9420008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:23.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.051+0000 7f94137fe700 1 -- 172.21.15.2:0/1748258080 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f942c0c9530 con 0x7f942c0d2b20 2023-08-25T03:35:23.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.250+0000 7f9431737700 1 -- 172.21.15.2:0/1748258080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f942c0c9530 con 0x7f942c0d2b20 2023-08-25T03:35:23.252 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.250+0000 7f94137fe700 1 -- 172.21.15.2:0/1748258080 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v37) v1 ==== 104+0+11 (secure 0 0 0) 0x7f942c0c9530 con 0x7f942c0d2b20 2023-08-25T03:35:23.253 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 32 2023-08-25T03:35:23.262 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.252+0000 7f9431737700 1 -- 172.21.15.2:0/1748258080 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f941806a0e0 msgr2=0x7f941806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:23.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.252+0000 7f9431737700 1 --2- 172.21.15.2:0/1748258080 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f941806a0e0 0x7f941806c5a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f94200090b0 tx=0x7f9420008030 comp rx=0 tx=0).stop 2023-08-25T03:35:23.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.252+0000 7f9431737700 1 -- 172.21.15.2:0/1748258080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f942c0d2b20 msgr2=0x7f942c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:23.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.252+0000 7f9431737700 1 --2- 172.21.15.2:0/1748258080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f942c0d2b20 0x7f942c0b4d90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f941c00c550 tx=0x7f941c00c910 comp rx=0 tx=0).stop 2023-08-25T03:35:23.263 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.252+0000 7f9431737700 1 -- 172.21.15.2:0/1748258080 shutdown_connections 2023-08-25T03:35:23.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.252+0000 7f9431737700 1 --2- 172.21.15.2:0/1748258080 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f942c1607b0 0x7f942c0b52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:23.264 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.252+0000 7f9431737700 1 --2- 172.21.15.2:0/1748258080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f942c0d1510 0x7f942c1f1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:23.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.252+0000 7f9431737700 1 --2- 172.21.15.2:0/1748258080 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f941806a0e0 0x7f941806c5a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:23.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.252+0000 7f9431737700 1 --2- 172.21.15.2:0/1748258080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f942c0d2b20 0x7f942c0b4d90 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:23.265 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.252+0000 7f9431737700 1 -- 172.21.15.2:0/1748258080 >> 172.21.15.2:0/1748258080 conn(0x7f942c0cfb70 msgr2=0x7f942c15eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:23.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.253+0000 7f9431737700 1 -- 172.21.15.2:0/1748258080 shutdown_connections 2023-08-25T03:35:23.266 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.253+0000 7f9431737700 1 -- 172.21.15.2:0/1748258080 wait complete. 2023-08-25T03:35:23.267 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-08-25T03:35:23.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.395+0000 7f6962892700 1 Processor -- start 2023-08-25T03:35:23.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.395+0000 7f6962892700 1 -- start start 2023-08-25T03:35:23.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.395+0000 7f6962892700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f695c15d360 0x7f695c06b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:23.399 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.395+0000 7f6962892700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f695c06e570 0x7f695c06ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:23.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.395+0000 7f6962892700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f695c0d1b40 con 0x7f695c0cfd80 2023-08-25T03:35:23.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.395+0000 7f6962892700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f695c06e0e0 con 0x7f695c15d360 2023-08-25T03:35:23.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.395+0000 7f6962892700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f695c0d0620 con 0x7f695c06e570 2023-08-25T03:35:23.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.396+0000 7f695bfff700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f695c15d360 0x7f695c06b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:23.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.396+0000 7f695bfff700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f695c15d360 0x7f695c06b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33586/0 (socket says 172.21.15.2:33586) 2023-08-25T03:35:23.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.396+0000 7f695bfff700 1 -- 172.21.15.2:0/3667754612 learned_addr learned my addr 172.21.15.2:0/3667754612 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:23.400 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.396+0000 7f6960e2f700 1 --2- 172.21.15.2:0/3667754612 >> v2:172.21.15.31:3300/0 conn(0x7f695c06e570 0x7f695c06ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:23.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.396+0000 7f695bfff700 1 -- 172.21.15.2:0/3667754612 >> v2:172.21.15.31:3300/0 conn(0x7f695c06e570 msgr2=0x7f695c06ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:23.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.396+0000 7f695bfff700 1 --2- 172.21.15.2:0/3667754612 >> v2:172.21.15.31:3300/0 conn(0x7f695c06e570 0x7f695c06ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:23.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.396+0000 7f695bfff700 1 -- 172.21.15.2:0/3667754612 >> v1:172.21.15.31:6789/0 conn(0x7f695c0cfd80 legacy=0x7f695c06ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:23.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.396+0000 7f695bfff700 1 -- 172.21.15.2:0/3667754612 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f695c0d0340 con 0x7f695c15d360 2023-08-25T03:35:23.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.396+0000 7f695bfff700 1 --2- 172.21.15.2:0/3667754612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f695c15d360 0x7f695c06b500 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f695c071aa0 tx=0x7f694c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe446d1483b787d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:23.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.396+0000 7f695affd700 1 -- 172.21.15.2:0/3667754612 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694c005e30 con 0x7f695c15d360 2023-08-25T03:35:23.401 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.396+0000 7f695affd700 1 -- 172.21.15.2:0/3667754612 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f694c004030 con 0x7f695c15d360 2023-08-25T03:35:23.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.396+0000 7f695affd700 1 -- 172.21.15.2:0/3667754612 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694c0043e0 con 0x7f695c15d360 2023-08-25T03:35:23.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.397+0000 7f6962892700 1 -- 172.21.15.2:0/3667754612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f695c15d360 msgr2=0x7f695c06b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:23.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.397+0000 7f6962892700 1 --2- 172.21.15.2:0/3667754612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f695c15d360 0x7f695c06b500 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f695c071aa0 tx=0x7f694c009bd0 comp rx=0 tx=0).stop 2023-08-25T03:35:23.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.397+0000 7f6962892700 1 -- 172.21.15.2:0/3667754612 shutdown_connections 2023-08-25T03:35:23.402 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.397+0000 7f6962892700 1 --2- 172.21.15.2:0/3667754612 >> v2:172.21.15.31:3300/0 conn(0x7f695c06e570 0x7f695c06ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:23.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.397+0000 7f6962892700 1 --2- 172.21.15.2:0/3667754612 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f695c15d360 0x7f695c06b500 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:23.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.397+0000 7f6962892700 1 -- 172.21.15.2:0/3667754612 >> 172.21.15.2:0/3667754612 conn(0x7f695c09a200 msgr2=0x7f695c0d0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:23.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.397+0000 7f6962892700 1 -- 172.21.15.2:0/3667754612 shutdown_connections 2023-08-25T03:35:23.403 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.397+0000 7f6962892700 1 -- 172.21.15.2:0/3667754612 wait complete. 2023-08-25T03:35:23.405 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.398+0000 7f6962892700 1 Processor -- start 2023-08-25T03:35:23.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.398+0000 7f6962892700 1 -- start start 2023-08-25T03:35:23.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.399+0000 7f6962892700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f695c06e570 0x7f695c15df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:23.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.399+0000 7f6962892700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f695c0cfd80 0x7f695c1f5e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:23.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.399+0000 7f6962892700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695c15d360 0x7f695c1f6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:23.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.399+0000 7f6962892700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695c15e890 con 0x7f695c06e570 2023-08-25T03:35:23.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.399+0000 7f6962892700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f695c15eb70 con 0x7f695c15d360 2023-08-25T03:35:23.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.399+0000 7f6962892700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f695c15ee50 con 0x7f695c0cfd80 2023-08-25T03:35:23.406 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.399+0000 7f695bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f695c06e570 0x7f695c15df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:23.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.399+0000 7f695bfff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f695c06e570 0x7f695c15df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33592/0 (socket says 172.21.15.2:33592) 2023-08-25T03:35:23.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.399+0000 7f695bfff700 1 -- 172.21.15.2:0/44926516 learned_addr learned my addr 172.21.15.2:0/44926516 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:23.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.399+0000 7f6960e2f700 1 --2- 172.21.15.2:0/44926516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695c15d360 0x7f695c1f6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:23.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.399+0000 7f695b7fe700 1 --2- 172.21.15.2:0/44926516 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f695c0cfd80 0x7f695c1f5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:23.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.399+0000 7f695bfff700 1 -- 172.21.15.2:0/44926516 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f695c0cfd80 msgr2=0x7f695c1f5e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:23.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.399+0000 7f695bfff700 1 --2- 172.21.15.2:0/44926516 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f695c0cfd80 0x7f695c1f5e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:23.407 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.399+0000 7f695bfff700 1 -- 172.21.15.2:0/44926516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695c15d360 msgr2=0x7f695c1f6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:23.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.399+0000 7f695bfff700 1 --2- 172.21.15.2:0/44926516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695c15d360 0x7f695c1f6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:23.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.399+0000 7f695bfff700 1 -- 172.21.15.2:0/44926516 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f694c009500 con 0x7f695c06e570 2023-08-25T03:35:23.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.400+0000 7f695bfff700 1 --2- 172.21.15.2:0/44926516 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f695c06e570 0x7f695c15df60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f694c004000 tx=0x7f694c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:23.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.400+0000 7f6958ff9700 1 -- 172.21.15.2:0/44926516 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694c003d60 con 0x7f695c06e570 2023-08-25T03:35:23.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.400+0000 7f6958ff9700 1 -- 172.21.15.2:0/44926516 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f694c01d430 con 0x7f695c06e570 2023-08-25T03:35:23.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.400+0000 7f6962892700 1 -- 172.21.15.2:0/44926516 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f695c15f130 con 0x7f695c06e570 2023-08-25T03:35:23.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.400+0000 7f6958ff9700 1 -- 172.21.15.2:0/44926516 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f694c01d7e0 con 0x7f695c06e570 2023-08-25T03:35:23.408 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.400+0000 7f6962892700 1 -- 172.21.15.2:0/44926516 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f695c15f930 con 0x7f695c06e570 2023-08-25T03:35:23.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.401+0000 7f6962892700 1 -- 172.21.15.2:0/44926516 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f695c0746d0 con 0x7f695c06e570 2023-08-25T03:35:23.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.401+0000 7f6958ff9700 1 -- 172.21.15.2:0/44926516 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f694c01d940 con 0x7f695c06e570 2023-08-25T03:35:23.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.401+0000 7f6958ff9700 1 --2- 172.21.15.2:0/44926516 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f694806a0e0 0x7f694806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:23.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.402+0000 7f6958ff9700 1 -- 172.21.15.2:0/44926516 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f694c089840 con 0x7f695c06e570 2023-08-25T03:35:23.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.402+0000 7f695b7fe700 1 --2- 172.21.15.2:0/44926516 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f694806a0e0 0x7f694806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:23.409 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.402+0000 7f695b7fe700 1 --2- 172.21.15.2:0/44926516 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f694806a0e0 0x7f694806c5a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6950000f60 tx=0x7f6950009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:23.410 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.406+0000 7f6958ff9700 1 -- 172.21.15.2:0/44926516 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f694c054fc0 con 0x7f695c06e570 2023-08-25T03:35:23.612 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.611+0000 7f6962892700 1 -- 172.21.15.2:0/44926516 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f695c0ce620 con 0x7f695c06e570 2023-08-25T03:35:23.613 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.611+0000 7f6958ff9700 1 -- 172.21.15.2:0/44926516 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v37) v1 ==== 100+0+11 (secure 0 0 0) 0x7f694c0585e0 con 0x7f695c06e570 2023-08-25T03:35:23.613 INFO:teuthology.orchestra.run.smithi002.stdout:pg_num: 64 2023-08-25T03:35:23.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.613+0000 7f6962892700 1 -- 172.21.15.2:0/44926516 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f694806a0e0 msgr2=0x7f694806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:23.615 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.613+0000 7f6962892700 1 --2- 172.21.15.2:0/44926516 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f694806a0e0 0x7f694806c5a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6950000f60 tx=0x7f6950009040 comp rx=0 tx=0).stop 2023-08-25T03:35:23.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.613+0000 7f6962892700 1 -- 172.21.15.2:0/44926516 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f695c06e570 msgr2=0x7f695c15df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:23.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.613+0000 7f6962892700 1 --2- 172.21.15.2:0/44926516 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f695c06e570 0x7f695c15df60 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f694c004000 tx=0x7f694c004750 comp rx=0 tx=0).stop 2023-08-25T03:35:23.616 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.613+0000 7f6962892700 1 -- 172.21.15.2:0/44926516 shutdown_connections 2023-08-25T03:35:23.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.613+0000 7f6962892700 1 --2- 172.21.15.2:0/44926516 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f694806a0e0 0x7f694806c5a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:23.617 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.613+0000 7f6962892700 1 --2- 172.21.15.2:0/44926516 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f695c06e570 0x7f695c15df60 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:23.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.613+0000 7f6962892700 1 --2- 172.21.15.2:0/44926516 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f695c15d360 0x7f695c1f6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:23.618 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.613+0000 7f6962892700 1 --2- 172.21.15.2:0/44926516 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f695c0cfd80 0x7f695c1f5e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:23.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.613+0000 7f6962892700 1 -- 172.21.15.2:0/44926516 >> 172.21.15.2:0/44926516 conn(0x7f695c09a200 msgr2=0x7f695c0aea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:23.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.613+0000 7f6962892700 1 -- 172.21.15.2:0/44926516 shutdown_connections 2023-08-25T03:35:23.619 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.613+0000 7f6962892700 1 -- 172.21.15.2:0/44926516 wait complete. 2023-08-25T03:35:23.628 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T03:35:23.752 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.750+0000 7fb6168b8700 1 Processor -- start 2023-08-25T03:35:23.753 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.751+0000 7fb6168b8700 1 -- start start 2023-08-25T03:35:23.754 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.751+0000 7fb6168b8700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb61015e6e0 0x7fb6100cd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:23.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.751+0000 7fb6168b8700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fb6100d16d0 0x7fb6100cde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:23.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.751+0000 7fb6168b8700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6100cf5d0 con 0x7fb61015eac0 2023-08-25T03:35:23.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.751+0000 7fb6168b8700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6100b3f40 con 0x7fb6100d16d0 2023-08-25T03:35:23.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.751+0000 7fb6168b8700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6100cf870 con 0x7fb61015e6e0 2023-08-25T03:35:23.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.751+0000 7fb60f7fe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fb6100d16d0 0x7fb6100cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:23.755 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.751+0000 7fb614e55700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fb61015eac0 0x7fb6100b4360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48502/0 (socket says 172.21.15.2:48502) 2023-08-25T03:35:23.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.752+0000 7fb60f7fe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fb6100d16d0 0x7fb6100cde40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33600/0 (socket says 172.21.15.2:33600) 2023-08-25T03:35:23.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.752+0000 7fb614e55700 1 -- 172.21.15.2:0/897163780 learned_addr learned my addr 172.21.15.2:0/897163780 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:23.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.752+0000 7fb60ffff700 1 --2- 172.21.15.2:0/897163780 >> v2:172.21.15.31:3300/0 conn(0x7fb61015e6e0 0x7fb6100cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:23.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.752+0000 7fb60f7fe700 1 -- 172.21.15.2:0/897163780 >> v2:172.21.15.31:3300/0 conn(0x7fb61015e6e0 msgr2=0x7fb6100cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:23.756 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.752+0000 7fb60f7fe700 1 --2- 172.21.15.2:0/897163780 >> v2:172.21.15.31:3300/0 conn(0x7fb61015e6e0 0x7fb6100cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:23.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.752+0000 7fb60f7fe700 1 -- 172.21.15.2:0/897163780 >> v1:172.21.15.2:6789/0 conn(0x7fb61015eac0 legacy=0x7fb6100b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:23.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.752+0000 7fb60f7fe700 1 -- 172.21.15.2:0/897163780 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6100cfb50 con 0x7fb6100d16d0 2023-08-25T03:35:23.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.752+0000 7fb60effd700 1 -- 172.21.15.2:0/897163780 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2818842387 0 0) 0x7fb6100cf5d0 con 0x7fb61015eac0 2023-08-25T03:35:23.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.752+0000 7fb60ffff700 1 --2- 172.21.15.2:0/897163780 >> v2:172.21.15.31:3300/0 conn(0x7fb61015e6e0 0x7fb6100cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:35:23.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.752+0000 7fb60f7fe700 1 --2- 172.21.15.2:0/897163780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6100d16d0 0x7fb6100cde40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009840 tx=0x7fb5f8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ccff5df5f098c3 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:23.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.752+0000 7fb60effd700 1 -- 172.21.15.2:0/897163780 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8011830 con 0x7fb6100d16d0 2023-08-25T03:35:23.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.752+0000 7fb60effd700 1 -- 172.21.15.2:0/897163780 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f8011990 con 0x7fb6100d16d0 2023-08-25T03:35:23.757 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.752+0000 7fb60effd700 1 -- 172.21.15.2:0/897163780 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8011d20 con 0x7fb6100d16d0 2023-08-25T03:35:23.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.753+0000 7fb6168b8700 1 -- 172.21.15.2:0/897163780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6100d16d0 msgr2=0x7fb6100cde40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:23.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.753+0000 7fb6168b8700 1 --2- 172.21.15.2:0/897163780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6100d16d0 0x7fb6100cde40 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009840 tx=0x7fb5f8009b50 comp rx=0 tx=0).stop 2023-08-25T03:35:23.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.753+0000 7fb6168b8700 1 -- 172.21.15.2:0/897163780 shutdown_connections 2023-08-25T03:35:23.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.753+0000 7fb6168b8700 1 --2- 172.21.15.2:0/897163780 >> v2:172.21.15.31:3300/0 conn(0x7fb61015e6e0 0x7fb6100cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:23.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.753+0000 7fb6168b8700 1 --2- 172.21.15.2:0/897163780 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb6100d16d0 0x7fb6100cde40 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:23.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.753+0000 7fb6168b8700 1 -- 172.21.15.2:0/897163780 >> 172.21.15.2:0/897163780 conn(0x7fb61009a2b0 msgr2=0x7fb6100d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:23.758 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.753+0000 7fb6168b8700 1 -- 172.21.15.2:0/897163780 shutdown_connections 2023-08-25T03:35:23.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.753+0000 7fb6168b8700 1 -- 172.21.15.2:0/897163780 wait complete. 2023-08-25T03:35:23.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.754+0000 7fb6168b8700 1 Processor -- start 2023-08-25T03:35:23.759 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.754+0000 7fb6168b8700 1 -- start start 2023-08-25T03:35:23.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.755+0000 7fb6168b8700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb61015e6e0 0x7fb61014b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:23.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.755+0000 7fb6168b8700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb61015eac0 0x7fb61006e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:23.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.755+0000 7fb6168b8700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb61006ee00 0x7fb6100b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:23.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.755+0000 7fb6168b8700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61006f2b0 con 0x7fb61015e6e0 2023-08-25T03:35:23.760 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.755+0000 7fb6168b8700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb61015abc0 con 0x7fb61015eac0 2023-08-25T03:35:23.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.755+0000 7fb6168b8700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6100b1b10 con 0x7fb61006ee00 2023-08-25T03:35:23.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.755+0000 7fb60ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb61015e6e0 0x7fb61014b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:23.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.755+0000 7fb60ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb61015e6e0 0x7fb61014b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33606/0 (socket says 172.21.15.2:33606) 2023-08-25T03:35:23.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.755+0000 7fb60ffff700 1 -- 172.21.15.2:0/3826381942 learned_addr learned my addr 172.21.15.2:0/3826381942 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:23.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.755+0000 7fb60f7fe700 1 --2- 172.21.15.2:0/3826381942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb61015eac0 0x7fb61006e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:23.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.755+0000 7fb614e55700 1 --2- 172.21.15.2:0/3826381942 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb61006ee00 0x7fb6100b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:23.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.755+0000 7fb60ffff700 1 -- 172.21.15.2:0/3826381942 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb61006ee00 msgr2=0x7fb6100b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:23.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.755+0000 7fb60ffff700 1 --2- 172.21.15.2:0/3826381942 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb61006ee00 0x7fb6100b04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:23.763 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.755+0000 7fb60ffff700 1 -- 172.21.15.2:0/3826381942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb61015eac0 msgr2=0x7fb61006e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:23.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.755+0000 7fb60ffff700 1 --2- 172.21.15.2:0/3826381942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb61015eac0 0x7fb61006e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:23.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.755+0000 7fb60ffff700 1 -- 172.21.15.2:0/3826381942 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f80094a0 con 0x7fb61015e6e0 2023-08-25T03:35:23.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.756+0000 7fb60ffff700 1 --2- 172.21.15.2:0/3826381942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb61015e6e0 0x7fb61014b390 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb60000c7f0 tx=0x7fb60000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:23.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.756+0000 7fb60cff9700 1 -- 172.21.15.2:0/3826381942 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb600005c20 con 0x7fb61015e6e0 2023-08-25T03:35:23.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.756+0000 7fb60cff9700 1 -- 172.21.15.2:0/3826381942 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb600005d80 con 0x7fb61015e6e0 2023-08-25T03:35:23.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.756+0000 7fb6168b8700 1 -- 172.21.15.2:0/3826381942 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb61014b8d0 con 0x7fb61015e6e0 2023-08-25T03:35:23.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.756+0000 7fb60cff9700 1 -- 172.21.15.2:0/3826381942 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb600011820 con 0x7fb61015e6e0 2023-08-25T03:35:23.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.756+0000 7fb6168b8700 1 -- 172.21.15.2:0/3826381942 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6100b1550 con 0x7fb61015e6e0 2023-08-25T03:35:23.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.757+0000 7fb6168b8700 1 -- 172.21.15.2:0/3826381942 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6100746d0 con 0x7fb61015e6e0 2023-08-25T03:35:23.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.757+0000 7fb60cff9700 1 -- 172.21.15.2:0/3826381942 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb600011980 con 0x7fb61015e6e0 2023-08-25T03:35:23.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.758+0000 7fb60cff9700 1 --2- 172.21.15.2:0/3826381942 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb5f006a0e0 0x7fb5f006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:23.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.758+0000 7fb60cff9700 1 -- 172.21.15.2:0/3826381942 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb600088760 con 0x7fb61015e6e0 2023-08-25T03:35:23.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.758+0000 7fb60f7fe700 1 --2- 172.21.15.2:0/3826381942 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb5f006a0e0 0x7fb5f006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:23.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.758+0000 7fb60f7fe700 1 --2- 172.21.15.2:0/3826381942 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb5f006a0e0 0x7fb5f006c5a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009840 tx=0x7fb5f8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:23.767 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:23.762+0000 7fb60cff9700 1 -- 172.21.15.2:0/3826381942 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb600053e50 con 0x7fb61015e6e0 2023-08-25T03:35:24.009 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.007+0000 7fb6168b8700 1 -- 172.21.15.2:0/3826381942 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6100c7de0 con 0x7fb61015e6e0 2023-08-25T03:35:24.010 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.008+0000 7fb60cff9700 1 -- 172.21.15.2:0/3826381942 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4869 (secure 0 0 0) 0x7fb600057470 con 0x7fb61015e6e0 2023-08-25T03:35:24.010 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:35:24.010 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4751,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/3802947908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3802947908},{"type":"v1","addr":"172.21.15.31:6834","nonce":3802947908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1470640897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6835","nonce":1470640897},{"type":"v1","addr":"172.21.15.2:6837","nonce":1470640897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6836/3706466654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3706466654},{"type":"v1","addr":"172.21.15.2:6836","nonce":3706466654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T03:35:07.333851+0000","modified":"2023-08-25T03:35:10.931788+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5229":{"gid":5229,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1140461626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1140461626},{"type":"v1","addr":"172.21.15.31:6835","nonce":1140461626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-25T03:35:24.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.010+0000 7fb6168b8700 1 -- 172.21.15.2:0/3826381942 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb5f006a0e0 msgr2=0x7fb5f006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:24.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.010+0000 7fb6168b8700 1 --2- 172.21.15.2:0/3826381942 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb5f006a0e0 0x7fb5f006c5a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fb5f8009840 tx=0x7fb5f8008030 comp rx=0 tx=0).stop 2023-08-25T03:35:24.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.010+0000 7fb6168b8700 1 -- 172.21.15.2:0/3826381942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb61015e6e0 msgr2=0x7fb61014b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:24.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.010+0000 7fb6168b8700 1 --2- 172.21.15.2:0/3826381942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb61015e6e0 0x7fb61014b390 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb60000c7f0 tx=0x7fb60000cb00 comp rx=0 tx=0).stop 2023-08-25T03:35:24.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.010+0000 7fb6168b8700 1 -- 172.21.15.2:0/3826381942 shutdown_connections 2023-08-25T03:35:24.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.010+0000 7fb6168b8700 1 --2- 172.21.15.2:0/3826381942 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb5f006a0e0 0x7fb5f006c5a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.010+0000 7fb6168b8700 1 --2- 172.21.15.2:0/3826381942 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb61015e6e0 0x7fb61014b390 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.010+0000 7fb6168b8700 1 --2- 172.21.15.2:0/3826381942 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb61006ee00 0x7fb6100b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.010+0000 7fb6168b8700 1 --2- 172.21.15.2:0/3826381942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb61015eac0 0x7fb61006e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.010+0000 7fb6168b8700 1 -- 172.21.15.2:0/3826381942 >> 172.21.15.2:0/3826381942 conn(0x7fb61009a2b0 msgr2=0x7fb61006e040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:24.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.010+0000 7fb6168b8700 1 -- 172.21.15.2:0/3826381942 shutdown_connections 2023-08-25T03:35:24.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.010+0000 7fb6168b8700 1 -- 172.21.15.2:0/3826381942 wait complete. 2023-08-25T03:35:24.015 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2023-08-25T03:35:24.024 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:35:24.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.147+0000 7f925d9cb700 1 Processor -- start 2023-08-25T03:35:24.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.147+0000 7f925d9cb700 1 -- start start 2023-08-25T03:35:24.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.148+0000 7f925d9cb700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f92580ae250 0x7f925806c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:24.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.148+0000 7f925d9cb700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f925806c560 con 0x7f9258070b20 2023-08-25T03:35:24.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.148+0000 7f925d9cb700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92580ccb30 con 0x7f92580ce000 2023-08-25T03:35:24.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.148+0000 7f925d9cb700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92580cce10 con 0x7f92580ae250 2023-08-25T03:35:24.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.148+0000 7f92567fc700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f9258070b20 0x7f92580cd8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48512/0 (socket says 172.21.15.2:48512) 2023-08-25T03:35:24.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.148+0000 7f92567fc700 1 -- 172.21.15.2:0/52472686 learned_addr learned my addr 172.21.15.2:0/52472686 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:24.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.148+0000 7f9256ffd700 1 --2- 172.21.15.2:0/52472686 >> v2:172.21.15.31:3300/0 conn(0x7f92580ae250 0x7f925806c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:24.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.148+0000 7f9255ffb700 1 -- 172.21.15.2:0/52472686 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2469432753 0 0) 0x7f925806c560 con 0x7f9258070b20 2023-08-25T03:35:24.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.148+0000 7f9255ffb700 1 -- 172.21.15.2:0/52472686 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92400035b0 con 0x7f9258070b20 2023-08-25T03:35:24.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.148+0000 7f9255ffb700 1 -- 172.21.15.2:0/52472686 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3325083737 0 0) 0x7f92400035b0 con 0x7f9258070b20 2023-08-25T03:35:24.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.148+0000 7f9255ffb700 1 -- 172.21.15.2:0/52472686 >> v2:172.21.15.31:3300/0 conn(0x7f92580ae250 msgr2=0x7f925806c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:24.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.148+0000 7f9255ffb700 1 --2- 172.21.15.2:0/52472686 >> v2:172.21.15.31:3300/0 conn(0x7f92580ae250 0x7f925806c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.148+0000 7f9255ffb700 1 -- 172.21.15.2:0/52472686 >> v1:172.21.15.31:6789/0 conn(0x7f92580ce000 legacy=0x7f92580c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:24.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.148+0000 7f9255ffb700 1 -- 172.21.15.2:0/52472686 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92580cd0f0 con 0x7f9258070b20 2023-08-25T03:35:24.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.148+0000 7f9255ffb700 1 -- 172.21.15.2:0/52472686 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f924c001670 con 0x7f9258070b20 2023-08-25T03:35:24.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.148+0000 7f9255ffb700 1 -- 172.21.15.2:0/52472686 >> v1:172.21.15.2:6789/0 conn(0x7f9258070b20 legacy=0x7f92580cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:24.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.148+0000 7f9255ffb700 1 --2- 172.21.15.2:0/52472686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9240003a00 0x7f9240003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:24.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.148+0000 7f9255ffb700 1 --2- 172.21.15.2:0/52472686 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9240004850 0x7f9240004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:24.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.148+0000 7f9255ffb700 1 --2- 172.21.15.2:0/52472686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9240005210 0x7f92400079f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:24.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.148+0000 7f9255ffb700 1 -- 172.21.15.2:0/52472686 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92580cce10 con 0x7f9240003a00 2023-08-25T03:35:24.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.148+0000 7f9255ffb700 1 -- 172.21.15.2:0/52472686 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9240008090 con 0x7f9240005210 2023-08-25T03:35:24.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.148+0000 7f9255ffb700 1 -- 172.21.15.2:0/52472686 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9240008250 con 0x7f9240004850 2023-08-25T03:35:24.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.149+0000 7f9256ffd700 1 --2- 172.21.15.2:0/52472686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9240003a00 0x7f9240003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:24.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.149+0000 7f92567fc700 1 --2- 172.21.15.2:0/52472686 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9240004850 0x7f9240004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:24.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.149+0000 7f92577fe700 1 --2- 172.21.15.2:0/52472686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9240005210 0x7f92400079f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:24.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.149+0000 7f9256ffd700 1 -- 172.21.15.2:0/52472686 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9240004850 msgr2=0x7f9240004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:24.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.149+0000 7f9256ffd700 1 --2- 172.21.15.2:0/52472686 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9240004850 0x7f9240004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.149+0000 7f9256ffd700 1 -- 172.21.15.2:0/52472686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9240005210 msgr2=0x7f92400079f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:24.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.149+0000 7f9256ffd700 1 --2- 172.21.15.2:0/52472686 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9240005210 0x7f92400079f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.149+0000 7f92577fe700 1 -- 172.21.15.2:0/52472686 reap_dead start 2023-08-25T03:35:24.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.149+0000 7f9256ffd700 1 -- 172.21.15.2:0/52472686 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92400035b0 con 0x7f9240003a00 2023-08-25T03:35:24.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.149+0000 7f9256ffd700 1 --2- 172.21.15.2:0/52472686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9240003a00 0x7f9240003e20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9248009b60 tx=0x7f9240005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4479ec8baf4b0fa9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:24.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.149+0000 7f9255ffb700 1 -- 172.21.15.2:0/52472686 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924800e6c0 con 0x7f9240003a00 2023-08-25T03:35:24.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.149+0000 7f9255ffb700 1 -- 172.21.15.2:0/52472686 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f924800e820 con 0x7f9240003a00 2023-08-25T03:35:24.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.149+0000 7f9255ffb700 1 -- 172.21.15.2:0/52472686 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924800eb90 con 0x7f9240003a00 2023-08-25T03:35:24.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.150+0000 7f925d9cb700 1 -- 172.21.15.2:0/52472686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9240003a00 msgr2=0x7f9240003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:24.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.150+0000 7f925d9cb700 1 --2- 172.21.15.2:0/52472686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9240003a00 0x7f9240003e20 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9248009b60 tx=0x7f9240005880 comp rx=0 tx=0).stop 2023-08-25T03:35:24.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.150+0000 7f925d9cb700 1 -- 172.21.15.2:0/52472686 shutdown_connections 2023-08-25T03:35:24.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.150+0000 7f925d9cb700 1 --2- 172.21.15.2:0/52472686 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9240003a00 0x7f9240003e20 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.150+0000 7f925d9cb700 1 -- 172.21.15.2:0/52472686 >> 172.21.15.2:0/52472686 conn(0x7f92580cbd30 msgr2=0x7f92580cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:24.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.150+0000 7f925d9cb700 1 -- 172.21.15.2:0/52472686 shutdown_connections 2023-08-25T03:35:24.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.150+0000 7f925d9cb700 1 -- 172.21.15.2:0/52472686 wait complete. 2023-08-25T03:35:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.151+0000 7f925d9cb700 1 Processor -- start 2023-08-25T03:35:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.151+0000 7f925d9cb700 1 -- start start 2023-08-25T03:35:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.152+0000 7f925d9cb700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9240003a00 0x7f9258158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.152+0000 7f925d9cb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92580ce840 0x7f92580cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.152+0000 7f925d9cb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92580b3250 0x7f92580b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:24.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.152+0000 7f925d9cb700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92580cf200 con 0x7f92580ce840 2023-08-25T03:35:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.152+0000 7f925d9cb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92580c80d0 con 0x7f92580b3250 2023-08-25T03:35:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.152+0000 7f925d9cb700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92580c83b0 con 0x7f9240003a00 2023-08-25T03:35:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.152+0000 7f92567fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92580ce840 0x7f92580cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.152+0000 7f92567fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92580ce840 0x7f92580cecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33626/0 (socket says 172.21.15.2:33626) 2023-08-25T03:35:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.152+0000 7f92567fc700 1 -- 172.21.15.2:0/119965934 learned_addr learned my addr 172.21.15.2:0/119965934 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.152+0000 7f9256ffd700 1 --2- 172.21.15.2:0/119965934 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9240003a00 0x7f9258158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:24.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.152+0000 7f92577fe700 1 --2- 172.21.15.2:0/119965934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92580b3250 0x7f92580b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:24.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.152+0000 7f92567fc700 1 -- 172.21.15.2:0/119965934 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9240003a00 msgr2=0x7f9258158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:24.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.152+0000 7f92567fc700 1 --2- 172.21.15.2:0/119965934 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9240003a00 0x7f9258158610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.152+0000 7f92567fc700 1 -- 172.21.15.2:0/119965934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92580b3250 msgr2=0x7f92580b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:24.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.152+0000 7f92567fc700 1 --2- 172.21.15.2:0/119965934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92580b3250 0x7f92580b36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.152+0000 7f92567fc700 1 -- 172.21.15.2:0/119965934 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92580c8690 con 0x7f92580ce840 2023-08-25T03:35:24.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.152+0000 7f92577fe700 1 --2- 172.21.15.2:0/119965934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92580b3250 0x7f92580b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:35:24.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.152+0000 7f92567fc700 1 --2- 172.21.15.2:0/119965934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92580ce840 0x7f92580cecc0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f924c007290 tx=0x7f924c007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.153+0000 7f923ffff700 1 -- 172.21.15.2:0/119965934 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c007830 con 0x7f92580ce840 2023-08-25T03:35:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.153+0000 7f923ffff700 1 -- 172.21.15.2:0/119965934 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f924c00f8d0 con 0x7f92580ce840 2023-08-25T03:35:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.153+0000 7f925d9cb700 1 -- 172.21.15.2:0/119965934 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92580c8970 con 0x7f92580ce840 2023-08-25T03:35:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.153+0000 7f923ffff700 1 -- 172.21.15.2:0/119965934 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924c00fc80 con 0x7f92580ce840 2023-08-25T03:35:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.153+0000 7f925d9cb700 1 -- 172.21.15.2:0/119965934 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9258158b50 con 0x7f92580ce840 2023-08-25T03:35:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.154+0000 7f925d9cb700 1 -- 172.21.15.2:0/119965934 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92580c8e90 con 0x7f92580ce840 2023-08-25T03:35:24.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.154+0000 7f923ffff700 1 -- 172.21.15.2:0/119965934 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f924c01d460 con 0x7f92580ce840 2023-08-25T03:35:24.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.155+0000 7f923ffff700 1 --2- 172.21.15.2:0/119965934 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f924406a0e0 0x7f924406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:24.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.155+0000 7f923ffff700 1 -- 172.21.15.2:0/119965934 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f924c088570 con 0x7f92580ce840 2023-08-25T03:35:24.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.155+0000 7f9256ffd700 1 --2- 172.21.15.2:0/119965934 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f924406a0e0 0x7f924406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:24.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.155+0000 7f9256ffd700 1 --2- 172.21.15.2:0/119965934 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f924406a0e0 0x7f924406c5a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9258158f10 tx=0x7f9248008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:24.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.159+0000 7f923ffff700 1 -- 172.21.15.2:0/119965934 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f92580c8e90 con 0x7f92580ce840 2023-08-25T03:35:24.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.361+0000 7f925d9cb700 1 -- 172.21.15.2:0/119965934 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92580c8e90 con 0x7f92580ce840 2023-08-25T03:35:24.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.363+0000 7f923ffff700 1 -- 172.21.15.2:0/119965934 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18207 (secure 0 0 0) 0x7f92580c8e90 con 0x7f92580ce840 2023-08-25T03:35:24.365 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:35:24.365 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":37,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:35:08.407381+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:35:05.870790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-08-25T03:35:06.296959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:35:24.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.365+0000 7f925d9cb700 1 -- 172.21.15.2:0/119965934 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f924406a0e0 msgr2=0x7f924406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:24.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.365+0000 7f925d9cb700 1 --2- 172.21.15.2:0/119965934 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f924406a0e0 0x7f924406c5a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9258158f10 tx=0x7f9248008030 comp rx=0 tx=0).stop 2023-08-25T03:35:24.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.365+0000 7f925d9cb700 1 -- 172.21.15.2:0/119965934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92580ce840 msgr2=0x7f92580cecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:24.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.365+0000 7f925d9cb700 1 --2- 172.21.15.2:0/119965934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92580ce840 0x7f92580cecc0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f924c007290 tx=0x7f924c007650 comp rx=0 tx=0).stop 2023-08-25T03:35:24.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.365+0000 7f925d9cb700 1 -- 172.21.15.2:0/119965934 shutdown_connections 2023-08-25T03:35:24.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.365+0000 7f925d9cb700 1 --2- 172.21.15.2:0/119965934 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f92580b3250 0x7f92580b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.365+0000 7f925d9cb700 1 --2- 172.21.15.2:0/119965934 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9240003a00 0x7f9258158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.365+0000 7f925d9cb700 1 --2- 172.21.15.2:0/119965934 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f924406a0e0 0x7f924406c5a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.365+0000 7f925d9cb700 1 --2- 172.21.15.2:0/119965934 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92580ce840 0x7f92580cecc0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.365+0000 7f925d9cb700 1 -- 172.21.15.2:0/119965934 >> 172.21.15.2:0/119965934 conn(0x7f92580cbd30 msgr2=0x7f92580cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:24.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.365+0000 7f925d9cb700 1 -- 172.21.15.2:0/119965934 shutdown_connections 2023-08-25T03:35:24.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.365+0000 7f925d9cb700 1 -- 172.21.15.2:0/119965934 wait complete. 2023-08-25T03:35:24.379 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-25T03:35:24.380 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T03:35:24.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.502+0000 7f6cf089a700 1 Processor -- start 2023-08-25T03:35:24.504 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.502+0000 7f6cf089a700 1 -- start start 2023-08-25T03:35:24.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.504+0000 7f6cf089a700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f6cec0cb5a0 0x7f6cec0ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:24.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.504+0000 7f6cf089a700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6cec0cd480 0x7f6cec06dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:24.508 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.504+0000 7f6cf089a700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cec0b36a0 con 0x7f6cec15b060 2023-08-25T03:35:24.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.504+0000 7f6cf089a700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cec0b6b30 con 0x7f6cec0cb5a0 2023-08-25T03:35:24.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.504+0000 7f6cf089a700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec0b2a90 con 0x7f6cec0cd480 2023-08-25T03:35:24.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.504+0000 7f6cea59c700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f6cec15b060 0x7f6cec15b440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48530/0 (socket says 172.21.15.2:48530) 2023-08-25T03:35:24.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.504+0000 7f6cea59c700 1 -- 172.21.15.2:0/3823678272 learned_addr learned my addr 172.21.15.2:0/3823678272 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:24.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.504+0000 7f6cead9d700 1 --2- 172.21.15.2:0/3823678272 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6cec0cd480 0x7f6cec06dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:24.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.504+0000 7f6ce9d9b700 1 --2- 172.21.15.2:0/3823678272 >> v2:172.21.15.31:3300/0 conn(0x7f6cec0cb5a0 0x7f6cec0ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:24.509 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.504+0000 7f6ce959a700 1 -- 172.21.15.2:0/3823678272 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 529805935 0 0) 0x7f6cec0b36a0 con 0x7f6cec15b060 2023-08-25T03:35:24.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.504+0000 7f6ce959a700 1 -- 172.21.15.2:0/3823678272 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cd80035b0 con 0x7f6cec15b060 2023-08-25T03:35:24.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.504+0000 7f6ce959a700 1 -- 172.21.15.2:0/3823678272 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 819571808 0 0) 0x7f6cd80035b0 con 0x7f6cec15b060 2023-08-25T03:35:24.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.504+0000 7f6ce959a700 1 -- 172.21.15.2:0/3823678272 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6cec0cd480 msgr2=0x7f6cec06dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:24.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.504+0000 7f6ce959a700 1 --2- 172.21.15.2:0/3823678272 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6cec0cd480 0x7f6cec06dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.504+0000 7f6ce959a700 1 -- 172.21.15.2:0/3823678272 >> v2:172.21.15.31:3300/0 conn(0x7f6cec0cb5a0 msgr2=0x7f6cec0ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:24.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.504+0000 7f6ce959a700 1 --2- 172.21.15.2:0/3823678272 >> v2:172.21.15.31:3300/0 conn(0x7f6cec0cb5a0 0x7f6cec0ccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.504+0000 7f6ce959a700 1 -- 172.21.15.2:0/3823678272 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cec0b2d70 con 0x7f6cec15b060 2023-08-25T03:35:24.510 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.504+0000 7f6ce959a700 1 -- 172.21.15.2:0/3823678272 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f6cdc001670 con 0x7f6cec15b060 2023-08-25T03:35:24.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.504+0000 7f6ce959a700 1 -- 172.21.15.2:0/3823678272 >> v1:172.21.15.2:6789/0 conn(0x7f6cec15b060 legacy=0x7f6cec15b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:24.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.504+0000 7f6ce959a700 1 --2- 172.21.15.2:0/3823678272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cd8003a00 0x7f6cd8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:24.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.504+0000 7f6ce959a700 1 --2- 172.21.15.2:0/3823678272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd8004850 0x7f6cd8004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:24.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.504+0000 7f6ce959a700 1 --2- 172.21.15.2:0/3823678272 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6cec0cd480 0x7f6cd8005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:24.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.505+0000 7f6ce959a700 1 -- 172.21.15.2:0/3823678272 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec0b6b30 con 0x7f6cd8003a00 2023-08-25T03:35:24.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.505+0000 7f6ce959a700 1 -- 172.21.15.2:0/3823678272 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec0b2a90 con 0x7f6cd8004850 2023-08-25T03:35:24.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.505+0000 7f6ce959a700 1 -- 172.21.15.2:0/3823678272 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd800ba40 con 0x7f6cec0cd480 2023-08-25T03:35:24.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.505+0000 7f6cea59c700 1 --2- 172.21.15.2:0/3823678272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cd8003a00 0x7f6cd8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:24.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.505+0000 7f6ce9d9b700 1 --2- 172.21.15.2:0/3823678272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd8004850 0x7f6cd8004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:24.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.505+0000 7f6cead9d700 1 --2- 172.21.15.2:0/3823678272 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6cec0cd480 0x7f6cd8005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:24.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.505+0000 7f6cea59c700 1 -- 172.21.15.2:0/3823678272 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6cec0cd480 msgr2=0x7f6cd8005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:24.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.505+0000 7f6cea59c700 1 --2- 172.21.15.2:0/3823678272 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6cec0cd480 0x7f6cd8005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.505+0000 7f6cea59c700 1 -- 172.21.15.2:0/3823678272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd8004850 msgr2=0x7f6cd8004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:24.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.505+0000 7f6cea59c700 1 --2- 172.21.15.2:0/3823678272 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cd8004850 0x7f6cd8004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.505+0000 7f6cea59c700 1 -- 172.21.15.2:0/3823678272 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd80035b0 con 0x7f6cd8003a00 2023-08-25T03:35:24.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.505+0000 7f6cead9d700 1 --2- 172.21.15.2:0/3823678272 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6cec0cd480 0x7f6cd8005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:24.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.505+0000 7f6cea59c700 1 --2- 172.21.15.2:0/3823678272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cd8003a00 0x7f6cd8003e20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00c370 tx=0x7f6cd8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8465f65e7290fa0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:24.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.505+0000 7f6ce959a700 1 -- 172.21.15.2:0/3823678272 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc007c60 con 0x7f6cd8003a00 2023-08-25T03:35:24.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.505+0000 7f6ce959a700 1 -- 172.21.15.2:0/3823678272 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cdc00ce50 con 0x7f6cd8003a00 2023-08-25T03:35:24.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.505+0000 7f6ce959a700 1 -- 172.21.15.2:0/3823678272 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc005ba0 con 0x7f6cd8003a00 2023-08-25T03:35:24.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.506+0000 7f6cf089a700 1 -- 172.21.15.2:0/3823678272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cd8003a00 msgr2=0x7f6cd8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:24.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.506+0000 7f6cf089a700 1 --2- 172.21.15.2:0/3823678272 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cd8003a00 0x7f6cd8003e20 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00c370 tx=0x7f6cd8005880 comp rx=0 tx=0).stop 2023-08-25T03:35:24.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.506+0000 7f6cead9d700 1 -- 172.21.15.2:0/3823678272 reap_dead start 2023-08-25T03:35:24.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.506+0000 7f6cf089a700 1 -- 172.21.15.2:0/3823678272 shutdown_connections 2023-08-25T03:35:24.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.506+0000 7f6cf089a700 1 -- 172.21.15.2:0/3823678272 >> 172.21.15.2:0/3823678272 conn(0x7f6cec0d0f70 msgr2=0x7f6cec0d1350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:24.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.506+0000 7f6cf089a700 1 -- 172.21.15.2:0/3823678272 shutdown_connections 2023-08-25T03:35:24.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.506+0000 7f6cf089a700 1 -- 172.21.15.2:0/3823678272 wait complete. 2023-08-25T03:35:24.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.507+0000 7f6cf089a700 1 Processor -- start 2023-08-25T03:35:24.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.507+0000 7f6cf089a700 1 -- start start 2023-08-25T03:35:24.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.508+0000 7f6cf089a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cd8003a00 0x7f6cec14f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:24.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.508+0000 7f6cf089a700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6cec1f4870 0x7f6cec0d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:24.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.508+0000 7f6cf089a700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cec0cfbc0 0x7f6cec0b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:24.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.508+0000 7f6cf089a700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec0d03f0 con 0x7f6cd8003a00 2023-08-25T03:35:24.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.508+0000 7f6cf089a700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec14f6c0 con 0x7f6cec0cfbc0 2023-08-25T03:35:24.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.508+0000 7f6cf089a700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec0d0110 con 0x7f6cec1f4870 2023-08-25T03:35:24.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.508+0000 7f6cea59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cd8003a00 0x7f6cec14f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:24.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.508+0000 7f6cea59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cd8003a00 0x7f6cec14f180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33644/0 (socket says 172.21.15.2:33644) 2023-08-25T03:35:24.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.508+0000 7f6cea59c700 1 -- 172.21.15.2:0/3473434620 learned_addr learned my addr 172.21.15.2:0/3473434620 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:24.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.508+0000 7f6ce9d9b700 1 --2- 172.21.15.2:0/3473434620 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6cec1f4870 0x7f6cec0d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:24.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.509+0000 7f6cead9d700 1 --2- 172.21.15.2:0/3473434620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cec0cfbc0 0x7f6cec0b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:24.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.509+0000 7f6cea59c700 1 -- 172.21.15.2:0/3473434620 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6cec1f4870 msgr2=0x7f6cec0d07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:24.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.509+0000 7f6cea59c700 1 --2- 172.21.15.2:0/3473434620 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6cec1f4870 0x7f6cec0d07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.509+0000 7f6cea59c700 1 -- 172.21.15.2:0/3473434620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cec0cfbc0 msgr2=0x7f6cec0b4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:24.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.509+0000 7f6cea59c700 1 --2- 172.21.15.2:0/3473434620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cec0cfbc0 0x7f6cec0b4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.509+0000 7f6cea59c700 1 -- 172.21.15.2:0/3473434620 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cec1f3d90 con 0x7f6cd8003a00 2023-08-25T03:35:24.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.509+0000 7f6cead9d700 1 --2- 172.21.15.2:0/3473434620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cec0cfbc0 0x7f6cec0b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:35:24.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.509+0000 7f6cea59c700 1 --2- 172.21.15.2:0/3473434620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cd8003a00 0x7f6cec14f180 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8004770 tx=0x7f6cdc000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:24.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.509+0000 7f6ce2ffd700 1 -- 172.21.15.2:0/3473434620 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc0070d0 con 0x7f6cd8003a00 2023-08-25T03:35:24.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.509+0000 7f6ce2ffd700 1 -- 172.21.15.2:0/3473434620 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cdc007230 con 0x7f6cd8003a00 2023-08-25T03:35:24.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.509+0000 7f6cf089a700 1 -- 172.21.15.2:0/3473434620 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cec1f4070 con 0x7f6cd8003a00 2023-08-25T03:35:24.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.509+0000 7f6ce2ffd700 1 -- 172.21.15.2:0/3473434620 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc007620 con 0x7f6cd8003a00 2023-08-25T03:35:24.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.509+0000 7f6cf089a700 1 -- 172.21.15.2:0/3473434620 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cec0d0ce0 con 0x7f6cd8003a00 2023-08-25T03:35:24.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.510+0000 7f6cf089a700 1 -- 172.21.15.2:0/3473434620 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cec1f46c0 con 0x7f6cd8003a00 2023-08-25T03:35:24.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.510+0000 7f6ce2ffd700 1 -- 172.21.15.2:0/3473434620 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f6cdc01e430 con 0x7f6cd8003a00 2023-08-25T03:35:24.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.511+0000 7f6ce2ffd700 1 --2- 172.21.15.2:0/3473434620 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6ccc06a0e0 0x7f6ccc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:24.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.511+0000 7f6ce2ffd700 1 -- 172.21.15.2:0/3473434620 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f6cdc0899f0 con 0x7f6cd8003a00 2023-08-25T03:35:24.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.511+0000 7f6ce9d9b700 1 --2- 172.21.15.2:0/3473434620 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6ccc06a0e0 0x7f6ccc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:24.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.511+0000 7f6ce9d9b700 1 --2- 172.21.15.2:0/3473434620 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6ccc06a0e0 0x7f6ccc06c5a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6cec0cf6f0 tx=0x7f6cd4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:24.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.515+0000 7f6ce2ffd700 1 -- 172.21.15.2:0/3473434620 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f6cdc055170 con 0x7f6cd8003a00 2023-08-25T03:35:24.761 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.760+0000 7f6cf089a700 1 -- 172.21.15.2:0/3473434620 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6cec1f3bf0 con 0x7f6cd8003a00 2023-08-25T03:35:24.762 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.760+0000 7f6ce2ffd700 1 -- 172.21.15.2:0/3473434620 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4869 (secure 0 0 0) 0x7f6cec1f3bf0 con 0x7f6cd8003a00 2023-08-25T03:35:24.762 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:35:24.763 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4751,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/3802947908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3802947908},{"type":"v1","addr":"172.21.15.31:6834","nonce":3802947908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1470640897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6835","nonce":1470640897},{"type":"v1","addr":"172.21.15.2:6837","nonce":1470640897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6836/3706466654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3706466654},{"type":"v1","addr":"172.21.15.2:6836","nonce":3706466654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T03:35:07.333851+0000","modified":"2023-08-25T03:35:10.931788+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5229":{"gid":5229,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1140461626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1140461626},{"type":"v1","addr":"172.21.15.31:6835","nonce":1140461626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-25T03:35:24.764 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.762+0000 7f6cf089a700 1 -- 172.21.15.2:0/3473434620 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6ccc06a0e0 msgr2=0x7f6ccc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:24.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.762+0000 7f6cf089a700 1 --2- 172.21.15.2:0/3473434620 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6ccc06a0e0 0x7f6ccc06c5a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6cec0cf6f0 tx=0x7f6cd4008040 comp rx=0 tx=0).stop 2023-08-25T03:35:24.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.762+0000 7f6cf089a700 1 -- 172.21.15.2:0/3473434620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cd8003a00 msgr2=0x7f6cec14f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:24.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.762+0000 7f6cf089a700 1 --2- 172.21.15.2:0/3473434620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cd8003a00 0x7f6cec14f180 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6cd8004770 tx=0x7f6cdc000fe0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.762+0000 7f6cf089a700 1 -- 172.21.15.2:0/3473434620 shutdown_connections 2023-08-25T03:35:24.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.762+0000 7f6cf089a700 1 --2- 172.21.15.2:0/3473434620 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f6ccc06a0e0 0x7f6ccc06c5a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.762+0000 7f6cf089a700 1 --2- 172.21.15.2:0/3473434620 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f6cd8003a00 0x7f6cec14f180 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.765 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.762+0000 7f6cf089a700 1 --2- 172.21.15.2:0/3473434620 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6cec0cfbc0 0x7f6cec0b4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.762+0000 7f6cf089a700 1 --2- 172.21.15.2:0/3473434620 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6cec1f4870 0x7f6cec0d07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.762+0000 7f6cf089a700 1 -- 172.21.15.2:0/3473434620 >> 172.21.15.2:0/3473434620 conn(0x7f6cec0d0f70 msgr2=0x7f6cec15a530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:24.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.762+0000 7f6cf089a700 1 -- 172.21.15.2:0/3473434620 shutdown_connections 2023-08-25T03:35:24.766 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.762+0000 7f6cf089a700 1 -- 172.21.15.2:0/3473434620 wait complete. 2023-08-25T03:35:24.766 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2023-08-25T03:35:24.776 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T03:35:24.901 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.900+0000 7f7d7539e700 1 Processor -- start 2023-08-25T03:35:24.902 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.900+0000 7f7d7539e700 1 -- start start 2023-08-25T03:35:24.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.900+0000 7f7d7539e700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7d7015af50 0x7f7d700b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:24.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.900+0000 7f7d7539e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7d700b33c0 0x7f7d7006e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:24.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.900+0000 7f7d7539e700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f7d7006ea70 0x7f7d700cd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:24.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.900+0000 7f7d7539e700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d700cd5b0 con 0x7f7d7006ea70 2023-08-25T03:35:24.905 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.900+0000 7f7d7539e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d7015b330 con 0x7f7d700b33c0 2023-08-25T03:35:24.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.900+0000 7f7d7539e700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d700ccd10 con 0x7f7d7015af50 2023-08-25T03:35:24.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.900+0000 7f7d74b9d700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f7d7006ea70 0x7f7d700cd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:24.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.901+0000 7f7d74b9d700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f7d7006ea70 0x7f7d700cd720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33656/0 (socket says 172.21.15.2:33656) 2023-08-25T03:35:24.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.901+0000 7f7d74b9d700 1 -- 172.21.15.2:0/2305696070 learned_addr learned my addr 172.21.15.2:0/2305696070 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:24.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.901+0000 7f7d6ffff700 1 --2- 172.21.15.2:0/2305696070 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7d7015af50 0x7f7d700b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:24.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.901+0000 7f7d6f7fe700 1 --2- 172.21.15.2:0/2305696070 >> v2:172.21.15.31:3300/0 conn(0x7f7d700b33c0 0x7f7d7006e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:24.906 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.901+0000 7f7d74b9d700 1 -- 172.21.15.2:0/2305696070 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7d7015af50 msgr2=0x7f7d700b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:24.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.901+0000 7f7d74b9d700 1 --2- 172.21.15.2:0/2305696070 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7d7015af50 0x7f7d700b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.901+0000 7f7d74b9d700 1 -- 172.21.15.2:0/2305696070 >> v2:172.21.15.31:3300/0 conn(0x7f7d700b33c0 msgr2=0x7f7d7006e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:24.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.901+0000 7f7d74b9d700 1 --2- 172.21.15.2:0/2305696070 >> v2:172.21.15.31:3300/0 conn(0x7f7d700b33c0 0x7f7d7006e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.901+0000 7f7d74b9d700 1 -- 172.21.15.2:0/2305696070 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d700ccff0 con 0x7f7d7006ea70 2023-08-25T03:35:24.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.901+0000 7f7d6f7fe700 1 --2- 172.21.15.2:0/2305696070 >> v2:172.21.15.31:3300/0 conn(0x7f7d700b33c0 0x7f7d7006e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:24.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.901+0000 7f7d6ffff700 1 --2- 172.21.15.2:0/2305696070 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7d7015af50 0x7f7d700b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:24.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.901+0000 7f7d74b9d700 1 --2- 172.21.15.2:0/2305696070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d7006ea70 0x7f7d700cd720 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7d700b38d0 tx=0x7f7d6800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e812f51ec8ed3680 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:24.907 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.901+0000 7f7d6effd700 1 -- 172.21.15.2:0/2305696070 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6800e070 con 0x7f7d7006ea70 2023-08-25T03:35:24.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.901+0000 7f7d6effd700 1 -- 172.21.15.2:0/2305696070 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d6800b9c0 con 0x7f7d7006ea70 2023-08-25T03:35:24.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.901+0000 7f7d6effd700 1 -- 172.21.15.2:0/2305696070 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6800bd70 con 0x7f7d7006ea70 2023-08-25T03:35:24.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.902+0000 7f7d7539e700 1 -- 172.21.15.2:0/2305696070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d7006ea70 msgr2=0x7f7d700cd720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:24.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.902+0000 7f7d7539e700 1 --2- 172.21.15.2:0/2305696070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d7006ea70 0x7f7d700cd720 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7d700b38d0 tx=0x7f7d6800b330 comp rx=0 tx=0).stop 2023-08-25T03:35:24.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.902+0000 7f7d7539e700 1 -- 172.21.15.2:0/2305696070 shutdown_connections 2023-08-25T03:35:24.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.902+0000 7f7d7539e700 1 --2- 172.21.15.2:0/2305696070 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d7006ea70 0x7f7d700cd720 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.908 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.902+0000 7f7d7539e700 1 --2- 172.21.15.2:0/2305696070 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7d7015af50 0x7f7d700b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.902+0000 7f7d7539e700 1 --2- 172.21.15.2:0/2305696070 >> v2:172.21.15.31:3300/0 conn(0x7f7d700b33c0 0x7f7d7006e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.902+0000 7f7d7539e700 1 -- 172.21.15.2:0/2305696070 >> 172.21.15.2:0/2305696070 conn(0x7f7d70065fe0 msgr2=0x7f7d700663c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:24.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.902+0000 7f7d7539e700 1 -- 172.21.15.2:0/2305696070 shutdown_connections 2023-08-25T03:35:24.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.902+0000 7f7d7539e700 1 -- 172.21.15.2:0/2305696070 wait complete. 2023-08-25T03:35:24.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.903+0000 7f7d7539e700 1 Processor -- start 2023-08-25T03:35:24.909 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.903+0000 7f7d7539e700 1 -- start start 2023-08-25T03:35:24.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.904+0000 7f7d7539e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d7006ea70 0x7f7d70155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:24.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.904+0000 7f7d7539e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d7015af50 0x7f7d700c9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:24.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.904+0000 7f7d7539e700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7d700ca390 0x7f7d700a9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:24.911 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.904+0000 7f7d7539e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d700ca820 con 0x7f7d7015af50 2023-08-25T03:35:24.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.904+0000 7f7d7539e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d701f2560 con 0x7f7d7006ea70 2023-08-25T03:35:24.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.904+0000 7f7d7539e700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d701f5b00 con 0x7f7d700ca390 2023-08-25T03:35:24.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.904+0000 7f7d6f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d7015af50 0x7f7d700c9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:24.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.904+0000 7f7d6f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d7015af50 0x7f7d700c9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33660/0 (socket says 172.21.15.2:33660) 2023-08-25T03:35:24.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.904+0000 7f7d6f7fe700 1 -- 172.21.15.2:0/1217482857 learned_addr learned my addr 172.21.15.2:0/1217482857 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:24.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.904+0000 7f7d74b9d700 1 --2- 172.21.15.2:0/1217482857 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7d700ca390 0x7f7d700a9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:24.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.904+0000 7f7d6ffff700 1 --2- 172.21.15.2:0/1217482857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d7006ea70 0x7f7d70155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:24.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.905+0000 7f7d6f7fe700 1 -- 172.21.15.2:0/1217482857 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7d700ca390 msgr2=0x7f7d700a9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:24.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.905+0000 7f7d6f7fe700 1 --2- 172.21.15.2:0/1217482857 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7d700ca390 0x7f7d700a9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.905+0000 7f7d6f7fe700 1 -- 172.21.15.2:0/1217482857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d7006ea70 msgr2=0x7f7d70155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:24.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.905+0000 7f7d6f7fe700 1 --2- 172.21.15.2:0/1217482857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d7006ea70 0x7f7d70155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:24.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.905+0000 7f7d6f7fe700 1 -- 172.21.15.2:0/1217482857 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d68009b30 con 0x7f7d7015af50 2023-08-25T03:35:24.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.905+0000 7f7d6f7fe700 1 --2- 172.21.15.2:0/1217482857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d7015af50 0x7f7d700c9e50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400e810 tx=0x7f7d6400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:24.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.905+0000 7f7d6cff9700 1 -- 172.21.15.2:0/1217482857 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6400c070 con 0x7f7d7015af50 2023-08-25T03:35:24.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.905+0000 7f7d6cff9700 1 -- 172.21.15.2:0/1217482857 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d64005c20 con 0x7f7d7015af50 2023-08-25T03:35:24.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.905+0000 7f7d7539e700 1 -- 172.21.15.2:0/1217482857 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d701f5d50 con 0x7f7d7015af50 2023-08-25T03:35:24.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.905+0000 7f7d6cff9700 1 -- 172.21.15.2:0/1217482857 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d64004030 con 0x7f7d7015af50 2023-08-25T03:35:24.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.905+0000 7f7d7539e700 1 -- 172.21.15.2:0/1217482857 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d701f6310 con 0x7f7d7015af50 2023-08-25T03:35:24.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.906+0000 7f7d7539e700 1 -- 172.21.15.2:0/1217482857 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d700c7710 con 0x7f7d7015af50 2023-08-25T03:35:24.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.906+0000 7f7d6cff9700 1 -- 172.21.15.2:0/1217482857 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7d64004190 con 0x7f7d7015af50 2023-08-25T03:35:24.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.907+0000 7f7d6cff9700 1 --2- 172.21.15.2:0/1217482857 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f7d5c06a0e0 0x7f7d5c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:24.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.907+0000 7f7d6ffff700 1 --2- 172.21.15.2:0/1217482857 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f7d5c06a0e0 0x7f7d5c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:24.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.907+0000 7f7d6cff9700 1 -- 172.21.15.2:0/1217482857 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f7d64087ba0 con 0x7f7d7015af50 2023-08-25T03:35:24.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.907+0000 7f7d6ffff700 1 --2- 172.21.15.2:0/1217482857 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f7d5c06a0e0 0x7f7d5c06c5a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000c940 tx=0x7f7d60008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:24.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:24.912+0000 7f7d6cff9700 1 -- 172.21.15.2:0/1217482857 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7d64053320 con 0x7f7d7015af50 2023-08-25T03:35:25.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.158+0000 7f7d7539e700 1 -- 172.21.15.2:0/1217482857 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d7004ee70 con 0x7f7d7015af50 2023-08-25T03:35:25.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.158+0000 7f7d6cff9700 1 -- 172.21.15.2:0/1217482857 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4869 (secure 0 0 0) 0x7f7d64017090 con 0x7f7d7015af50 2023-08-25T03:35:25.161 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:35:25.161 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4751,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/3802947908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3802947908},{"type":"v1","addr":"172.21.15.31:6834","nonce":3802947908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1470640897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6835","nonce":1470640897},{"type":"v1","addr":"172.21.15.2:6837","nonce":1470640897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6836/3706466654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3706466654},{"type":"v1","addr":"172.21.15.2:6836","nonce":3706466654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-08-25T03:35:07.333851+0000","modified":"2023-08-25T03:35:10.931788+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5229},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5229":{"gid":5229,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.31:6835/1140461626","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1140461626},{"type":"v1","addr":"172.21.15.31:6835","nonce":1140461626}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-08-25T03:35:25.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.161+0000 7f7d7539e700 1 -- 172.21.15.2:0/1217482857 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f7d5c06a0e0 msgr2=0x7f7d5c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:25.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.161+0000 7f7d7539e700 1 --2- 172.21.15.2:0/1217482857 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f7d5c06a0e0 0x7f7d5c06c5a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7d6000c940 tx=0x7f7d60008040 comp rx=0 tx=0).stop 2023-08-25T03:35:25.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.161+0000 7f7d7539e700 1 -- 172.21.15.2:0/1217482857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d7015af50 msgr2=0x7f7d700c9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:25.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.161+0000 7f7d7539e700 1 --2- 172.21.15.2:0/1217482857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d7015af50 0x7f7d700c9e50 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7d6400e810 tx=0x7f7d6400eb20 comp rx=0 tx=0).stop 2023-08-25T03:35:25.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.161+0000 7f7d7539e700 1 -- 172.21.15.2:0/1217482857 shutdown_connections 2023-08-25T03:35:25.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.161+0000 7f7d7539e700 1 --2- 172.21.15.2:0/1217482857 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7d700ca390 0x7f7d700a9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:25.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.161+0000 7f7d7539e700 1 --2- 172.21.15.2:0/1217482857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7d7006ea70 0x7f7d70155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:25.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.161+0000 7f7d7539e700 1 --2- 172.21.15.2:0/1217482857 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f7d5c06a0e0 0x7f7d5c06c5a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:25.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.161+0000 7f7d7539e700 1 --2- 172.21.15.2:0/1217482857 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7d7015af50 0x7f7d700c9e50 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:25.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.161+0000 7f7d7539e700 1 -- 172.21.15.2:0/1217482857 >> 172.21.15.2:0/1217482857 conn(0x7f7d70065fe0 msgr2=0x7f7d700b6630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:25.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.161+0000 7f7d7539e700 1 -- 172.21.15.2:0/1217482857 shutdown_connections 2023-08-25T03:35:25.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.161+0000 7f7d7539e700 1 -- 172.21.15.2:0/1217482857 wait complete. 2023-08-25T03:35:25.164 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 17 2023-08-25T03:35:25.175 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:35:25.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.298+0000 7fb79f418700 1 Processor -- start 2023-08-25T03:35:25.300 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.298+0000 7fb79f418700 1 -- start start 2023-08-25T03:35:25.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.299+0000 7fb79f418700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fb7980cdb70 0x7fb79806b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:25.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.299+0000 7fb79f418700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fb7980ce000 0x7fb79806be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:25.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.299+0000 7fb79f418700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb798161620 con 0x7fb7980b1fb0 2023-08-25T03:35:25.306 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.299+0000 7fb79f418700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb79806d450 con 0x7fb7980cdb70 2023-08-25T03:35:25.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.299+0000 7fb79f418700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb79806d730 con 0x7fb7980ce000 2023-08-25T03:35:25.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.299+0000 7fb79c9b3700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fb7980cdb70 0x7fb79806b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:25.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.299+0000 7fb79c9b3700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fb7980cdb70 0x7fb79806b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33668/0 (socket says 172.21.15.2:33668) 2023-08-25T03:35:25.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.299+0000 7fb79c9b3700 1 -- 172.21.15.2:0/32471499 learned_addr learned my addr 172.21.15.2:0/32471499 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:25.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.299+0000 7fb79d9b5700 1 --2- 172.21.15.2:0/32471499 >> v2:172.21.15.31:3300/0 conn(0x7fb7980ce000 0x7fb79806be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:25.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.299+0000 7fb79c9b3700 1 -- 172.21.15.2:0/32471499 >> v2:172.21.15.31:3300/0 conn(0x7fb7980ce000 msgr2=0x7fb79806be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:25.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.299+0000 7fb79c9b3700 1 --2- 172.21.15.2:0/32471499 >> v2:172.21.15.31:3300/0 conn(0x7fb7980ce000 0x7fb79806be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:25.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.299+0000 7fb79c9b3700 1 -- 172.21.15.2:0/32471499 >> v1:172.21.15.31:6789/0 conn(0x7fb7980b1fb0 legacy=0x7fb798160f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:25.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.299+0000 7fb79c9b3700 1 -- 172.21.15.2:0/32471499 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79806da10 con 0x7fb7980cdb70 2023-08-25T03:35:25.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.300+0000 7fb79c9b3700 1 --2- 172.21.15.2:0/32471499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7980cdb70 0x7fb79806b8c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb798072360 tx=0x7fb794009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7dea8f176e9f67 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:25.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.300+0000 7fb78ffff700 1 -- 172.21.15.2:0/32471499 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794004230 con 0x7fb7980cdb70 2023-08-25T03:35:25.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.300+0000 7fb78ffff700 1 -- 172.21.15.2:0/32471499 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb794004390 con 0x7fb7980cdb70 2023-08-25T03:35:25.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.300+0000 7fb78ffff700 1 -- 172.21.15.2:0/32471499 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7940036a0 con 0x7fb7980cdb70 2023-08-25T03:35:25.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.300+0000 7fb79f418700 1 -- 172.21.15.2:0/32471499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7980cdb70 msgr2=0x7fb79806b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:25.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.300+0000 7fb79f418700 1 --2- 172.21.15.2:0/32471499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7980cdb70 0x7fb79806b8c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb798072360 tx=0x7fb794009bd0 comp rx=0 tx=0).stop 2023-08-25T03:35:25.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.300+0000 7fb79f418700 1 -- 172.21.15.2:0/32471499 shutdown_connections 2023-08-25T03:35:25.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.300+0000 7fb79f418700 1 --2- 172.21.15.2:0/32471499 >> v2:172.21.15.31:3300/0 conn(0x7fb7980ce000 0x7fb79806be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:25.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.300+0000 7fb79f418700 1 --2- 172.21.15.2:0/32471499 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7980cdb70 0x7fb79806b8c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:25.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.300+0000 7fb79f418700 1 -- 172.21.15.2:0/32471499 >> 172.21.15.2:0/32471499 conn(0x7fb7980cfea0 msgr2=0x7fb7980d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:25.310 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.300+0000 7fb79f418700 1 -- 172.21.15.2:0/32471499 shutdown_connections 2023-08-25T03:35:25.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.300+0000 7fb79f418700 1 -- 172.21.15.2:0/32471499 wait complete. 2023-08-25T03:35:25.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.302+0000 7fb79f418700 1 Processor -- start 2023-08-25T03:35:25.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.302+0000 7fb79f418700 1 -- start start 2023-08-25T03:35:25.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.302+0000 7fb79f418700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb7980b1fb0 0x7fb7981512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:25.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.302+0000 7fb79f418700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7980cdb70 0x7fb7981f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:25.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.302+0000 7fb79f418700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7980ce000 0x7fb7981f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:25.311 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.302+0000 7fb79f418700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7980b3610 con 0x7fb7980cdb70 2023-08-25T03:35:25.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.302+0000 7fb79f418700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7980c9eb0 con 0x7fb7980ce000 2023-08-25T03:35:25.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.302+0000 7fb79f418700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7980b5080 con 0x7fb7980b1fb0 2023-08-25T03:35:25.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.302+0000 7fb79c9b3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7980cdb70 0x7fb7981f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:25.312 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.302+0000 7fb79c9b3700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7980cdb70 0x7fb7981f5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33672/0 (socket says 172.21.15.2:33672) 2023-08-25T03:35:25.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.302+0000 7fb79c9b3700 1 -- 172.21.15.2:0/1998059095 learned_addr learned my addr 172.21.15.2:0/1998059095 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:25.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.303+0000 7fb79d9b5700 1 --2- 172.21.15.2:0/1998059095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7980ce000 0x7fb7981f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:25.313 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.303+0000 7fb79d1b4700 1 --2- 172.21.15.2:0/1998059095 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb7980b1fb0 0x7fb7981512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:25.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.303+0000 7fb79c9b3700 1 -- 172.21.15.2:0/1998059095 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb7980b1fb0 msgr2=0x7fb7981512e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:25.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.303+0000 7fb79c9b3700 1 --2- 172.21.15.2:0/1998059095 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb7980b1fb0 0x7fb7981512e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:25.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.303+0000 7fb79c9b3700 1 -- 172.21.15.2:0/1998059095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7980ce000 msgr2=0x7fb7981f62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:25.314 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.303+0000 7fb79c9b3700 1 --2- 172.21.15.2:0/1998059095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7980ce000 0x7fb7981f62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:25.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.303+0000 7fb79c9b3700 1 -- 172.21.15.2:0/1998059095 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb794009550 con 0x7fb7980cdb70 2023-08-25T03:35:25.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.303+0000 7fb79d9b5700 1 --2- 172.21.15.2:0/1998059095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7980ce000 0x7fb7981f62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:35:25.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.303+0000 7fb79c9b3700 1 --2- 172.21.15.2:0/1998059095 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7980cdb70 0x7fb7981f5d90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb79806e2c0 tx=0x7fb794004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:25.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.303+0000 7fb78dffb700 1 -- 172.21.15.2:0/1998059095 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794011630 con 0x7fb7980cdb70 2023-08-25T03:35:25.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.303+0000 7fb78dffb700 1 -- 172.21.15.2:0/1998059095 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb794011790 con 0x7fb7980cdb70 2023-08-25T03:35:25.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.303+0000 7fb79f418700 1 -- 172.21.15.2:0/1998059095 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7980b5340 con 0x7fb7980cdb70 2023-08-25T03:35:25.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.303+0000 7fb78dffb700 1 -- 172.21.15.2:0/1998059095 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794011b40 con 0x7fb7980cdb70 2023-08-25T03:35:25.315 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.303+0000 7fb79f418700 1 -- 172.21.15.2:0/1998059095 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb798160630 con 0x7fb7980cdb70 2023-08-25T03:35:25.316 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.304+0000 7fb79f418700 1 -- 172.21.15.2:0/1998059095 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb798066f70 con 0x7fb7980cdb70 2023-08-25T03:35:25.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.305+0000 7fb78dffb700 1 -- 172.21.15.2:0/1998059095 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb79401b440 con 0x7fb7980cdb70 2023-08-25T03:35:25.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.305+0000 7fb78dffb700 1 --2- 172.21.15.2:0/1998059095 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb77c06a0e0 0x7fb77c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:25.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.305+0000 7fb79d1b4700 1 --2- 172.21.15.2:0/1998059095 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb77c06a0e0 0x7fb77c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:25.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.305+0000 7fb78dffb700 1 -- 172.21.15.2:0/1998059095 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7fb794089bf0 con 0x7fb7980cdb70 2023-08-25T03:35:25.317 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.306+0000 7fb79d1b4700 1 --2- 172.21.15.2:0/1998059095 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb77c06a0e0 0x7fb77c06c5a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb78800a7d0 tx=0x7fb788008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:25.318 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.310+0000 7fb78dffb700 1 -- 172.21.15.2:0/1998059095 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb794055370 con 0x7fb7980cdb70 2023-08-25T03:35:25.511 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.509+0000 7fb79f418700 1 -- 172.21.15.2:0/1998059095 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb798066f70 con 0x7fb7980cdb70 2023-08-25T03:35:25.512 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.510+0000 7fb78dffb700 1 -- 172.21.15.2:0/1998059095 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) v1 ==== 74+0+18207 (secure 0 0 0) 0x7fb794058990 con 0x7fb7980cdb70 2023-08-25T03:35:25.513 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:35:25.513 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":37,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:35:08.407381+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-08-25T03:35:05.870790+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-08-25T03:35:06.296959+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:35:25.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.512+0000 7fb79f418700 1 -- 172.21.15.2:0/1998059095 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb77c06a0e0 msgr2=0x7fb77c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:25.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.512+0000 7fb79f418700 1 --2- 172.21.15.2:0/1998059095 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb77c06a0e0 0x7fb77c06c5a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb78800a7d0 tx=0x7fb788008040 comp rx=0 tx=0).stop 2023-08-25T03:35:25.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.512+0000 7fb79f418700 1 -- 172.21.15.2:0/1998059095 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7980cdb70 msgr2=0x7fb7981f5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:25.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.512+0000 7fb79f418700 1 --2- 172.21.15.2:0/1998059095 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7980cdb70 0x7fb7981f5d90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb79806e2c0 tx=0x7fb794004750 comp rx=0 tx=0).stop 2023-08-25T03:35:25.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.512+0000 7fb79f418700 1 -- 172.21.15.2:0/1998059095 shutdown_connections 2023-08-25T03:35:25.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.512+0000 7fb79f418700 1 --2- 172.21.15.2:0/1998059095 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb7980ce000 0x7fb7981f62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:25.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.512+0000 7fb79f418700 1 --2- 172.21.15.2:0/1998059095 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb7980b1fb0 0x7fb7981512e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:25.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.512+0000 7fb79f418700 1 --2- 172.21.15.2:0/1998059095 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb77c06a0e0 0x7fb77c06c5a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:25.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.512+0000 7fb79f418700 1 --2- 172.21.15.2:0/1998059095 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb7980cdb70 0x7fb7981f5d90 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:25.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.512+0000 7fb79f418700 1 -- 172.21.15.2:0/1998059095 >> 172.21.15.2:0/1998059095 conn(0x7fb7980cfea0 msgr2=0x7fb79815cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:25.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.512+0000 7fb79f418700 1 -- 172.21.15.2:0/1998059095 shutdown_connections 2023-08-25T03:35:25.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.512+0000 7fb79f418700 1 -- 172.21.15.2:0/1998059095 wait complete. 2023-08-25T03:35:25.527 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-08-25T03:35:25.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.653+0000 7f96feb51700 1 Processor -- start 2023-08-25T03:35:25.655 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.653+0000 7f96feb51700 1 -- start start 2023-08-25T03:35:25.656 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.654+0000 7f96feb51700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f96f8163030 0x7f96f8165420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:25.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.654+0000 7f96feb51700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f8071b40 con 0x7f96f815e890 2023-08-25T03:35:25.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.654+0000 7f96feb51700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96f80720b0 con 0x7f96f80b19a0 2023-08-25T03:35:25.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.654+0000 7f96feb51700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f8071dd0 con 0x7f96f8163030 2023-08-25T03:35:25.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.654+0000 7f96f7fff700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f96f815e890 0x7f96f8071430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48580/0 (socket says 172.21.15.2:48580) 2023-08-25T03:35:25.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.654+0000 7f96f7fff700 1 -- 172.21.15.2:0/494301275 learned_addr learned my addr 172.21.15.2:0/494301275 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:25.657 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.654+0000 7f96fd0ee700 1 --2- 172.21.15.2:0/494301275 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f96f8163030 0x7f96f8165420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:25.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.654+0000 7f96f77fe700 1 -- 172.21.15.2:0/494301275 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4090447575 0 0) 0x7f96f8071b40 con 0x7f96f815e890 2023-08-25T03:35:25.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.654+0000 7f96f77fe700 1 -- 172.21.15.2:0/494301275 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96e80035b0 con 0x7f96f815e890 2023-08-25T03:35:25.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.655+0000 7f96fd0ee700 1 -- 172.21.15.2:0/494301275 >> v1:172.21.15.31:6789/0 conn(0x7f96f80b19a0 legacy=0x7f96f80b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:25.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.655+0000 7f96fd0ee700 1 -- 172.21.15.2:0/494301275 >> v1:172.21.15.2:6789/0 conn(0x7f96f815e890 legacy=0x7f96f8071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:25.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.655+0000 7f96f77fe700 1 -- 172.21.15.2:0/494301275 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 57317382 0 0) 0x7f96f80720b0 con 0x7f96f80b19a0 2023-08-25T03:35:25.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.655+0000 7f96fd0ee700 1 -- 172.21.15.2:0/494301275 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f815ddb0 con 0x7f96f8163030 2023-08-25T03:35:25.658 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.655+0000 7f96fd0ee700 1 --2- 172.21.15.2:0/494301275 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f96f8163030 0x7f96f8165420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009dc0 tx=0x7f96f000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5af7134aba74dd1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:25.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.655+0000 7f96f77fe700 1 -- 172.21.15.2:0/494301275 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96f000e070 con 0x7f96f8163030 2023-08-25T03:35:25.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.655+0000 7f96f77fe700 1 -- 172.21.15.2:0/494301275 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96f0003a20 con 0x7f96f8163030 2023-08-25T03:35:25.659 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.655+0000 7f96f77fe700 1 -- 172.21.15.2:0/494301275 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e40015f0 con 0x7f96f8163030 2023-08-25T03:35:25.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.655+0000 7f96feb51700 1 -- 172.21.15.2:0/494301275 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f96f8163030 msgr2=0x7f96f8165420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:25.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.655+0000 7f96feb51700 1 --2- 172.21.15.2:0/494301275 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f96f8163030 0x7f96f8165420 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009dc0 tx=0x7f96f000b330 comp rx=0 tx=0).stop 2023-08-25T03:35:25.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.656+0000 7f96feb51700 1 -- 172.21.15.2:0/494301275 shutdown_connections 2023-08-25T03:35:25.661 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.656+0000 7f96feb51700 1 --2- 172.21.15.2:0/494301275 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f96f8163030 0x7f96f8165420 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:25.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.656+0000 7f96feb51700 1 -- 172.21.15.2:0/494301275 >> 172.21.15.2:0/494301275 conn(0x7f96f80cfea0 msgr2=0x7f96f80d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:25.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.656+0000 7f96feb51700 1 -- 172.21.15.2:0/494301275 shutdown_connections 2023-08-25T03:35:25.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.656+0000 7f96feb51700 1 -- 172.21.15.2:0/494301275 wait complete. 2023-08-25T03:35:25.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.657+0000 7f96feb51700 1 Processor -- start 2023-08-25T03:35:25.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.657+0000 7f96feb51700 1 -- start start 2023-08-25T03:35:25.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.658+0000 7f96feb51700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96f80b19a0 0x7f96f80c6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:25.662 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.658+0000 7f96feb51700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96f815e890 0x7f96f80acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:25.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.658+0000 7f96feb51700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f96f8163030 0x7f96f80ad090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:25.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.658+0000 7f96feb51700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f80c6ad0 con 0x7f96f815e890 2023-08-25T03:35:25.663 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.658+0000 7f96feb51700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f8151900 con 0x7f96f80b19a0 2023-08-25T03:35:25.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.658+0000 7f96feb51700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f8151be0 con 0x7f96f8163030 2023-08-25T03:35:25.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.658+0000 7f96f7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96f815e890 0x7f96f80acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:25.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.658+0000 7f96f7fff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96f815e890 0x7f96f80acb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33688/0 (socket says 172.21.15.2:33688) 2023-08-25T03:35:25.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.658+0000 7f96f7fff700 1 -- 172.21.15.2:0/3576695167 learned_addr learned my addr 172.21.15.2:0/3576695167 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:25.664 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.658+0000 7f96fc8ed700 1 --2- 172.21.15.2:0/3576695167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96f80b19a0 0x7f96f80c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:25.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.658+0000 7f96f7fff700 1 -- 172.21.15.2:0/3576695167 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f96f8163030 msgr2=0x7f96f80ad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:25.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.658+0000 7f96f7fff700 1 --2- 172.21.15.2:0/3576695167 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f96f8163030 0x7f96f80ad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:25.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.658+0000 7f96f7fff700 1 -- 172.21.15.2:0/3576695167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96f80b19a0 msgr2=0x7f96f80c6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:25.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.658+0000 7f96f7fff700 1 --2- 172.21.15.2:0/3576695167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96f80b19a0 0x7f96f80c6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:25.665 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.658+0000 7f96f7fff700 1 -- 172.21.15.2:0/3576695167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f0009a70 con 0x7f96f815e890 2023-08-25T03:35:25.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.658+0000 7f96f7fff700 1 --2- 172.21.15.2:0/3576695167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96f815e890 0x7f96f80acb50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f96e400c730 tx=0x7f96e400caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:25.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.658+0000 7f96f57fa700 1 -- 172.21.15.2:0/3576695167 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e4004c80 con 0x7f96f815e890 2023-08-25T03:35:25.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.658+0000 7f96f57fa700 1 -- 172.21.15.2:0/3576695167 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e4004e00 con 0x7f96f815e890 2023-08-25T03:35:25.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.658+0000 7f96feb51700 1 -- 172.21.15.2:0/3576695167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96f8151ec0 con 0x7f96f815e890 2023-08-25T03:35:25.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.659+0000 7f96f57fa700 1 -- 172.21.15.2:0/3576695167 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e4007310 con 0x7f96f815e890 2023-08-25T03:35:25.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.659+0000 7f96feb51700 1 -- 172.21.15.2:0/3576695167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96f8152480 con 0x7f96f815e890 2023-08-25T03:35:25.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.659+0000 7f96feb51700 1 -- 172.21.15.2:0/3576695167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96f8066f70 con 0x7f96f815e890 2023-08-25T03:35:25.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.660+0000 7f96f57fa700 1 -- 172.21.15.2:0/3576695167 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f96e4007470 con 0x7f96f815e890 2023-08-25T03:35:25.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.660+0000 7f96f57fa700 1 --2- 172.21.15.2:0/3576695167 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f96dc06a0e0 0x7f96dc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:25.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.660+0000 7f96f57fa700 1 -- 172.21.15.2:0/3576695167 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f96e4088390 con 0x7f96f815e890 2023-08-25T03:35:25.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.660+0000 7f96fc8ed700 1 --2- 172.21.15.2:0/3576695167 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f96dc06a0e0 0x7f96dc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:25.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.661+0000 7f96fc8ed700 1 --2- 172.21.15.2:0/3576695167 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f96dc06a0e0 0x7f96dc06c5a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f96ec000f60 tx=0x7f96ec009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:25.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.665+0000 7f96f57fa700 1 -- 172.21.15.2:0/3576695167 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f96e4053a80 con 0x7f96f815e890 2023-08-25T03:35:25.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.912+0000 7f96feb51700 1 -- 172.21.15.2:0/3576695167 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f96f815e070 con 0x7f96f815e890 2023-08-25T03:35:25.921 INFO:tasks.ceph.mon.a.smithi002.stderr:2023-08-25T03:35:25.913+0000 7efd0737a700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-08-25T03:35:25.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.916+0000 7f96f57fa700 1 -- 172.21.15.2:0/3576695167 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f96f815e070 con 0x7f96f815e890 2023-08-25T03:35:25.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.918+0000 7f96feb51700 1 -- 172.21.15.2:0/3576695167 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f96dc06a0e0 msgr2=0x7f96dc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:25.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.918+0000 7f96feb51700 1 --2- 172.21.15.2:0/3576695167 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f96dc06a0e0 0x7f96dc06c5a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f96ec000f60 tx=0x7f96ec009040 comp rx=0 tx=0).stop 2023-08-25T03:35:25.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.918+0000 7f96feb51700 1 -- 172.21.15.2:0/3576695167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96f815e890 msgr2=0x7f96f80acb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:25.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.918+0000 7f96feb51700 1 --2- 172.21.15.2:0/3576695167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96f815e890 0x7f96f80acb50 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f96e400c730 tx=0x7f96e400caf0 comp rx=0 tx=0).stop 2023-08-25T03:35:25.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.919+0000 7f96feb51700 1 -- 172.21.15.2:0/3576695167 shutdown_connections 2023-08-25T03:35:25.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.919+0000 7f96feb51700 1 --2- 172.21.15.2:0/3576695167 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f96f8163030 0x7f96f80ad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:25.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.919+0000 7f96feb51700 1 --2- 172.21.15.2:0/3576695167 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96f80b19a0 0x7f96f80c6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:25.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.919+0000 7f96feb51700 1 --2- 172.21.15.2:0/3576695167 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f96dc06a0e0 0x7f96dc06c5a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:25.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.919+0000 7f96feb51700 1 --2- 172.21.15.2:0/3576695167 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f96f815e890 0x7f96f80acb50 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:25.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.919+0000 7f96feb51700 1 -- 172.21.15.2:0/3576695167 >> 172.21.15.2:0/3576695167 conn(0x7f96f80cfea0 msgr2=0x7f96f80afed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:25.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.919+0000 7f96feb51700 1 -- 172.21.15.2:0/3576695167 shutdown_connections 2023-08-25T03:35:25.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:25.919+0000 7f96feb51700 1 -- 172.21.15.2:0/3576695167 wait complete. 2023-08-25T03:35:25.925 INFO:teuthology.orchestra.run.smithi002.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-08-25T03:35:25.925 INFO:tasks.ceph.mds.d.smithi031.stderr: -3141> 2023-08-25T03:35:11.896+0000 7fb4142a2700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-08-25T03:35:25.933 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-08-25T03:35:25.952 INFO:tasks.ceph.mds.d.smithi031.stdout:starting mds.d at 2023-08-25T03:35:26.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.059+0000 7fe35cbfd700 1 Processor -- start 2023-08-25T03:35:26.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.059+0000 7fe35cbfd700 1 -- start start 2023-08-25T03:35:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.060+0000 7fe35cbfd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fe3581607f0 0x7fe358070e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.060+0000 7fe35cbfd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe3580af180 0x7fe3580ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.060+0000 7fe35cbfd700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe358160bd0 con 0x7fe35806a1e0 2023-08-25T03:35:26.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.060+0000 7fe35cbfd700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3580713a0 con 0x7fe3580af180 2023-08-25T03:35:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.060+0000 7fe35cbfd700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3580718a0 con 0x7fe3581607f0 2023-08-25T03:35:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.060+0000 7fe355d9b700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fe3580af180 0x7fe3580ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.060+0000 7fe356d9d700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fe35806a1e0 0x7fe35806ed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48596/0 (socket says 172.21.15.2:48596) 2023-08-25T03:35:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.060+0000 7fe356d9d700 1 -- 172.21.15.2:0/246966201 learned_addr learned my addr 172.21.15.2:0/246966201 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.060+0000 7fe35659c700 1 --2- 172.21.15.2:0/246966201 >> v2:172.21.15.31:3300/0 conn(0x7fe3581607f0 0x7fe358070e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.060+0000 7fe355d9b700 1 -- 172.21.15.2:0/246966201 >> v2:172.21.15.31:3300/0 conn(0x7fe3581607f0 msgr2=0x7fe358070e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:26.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.060+0000 7fe355d9b700 1 --2- 172.21.15.2:0/246966201 >> v2:172.21.15.31:3300/0 conn(0x7fe3581607f0 0x7fe358070e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:26.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.060+0000 7fe355d9b700 1 -- 172.21.15.2:0/246966201 >> v1:172.21.15.2:6789/0 conn(0x7fe35806a1e0 legacy=0x7fe35806ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:26.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.060+0000 7fe355d9b700 1 -- 172.21.15.2:0/246966201 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3580cdd00 con 0x7fe3580af180 2023-08-25T03:35:26.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.060+0000 7fe35559a700 1 -- 172.21.15.2:0/246966201 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2208280740 0 0) 0x7fe358160bd0 con 0x7fe35806a1e0 2023-08-25T03:35:26.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.060+0000 7fe355d9b700 1 --2- 172.21.15.2:0/246966201 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3580af180 0x7fe3580ac8e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe35806df30 tx=0x7fe34c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8cd463dd6853a2 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:26.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.060+0000 7fe35559a700 1 -- 172.21.15.2:0/246966201 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c011830 con 0x7fe3580af180 2023-08-25T03:35:26.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.061+0000 7fe35559a700 1 -- 172.21.15.2:0/246966201 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c011990 con 0x7fe3580af180 2023-08-25T03:35:26.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.061+0000 7fe35559a700 1 -- 172.21.15.2:0/246966201 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c011d20 con 0x7fe3580af180 2023-08-25T03:35:26.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.061+0000 7fe35cbfd700 1 -- 172.21.15.2:0/246966201 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3580af180 msgr2=0x7fe3580ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:26.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.061+0000 7fe35cbfd700 1 --2- 172.21.15.2:0/246966201 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3580af180 0x7fe3580ac8e0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe35806df30 tx=0x7fe34c009b20 comp rx=0 tx=0).stop 2023-08-25T03:35:26.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.061+0000 7fe35cbfd700 1 -- 172.21.15.2:0/246966201 shutdown_connections 2023-08-25T03:35:26.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.061+0000 7fe35cbfd700 1 --2- 172.21.15.2:0/246966201 >> v2:172.21.15.31:3300/0 conn(0x7fe3581607f0 0x7fe358070e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:26.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.061+0000 7fe35cbfd700 1 --2- 172.21.15.2:0/246966201 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3580af180 0x7fe3580ac8e0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:26.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.061+0000 7fe35cbfd700 1 -- 172.21.15.2:0/246966201 >> 172.21.15.2:0/246966201 conn(0x7fe3580cfd90 msgr2=0x7fe3580d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:26.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.061+0000 7fe35cbfd700 1 -- 172.21.15.2:0/246966201 shutdown_connections 2023-08-25T03:35:26.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.061+0000 7fe35cbfd700 1 -- 172.21.15.2:0/246966201 wait complete. 2023-08-25T03:35:26.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.062+0000 7fe35cbfd700 1 Processor -- start 2023-08-25T03:35:26.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.063+0000 7fe35cbfd700 1 -- start start 2023-08-25T03:35:26.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.063+0000 7fe35cbfd700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe35806a1e0 0x7fe3580c36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:26.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.063+0000 7fe35cbfd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3580af180 0x7fe3581f2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:26.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.063+0000 7fe35cbfd700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe3581607f0 0x7fe3581f2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:26.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.063+0000 7fe35cbfd700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3580c81d0 con 0x7fe3580af180 2023-08-25T03:35:26.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.063+0000 7fe35cbfd700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3580c40d0 con 0x7fe35806a1e0 2023-08-25T03:35:26.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.063+0000 7fe35cbfd700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3580c43b0 con 0x7fe3581607f0 2023-08-25T03:35:26.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.063+0000 7fe355d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3580af180 0x7fe3581f2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:26.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.063+0000 7fe355d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3580af180 0x7fe3581f2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33702/0 (socket says 172.21.15.2:33702) 2023-08-25T03:35:26.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.063+0000 7fe355d9b700 1 -- 172.21.15.2:0/262600785 learned_addr learned my addr 172.21.15.2:0/262600785 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:26.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.064+0000 7fe35659c700 1 --2- 172.21.15.2:0/262600785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe35806a1e0 0x7fe3580c36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:26.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.064+0000 7fe356d9d700 1 --2- 172.21.15.2:0/262600785 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe3581607f0 0x7fe3581f2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:26.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.064+0000 7fe355d9b700 1 -- 172.21.15.2:0/262600785 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe3581607f0 msgr2=0x7fe3581f2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:26.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.064+0000 7fe355d9b700 1 --2- 172.21.15.2:0/262600785 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe3581607f0 0x7fe3581f2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:26.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.064+0000 7fe355d9b700 1 -- 172.21.15.2:0/262600785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe35806a1e0 msgr2=0x7fe3580c36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:26.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.064+0000 7fe355d9b700 1 --2- 172.21.15.2:0/262600785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe35806a1e0 0x7fe3580c36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:26.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.064+0000 7fe355d9b700 1 -- 172.21.15.2:0/262600785 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34c0094a0 con 0x7fe3580af180 2023-08-25T03:35:26.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.064+0000 7fe356d9d700 1 --2- 172.21.15.2:0/262600785 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe3581607f0 0x7fe3581f2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:35:26.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.064+0000 7fe35659c700 1 --2- 172.21.15.2:0/262600785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe35806a1e0 0x7fe3580c36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:26.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.064+0000 7fe355d9b700 1 --2- 172.21.15.2:0/262600785 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3580af180 0x7fe3581f2690 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe34c000c00 tx=0x7fe34c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:26.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.064+0000 7fe346ffd700 1 -- 172.21.15.2:0/262600785 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c0036a0 con 0x7fe3580af180 2023-08-25T03:35:26.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.064+0000 7fe346ffd700 1 -- 172.21.15.2:0/262600785 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c01f070 con 0x7fe3580af180 2023-08-25T03:35:26.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.064+0000 7fe35cbfd700 1 -- 172.21.15.2:0/262600785 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3580c4690 con 0x7fe3580af180 2023-08-25T03:35:26.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.064+0000 7fe346ffd700 1 -- 172.21.15.2:0/262600785 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c003a60 con 0x7fe3580af180 2023-08-25T03:35:26.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.064+0000 7fe35cbfd700 1 -- 172.21.15.2:0/262600785 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35815a720 con 0x7fe3580af180 2023-08-25T03:35:26.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.065+0000 7fe35cbfd700 1 -- 172.21.15.2:0/262600785 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3580c7f30 con 0x7fe3580af180 2023-08-25T03:35:26.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.065+0000 7fe346ffd700 1 -- 172.21.15.2:0/262600785 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe34c003bc0 con 0x7fe3580af180 2023-08-25T03:35:26.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.066+0000 7fe346ffd700 1 --2- 172.21.15.2:0/262600785 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe33806a0e0 0x7fe33806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:26.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.066+0000 7fe346ffd700 1 -- 172.21.15.2:0/262600785 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fe34c021070 con 0x7fe3580af180 2023-08-25T03:35:26.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.066+0000 7fe35659c700 1 --2- 172.21.15.2:0/262600785 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe33806a0e0 0x7fe33806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:26.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.066+0000 7fe35659c700 1 --2- 172.21.15.2:0/262600785 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe33806a0e0 0x7fe33806c5a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe348007a50 tx=0x7fe348008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:26.076 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.070+0000 7fe346ffd700 1 -- 172.21.15.2:0/262600785 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe3580c7f30 con 0x7fe3580af180 2023-08-25T03:35:26.320 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.318+0000 7fe35cbfd700 1 -- 172.21.15.2:0/262600785 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe3580c8310 con 0x7fe3580af180 2023-08-25T03:35:26.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.918+0000 7fe346ffd700 1 -- 172.21.15.2:0/262600785 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe3580c8310 con 0x7fe3580af180 2023-08-25T03:35:26.922 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.920+0000 7fe35cbfd700 1 -- 172.21.15.2:0/262600785 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe33806a0e0 msgr2=0x7fe33806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:26.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.920+0000 7fe35cbfd700 1 --2- 172.21.15.2:0/262600785 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe33806a0e0 0x7fe33806c5a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe348007a50 tx=0x7fe348008040 comp rx=0 tx=0).stop 2023-08-25T03:35:26.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.920+0000 7fe35cbfd700 1 -- 172.21.15.2:0/262600785 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3580af180 msgr2=0x7fe3581f2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:26.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.920+0000 7fe35cbfd700 1 --2- 172.21.15.2:0/262600785 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3580af180 0x7fe3581f2690 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe34c000c00 tx=0x7fe34c004750 comp rx=0 tx=0).stop 2023-08-25T03:35:26.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.921+0000 7fe35cbfd700 1 -- 172.21.15.2:0/262600785 shutdown_connections 2023-08-25T03:35:26.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.921+0000 7fe35cbfd700 1 --2- 172.21.15.2:0/262600785 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe3581607f0 0x7fe3581f2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:26.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.921+0000 7fe35cbfd700 1 --2- 172.21.15.2:0/262600785 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe35806a1e0 0x7fe3580c36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:26.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.921+0000 7fe35cbfd700 1 --2- 172.21.15.2:0/262600785 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe33806a0e0 0x7fe33806c5a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:26.924 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.921+0000 7fe35cbfd700 1 --2- 172.21.15.2:0/262600785 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe3580af180 0x7fe3581f2690 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:26.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.921+0000 7fe35cbfd700 1 -- 172.21.15.2:0/262600785 >> 172.21.15.2:0/262600785 conn(0x7fe3580cfd90 msgr2=0x7fe35815edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:26.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.921+0000 7fe35cbfd700 1 -- 172.21.15.2:0/262600785 shutdown_connections 2023-08-25T03:35:26.925 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:26.921+0000 7fe35cbfd700 1 -- 172.21.15.2:0/262600785 wait complete. 2023-08-25T03:35:26.938 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-08-25T03:35:27.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.072+0000 7fd7448ed700 1 Processor -- start 2023-08-25T03:35:27.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.072+0000 7fd7448ed700 1 -- start start 2023-08-25T03:35:27.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.073+0000 7fd7448ed700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd74006f250 0x7fd74006d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:27.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.073+0000 7fd7448ed700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd74015fb30 con 0x7fd74015b030 2023-08-25T03:35:27.083 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.073+0000 7fd7448ed700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd74015eca0 con 0x7fd74006a8f0 2023-08-25T03:35:27.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.073+0000 7fd7448ed700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd74006f6c0 con 0x7fd74006f250 2023-08-25T03:35:27.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.073+0000 7fd73e59c700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fd74015b030 0x7fd74006a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48606/0 (socket says 172.21.15.2:48606) 2023-08-25T03:35:27.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.073+0000 7fd73e59c700 1 -- 172.21.15.2:0/2269206321 learned_addr learned my addr 172.21.15.2:0/2269206321 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:27.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.073+0000 7fd73d59a700 1 -- 172.21.15.2:0/2269206321 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2493471945 0 0) 0x7fd74015fb30 con 0x7fd74015b030 2023-08-25T03:35:27.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.073+0000 7fd73ed9d700 1 --2- 172.21.15.2:0/2269206321 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd74006f250 0x7fd74006d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:27.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.073+0000 7fd73d59a700 1 -- 172.21.15.2:0/2269206321 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7280035b0 con 0x7fd74015b030 2023-08-25T03:35:27.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.074+0000 7fd73d59a700 1 -- 172.21.15.2:0/2269206321 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2621087133 0 0) 0x7fd7280035b0 con 0x7fd74015b030 2023-08-25T03:35:27.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.074+0000 7fd73d59a700 1 -- 172.21.15.2:0/2269206321 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd74006f250 msgr2=0x7fd74006d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:27.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.074+0000 7fd73d59a700 1 --2- 172.21.15.2:0/2269206321 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd74006f250 0x7fd74006d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:27.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.074+0000 7fd73d59a700 1 -- 172.21.15.2:0/2269206321 >> v1:172.21.15.31:6789/0 conn(0x7fd74006a8f0 legacy=0x7fd7400ca940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:27.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.074+0000 7fd73d59a700 1 -- 172.21.15.2:0/2269206321 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74015ef20 con 0x7fd74015b030 2023-08-25T03:35:27.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.074+0000 7fd73d59a700 1 -- 172.21.15.2:0/2269206321 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7fd734001670 con 0x7fd74015b030 2023-08-25T03:35:27.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.074+0000 7fd73d59a700 1 -- 172.21.15.2:0/2269206321 >> v1:172.21.15.2:6789/0 conn(0x7fd74015b030 legacy=0x7fd74006a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:27.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.074+0000 7fd73d59a700 1 --2- 172.21.15.2:0/2269206321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd728003a30 0x7fd728003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:27.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.074+0000 7fd73d59a700 1 --2- 172.21.15.2:0/2269206321 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd74006f250 0x7fd728004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:27.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.074+0000 7fd73d59a700 1 --2- 172.21.15.2:0/2269206321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd728004d90 0x7fd728005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:27.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.074+0000 7fd73d59a700 1 -- 172.21.15.2:0/2269206321 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74006f6c0 con 0x7fd728003a30 2023-08-25T03:35:27.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.074+0000 7fd73d59a700 1 -- 172.21.15.2:0/2269206321 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72800bb40 con 0x7fd728004d90 2023-08-25T03:35:27.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.074+0000 7fd73d59a700 1 -- 172.21.15.2:0/2269206321 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd72800bd00 con 0x7fd74006f250 2023-08-25T03:35:27.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.074+0000 7fd73e59c700 1 --2- 172.21.15.2:0/2269206321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd728003a30 0x7fd728003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:27.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.074+0000 7fd73dd9b700 1 --2- 172.21.15.2:0/2269206321 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd74006f250 0x7fd728004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:27.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.074+0000 7fd73ed9d700 1 --2- 172.21.15.2:0/2269206321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd728004d90 0x7fd728005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:27.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.074+0000 7fd73e59c700 1 -- 172.21.15.2:0/2269206321 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd74006f250 msgr2=0x7fd728004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:27.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.074+0000 7fd73e59c700 1 --2- 172.21.15.2:0/2269206321 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd74006f250 0x7fd728004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:27.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.074+0000 7fd73e59c700 1 -- 172.21.15.2:0/2269206321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd728004d90 msgr2=0x7fd728005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:27.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.074+0000 7fd73e59c700 1 --2- 172.21.15.2:0/2269206321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd728004d90 0x7fd728005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:27.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.074+0000 7fd73e59c700 1 -- 172.21.15.2:0/2269206321 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7280035b0 con 0x7fd728003a30 2023-08-25T03:35:27.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.074+0000 7fd73e59c700 1 --2- 172.21.15.2:0/2269206321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd728003a30 0x7fd728003e50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd73400c240 tx=0x7fd728005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68285f2bc4607828 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:27.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.075+0000 7fd73d59a700 1 -- 172.21.15.2:0/2269206321 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd734007c10 con 0x7fd728003a30 2023-08-25T03:35:27.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.075+0000 7fd73d59a700 1 -- 172.21.15.2:0/2269206321 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd73400cd40 con 0x7fd728003a30 2023-08-25T03:35:27.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.075+0000 7fd73d59a700 1 -- 172.21.15.2:0/2269206321 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd734005ba0 con 0x7fd728003a30 2023-08-25T03:35:27.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.075+0000 7fd7448ed700 1 -- 172.21.15.2:0/2269206321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd728003a30 msgr2=0x7fd728003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:27.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.075+0000 7fd7448ed700 1 --2- 172.21.15.2:0/2269206321 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd728003a30 0x7fd728003e50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd73400c240 tx=0x7fd728005880 comp rx=0 tx=0).stop 2023-08-25T03:35:27.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.075+0000 7fd73ed9d700 1 -- 172.21.15.2:0/2269206321 reap_dead start 2023-08-25T03:35:27.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.075+0000 7fd7448ed700 1 -- 172.21.15.2:0/2269206321 shutdown_connections 2023-08-25T03:35:27.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.075+0000 7fd7448ed700 1 -- 172.21.15.2:0/2269206321 >> 172.21.15.2:0/2269206321 conn(0x7fd7400b1990 msgr2=0x7fd7400b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:27.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.075+0000 7fd7448ed700 1 -- 172.21.15.2:0/2269206321 shutdown_connections 2023-08-25T03:35:27.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.075+0000 7fd7448ed700 1 -- 172.21.15.2:0/2269206321 wait complete. 2023-08-25T03:35:27.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.077+0000 7fd7448ed700 1 Processor -- start 2023-08-25T03:35:27.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.077+0000 7fd7448ed700 1 -- start start 2023-08-25T03:35:27.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.077+0000 7fd7448ed700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd728003a30 0x7fd74015a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:27.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.077+0000 7fd7448ed700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd74015dc20 0x7fd7400c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:27.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.077+0000 7fd7448ed700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd74015e0a0 0x7fd7400c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:27.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.077+0000 7fd7448ed700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74015b760 con 0x7fd728003a30 2023-08-25T03:35:27.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.077+0000 7fd7448ed700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7400c6f70 con 0x7fd74015dc20 2023-08-25T03:35:27.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.077+0000 7fd7448ed700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7400c67e0 con 0x7fd74015e0a0 2023-08-25T03:35:27.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.077+0000 7fd73e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd728003a30 0x7fd74015a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:27.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.077+0000 7fd73e59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd728003a30 0x7fd74015a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33722/0 (socket says 172.21.15.2:33722) 2023-08-25T03:35:27.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.078+0000 7fd73e59c700 1 -- 172.21.15.2:0/1610951550 learned_addr learned my addr 172.21.15.2:0/1610951550 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:27.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.078+0000 7fd73ed9d700 1 --2- 172.21.15.2:0/1610951550 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd74015e0a0 0x7fd7400c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:27.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.078+0000 7fd73dd9b700 1 --2- 172.21.15.2:0/1610951550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd74015dc20 0x7fd7400c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:27.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.078+0000 7fd73e59c700 1 -- 172.21.15.2:0/1610951550 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd74015e0a0 msgr2=0x7fd7400c62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:27.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.078+0000 7fd73e59c700 1 --2- 172.21.15.2:0/1610951550 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd74015e0a0 0x7fd7400c62a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:27.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.078+0000 7fd73e59c700 1 -- 172.21.15.2:0/1610951550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd74015dc20 msgr2=0x7fd7400c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:27.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.078+0000 7fd73e59c700 1 --2- 172.21.15.2:0/1610951550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd74015dc20 0x7fd7400c5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:27.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.078+0000 7fd73e59c700 1 -- 172.21.15.2:0/1610951550 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7400c7210 con 0x7fd728003a30 2023-08-25T03:35:27.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.078+0000 7fd73e59c700 1 --2- 172.21.15.2:0/1610951550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd728003a30 0x7fd74015a220 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd73400c830 tx=0x7fd734001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:27.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.078+0000 7fd72effd700 1 -- 172.21.15.2:0/1610951550 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7340072d0 con 0x7fd728003a30 2023-08-25T03:35:27.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.078+0000 7fd72effd700 1 -- 172.21.15.2:0/1610951550 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd734007430 con 0x7fd728003a30 2023-08-25T03:35:27.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.078+0000 7fd7448ed700 1 -- 172.21.15.2:0/1610951550 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7400c74f0 con 0x7fd728003a30 2023-08-25T03:35:27.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.078+0000 7fd72effd700 1 -- 172.21.15.2:0/1610951550 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7340077e0 con 0x7fd728003a30 2023-08-25T03:35:27.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.078+0000 7fd7448ed700 1 -- 172.21.15.2:0/1610951550 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7400c7ab0 con 0x7fd728003a30 2023-08-25T03:35:27.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.080+0000 7fd7448ed700 1 -- 172.21.15.2:0/1610951550 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd740066f70 con 0x7fd728003a30 2023-08-25T03:35:27.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.080+0000 7fd72effd700 1 -- 172.21.15.2:0/1610951550 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd734019030 con 0x7fd728003a30 2023-08-25T03:35:27.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.080+0000 7fd72effd700 1 --2- 172.21.15.2:0/1610951550 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd72006a0e0 0x7fd72006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:27.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.080+0000 7fd72effd700 1 -- 172.21.15.2:0/1610951550 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6148+0+0 (secure 0 0 0) 0x7fd7340896d0 con 0x7fd728003a30 2023-08-25T03:35:27.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.080+0000 7fd73dd9b700 1 --2- 172.21.15.2:0/1610951550 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd72006a0e0 0x7fd72006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:27.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.080+0000 7fd73dd9b700 1 --2- 172.21.15.2:0/1610951550 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd72006a0e0 0x7fd72006c5a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd730000c00 tx=0x7fd730009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:27.158 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.085+0000 7fd72effd700 1 -- 172.21.15.2:0/1610951550 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd734054df0 con 0x7fd728003a30 2023-08-25T03:35:27.286 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.284+0000 7fd7448ed700 1 -- 172.21.15.2:0/1610951550 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd7400746d0 con 0x7fd728003a30 2023-08-25T03:35:27.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.921+0000 7fd72effd700 1 -- 172.21.15.2:0/1610951550 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v39) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd734087e60 con 0x7fd728003a30 2023-08-25T03:35:27.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.926+0000 7fd7448ed700 1 -- 172.21.15.2:0/1610951550 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd72006a0e0 msgr2=0x7fd72006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:27.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.926+0000 7fd7448ed700 1 --2- 172.21.15.2:0/1610951550 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd72006a0e0 0x7fd72006c5a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd730000c00 tx=0x7fd730009040 comp rx=0 tx=0).stop 2023-08-25T03:35:27.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.926+0000 7fd7448ed700 1 -- 172.21.15.2:0/1610951550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd728003a30 msgr2=0x7fd74015a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:27.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.926+0000 7fd7448ed700 1 --2- 172.21.15.2:0/1610951550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd728003a30 0x7fd74015a220 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd73400c830 tx=0x7fd734001a00 comp rx=0 tx=0).stop 2023-08-25T03:35:27.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.926+0000 7fd7448ed700 1 -- 172.21.15.2:0/1610951550 shutdown_connections 2023-08-25T03:35:27.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.926+0000 7fd7448ed700 1 --2- 172.21.15.2:0/1610951550 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd72006a0e0 0x7fd72006c5a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:27.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.926+0000 7fd7448ed700 1 --2- 172.21.15.2:0/1610951550 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd728003a30 0x7fd74015a220 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:27.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.926+0000 7fd7448ed700 1 --2- 172.21.15.2:0/1610951550 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd74015e0a0 0x7fd7400c62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:27.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.926+0000 7fd7448ed700 1 --2- 172.21.15.2:0/1610951550 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd74015dc20 0x7fd7400c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:27.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.926+0000 7fd7448ed700 1 -- 172.21.15.2:0/1610951550 >> 172.21.15.2:0/1610951550 conn(0x7fd7400b1990 msgr2=0x7fd7400b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:27.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.927+0000 7fd7448ed700 1 -- 172.21.15.2:0/1610951550 shutdown_connections 2023-08-25T03:35:27.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:27.927+0000 7fd7448ed700 1 -- 172.21.15.2:0/1610951550 wait complete. 2023-08-25T03:35:27.930 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_metadata' removed 2023-08-25T03:35:27.944 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-08-25T03:35:28.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.080+0000 7f0566971700 1 Processor -- start 2023-08-25T03:35:28.082 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.081+0000 7f0566971700 1 -- start start 2023-08-25T03:35:28.084 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.081+0000 7f0566971700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f05600cb1d0 0x7f0560160f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:28.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.081+0000 7f0566971700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05600cddc0 con 0x7f05601614d0 2023-08-25T03:35:28.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.081+0000 7f0566971700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0560161a50 con 0x7f05600cadf0 2023-08-25T03:35:28.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.081+0000 7f0566971700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0560161d30 con 0x7f05600cb1d0 2023-08-25T03:35:28.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.081+0000 7f0564f0e700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f05601614d0 0x7f056006a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48630/0 (socket says 172.21.15.2:48630) 2023-08-25T03:35:28.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.081+0000 7f0564f0e700 1 -- 172.21.15.2:0/1513027342 learned_addr learned my addr 172.21.15.2:0/1513027342 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:28.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.082+0000 7f055effd700 1 -- 172.21.15.2:0/1513027342 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 966642969 0 0) 0x7f05600cddc0 con 0x7f05601614d0 2023-08-25T03:35:28.085 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.082+0000 7f055effd700 1 -- 172.21.15.2:0/1513027342 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f054c0035b0 con 0x7f05601614d0 2023-08-25T03:35:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.082+0000 7f055f7fe700 1 --2- 172.21.15.2:0/1513027342 >> v2:172.21.15.31:3300/0 conn(0x7f05600cb1d0 0x7f0560160f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.082+0000 7f055effd700 1 -- 172.21.15.2:0/1513027342 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2753642977 0 0) 0x7f054c0035b0 con 0x7f05601614d0 2023-08-25T03:35:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.082+0000 7f055effd700 1 -- 172.21.15.2:0/1513027342 >> v2:172.21.15.31:3300/0 conn(0x7f05600cb1d0 msgr2=0x7f0560160f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.082+0000 7f055effd700 1 --2- 172.21.15.2:0/1513027342 >> v2:172.21.15.31:3300/0 conn(0x7f05600cb1d0 0x7f0560160f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.082+0000 7f055effd700 1 -- 172.21.15.2:0/1513027342 >> v1:172.21.15.31:6789/0 conn(0x7f05600cadf0 legacy=0x7f05600ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.082+0000 7f055effd700 1 -- 172.21.15.2:0/1513027342 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05600ce080 con 0x7f05601614d0 2023-08-25T03:35:28.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.082+0000 7f055effd700 1 -- 172.21.15.2:0/1513027342 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f05580019f0 con 0x7f05601614d0 2023-08-25T03:35:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.082+0000 7f055effd700 1 -- 172.21.15.2:0/1513027342 >> v1:172.21.15.2:6789/0 conn(0x7f05601614d0 legacy=0x7f056006a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.082+0000 7f055effd700 1 --2- 172.21.15.2:0/1513027342 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f054c003a00 0x7f054c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.082+0000 7f055effd700 1 --2- 172.21.15.2:0/1513027342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f054c004850 0x7f054c004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.082+0000 7f055effd700 1 --2- 172.21.15.2:0/1513027342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f054c005210 0x7f054c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.082+0000 7f055effd700 1 -- 172.21.15.2:0/1513027342 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0560161d30 con 0x7f054c004850 2023-08-25T03:35:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.082+0000 7f055effd700 1 -- 172.21.15.2:0/1513027342 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c00c160 con 0x7f054c005210 2023-08-25T03:35:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.082+0000 7f055effd700 1 -- 172.21.15.2:0/1513027342 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054c00c320 con 0x7f054c003a00 2023-08-25T03:35:28.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.082+0000 7f055f7fe700 1 --2- 172.21.15.2:0/1513027342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f054c004850 0x7f054c004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.082+0000 7f055ffff700 1 --2- 172.21.15.2:0/1513027342 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f054c003a00 0x7f054c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.082+0000 7f0564f0e700 1 --2- 172.21.15.2:0/1513027342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f054c005210 0x7f054c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.082+0000 7f055f7fe700 1 -- 172.21.15.2:0/1513027342 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f054c003a00 msgr2=0x7f054c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.082+0000 7f055f7fe700 1 --2- 172.21.15.2:0/1513027342 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f054c003a00 0x7f054c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.083+0000 7f055f7fe700 1 -- 172.21.15.2:0/1513027342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f054c005210 msgr2=0x7f054c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.083+0000 7f055f7fe700 1 --2- 172.21.15.2:0/1513027342 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f054c005210 0x7f054c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:28.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.083+0000 7f055f7fe700 1 -- 172.21.15.2:0/1513027342 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c0035b0 con 0x7f054c004850 2023-08-25T03:35:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.083+0000 7f0564f0e700 1 -- 172.21.15.2:0/1513027342 reap_dead start 2023-08-25T03:35:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.083+0000 7f055f7fe700 1 --2- 172.21.15.2:0/1513027342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f054c004850 0x7f054c004cd0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0550009890 tx=0x7f054c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540544267b66c369 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.083+0000 7f055effd700 1 -- 172.21.15.2:0/1513027342 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f055000c070 con 0x7f054c004850 2023-08-25T03:35:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.083+0000 7f055effd700 1 -- 172.21.15.2:0/1513027342 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0550013610 con 0x7f054c004850 2023-08-25T03:35:28.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.083+0000 7f055effd700 1 -- 172.21.15.2:0/1513027342 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05500139a0 con 0x7f054c004850 2023-08-25T03:35:28.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.083+0000 7f0566971700 1 -- 172.21.15.2:0/1513027342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f054c004850 msgr2=0x7f054c004cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:28.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.083+0000 7f0566971700 1 --2- 172.21.15.2:0/1513027342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f054c004850 0x7f054c004cd0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0550009890 tx=0x7f054c005880 comp rx=0 tx=0).stop 2023-08-25T03:35:28.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.083+0000 7f0566971700 1 -- 172.21.15.2:0/1513027342 shutdown_connections 2023-08-25T03:35:28.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.083+0000 7f0566971700 1 --2- 172.21.15.2:0/1513027342 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f054c004850 0x7f054c004cd0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:28.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.083+0000 7f0566971700 1 -- 172.21.15.2:0/1513027342 >> 172.21.15.2:0/1513027342 conn(0x7f05600b1990 msgr2=0x7f05600b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:28.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.083+0000 7f0566971700 1 -- 172.21.15.2:0/1513027342 shutdown_connections 2023-08-25T03:35:28.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.084+0000 7f0566971700 1 -- 172.21.15.2:0/1513027342 wait complete. 2023-08-25T03:35:28.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.085+0000 7f0566971700 1 Processor -- start 2023-08-25T03:35:28.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.085+0000 7f0566971700 1 -- start start 2023-08-25T03:35:28.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.086+0000 7f0566971700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f054c004850 0x7f05600b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:28.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.086+0000 7f0566971700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0560158e00 0x7f056006dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:28.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.086+0000 7f0566971700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f05601f6a30 0x7f05600c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:28.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.086+0000 7f0566971700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05601f62a0 con 0x7f0560158e00 2023-08-25T03:35:28.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.086+0000 7f0566971700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05600c66c0 con 0x7f054c004850 2023-08-25T03:35:28.093 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.086+0000 7f0566971700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05600c69a0 con 0x7f05601f6a30 2023-08-25T03:35:28.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.086+0000 7f055f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0560158e00 0x7f056006dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:28.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.086+0000 7f055f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0560158e00 0x7f056006dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33742/0 (socket says 172.21.15.2:33742) 2023-08-25T03:35:28.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.086+0000 7f055f7fe700 1 -- 172.21.15.2:0/1472640162 learned_addr learned my addr 172.21.15.2:0/1472640162 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:28.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.087+0000 7f055f7fe700 1 -- 172.21.15.2:0/1472640162 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f05601f6a30 msgr2=0x7f05600c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:28.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.087+0000 7f0564f0e700 1 --2- 172.21.15.2:0/1472640162 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f05601f6a30 0x7f05600c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:28.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.087+0000 7f055ffff700 1 --2- 172.21.15.2:0/1472640162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f054c004850 0x7f05600b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:28.094 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.087+0000 7f055f7fe700 1 --2- 172.21.15.2:0/1472640162 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f05601f6a30 0x7f05600c79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:28.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.087+0000 7f055f7fe700 1 -- 172.21.15.2:0/1472640162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f054c004850 msgr2=0x7f05600b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:28.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.087+0000 7f055f7fe700 1 --2- 172.21.15.2:0/1472640162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f054c004850 0x7f05600b3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:28.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.087+0000 7f055f7fe700 1 -- 172.21.15.2:0/1472640162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05601f6510 con 0x7f0560158e00 2023-08-25T03:35:28.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.087+0000 7f055f7fe700 1 --2- 172.21.15.2:0/1472640162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0560158e00 0x7f056006dcc0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0550016040 tx=0x7f05500047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:28.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.087+0000 7f055cff9700 1 -- 172.21.15.2:0/1472640162 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0550004520 con 0x7f0560158e00 2023-08-25T03:35:28.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.087+0000 7f055cff9700 1 -- 172.21.15.2:0/1472640162 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05500036a0 con 0x7f0560158e00 2023-08-25T03:35:28.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.087+0000 7f0566971700 1 -- 172.21.15.2:0/1472640162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05600c6c80 con 0x7f0560158e00 2023-08-25T03:35:28.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.087+0000 7f055cff9700 1 -- 172.21.15.2:0/1472640162 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0550003a50 con 0x7f0560158e00 2023-08-25T03:35:28.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.087+0000 7f0566971700 1 -- 172.21.15.2:0/1472640162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f056006e200 con 0x7f0560158e00 2023-08-25T03:35:28.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.088+0000 7f0566971700 1 -- 172.21.15.2:0/1472640162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f056006e3f0 con 0x7f0560158e00 2023-08-25T03:35:28.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.088+0000 7f055cff9700 1 -- 172.21.15.2:0/1472640162 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f0550003bb0 con 0x7f0560158e00 2023-08-25T03:35:28.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.089+0000 7f055cff9700 1 --2- 172.21.15.2:0/1472640162 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f054006a0e0 0x7f054006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:28.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.089+0000 7f055cff9700 1 -- 172.21.15.2:0/1472640162 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5710+0+0 (secure 0 0 0) 0x7f05500894c0 con 0x7f0560158e00 2023-08-25T03:35:28.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.089+0000 7f055ffff700 1 --2- 172.21.15.2:0/1472640162 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f054006a0e0 0x7f054006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:28.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.089+0000 7f055ffff700 1 --2- 172.21.15.2:0/1472640162 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f054006a0e0 0x7f054006c5a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0548000c00 tx=0x7f0548009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:28.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.094+0000 7f055cff9700 1 -- 172.21.15.2:0/1472640162 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f0550054da0 con 0x7f0560158e00 2023-08-25T03:35:28.299 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.297+0000 7f0566971700 1 -- 172.21.15.2:0/1472640162 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0560066f70 con 0x7f0560158e00 2023-08-25T03:35:28.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.922+0000 7f055cff9700 1 -- 172.21.15.2:0/1472640162 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v40) v1 ==== 169+0+0 (secure 0 0 0) 0x7f05500583c0 con 0x7f0560158e00 2023-08-25T03:35:28.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.928+0000 7f0566971700 1 -- 172.21.15.2:0/1472640162 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f054006a0e0 msgr2=0x7f054006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:28.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.928+0000 7f0566971700 1 --2- 172.21.15.2:0/1472640162 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f054006a0e0 0x7f054006c5a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0548000c00 tx=0x7f0548009040 comp rx=0 tx=0).stop 2023-08-25T03:35:28.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.928+0000 7f0566971700 1 -- 172.21.15.2:0/1472640162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0560158e00 msgr2=0x7f056006dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:28.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.928+0000 7f0566971700 1 --2- 172.21.15.2:0/1472640162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0560158e00 0x7f056006dcc0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0550016040 tx=0x7f05500047c0 comp rx=0 tx=0).stop 2023-08-25T03:35:28.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.928+0000 7f0566971700 1 -- 172.21.15.2:0/1472640162 shutdown_connections 2023-08-25T03:35:28.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.928+0000 7f0566971700 1 --2- 172.21.15.2:0/1472640162 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f05601f6a30 0x7f05600c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:28.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.928+0000 7f0566971700 1 --2- 172.21.15.2:0/1472640162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f054c004850 0x7f05600b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:28.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.928+0000 7f0566971700 1 --2- 172.21.15.2:0/1472640162 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f054006a0e0 0x7f054006c5a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:28.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.928+0000 7f0566971700 1 --2- 172.21.15.2:0/1472640162 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0560158e00 0x7f056006dcc0 secure :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f0550016040 tx=0x7f05500047c0 comp rx=0 tx=0).stop 2023-08-25T03:35:28.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.929+0000 7f0566971700 1 -- 172.21.15.2:0/1472640162 >> 172.21.15.2:0/1472640162 conn(0x7f05600b1990 msgr2=0x7f056015f630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:28.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.929+0000 7f0566971700 1 -- 172.21.15.2:0/1472640162 shutdown_connections 2023-08-25T03:35:28.932 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:28.929+0000 7f0566971700 1 -- 172.21.15.2:0/1472640162 wait complete. 2023-08-25T03:35:28.934 INFO:teuthology.orchestra.run.smithi002.stderr:pool 'cephfs_data' removed 2023-08-25T03:35:28.946 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-08-25T03:35:29.086 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.083+0000 7fb536306700 1 Processor -- start 2023-08-25T03:35:29.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.083+0000 7fb536306700 1 -- start start 2023-08-25T03:35:29.087 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.083+0000 7fb536306700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fb5300b3d80 0x7fb5300d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:29.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.083+0000 7fb536306700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5300cc470 con 0x7fb53006a830 2023-08-25T03:35:29.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.083+0000 7fb536306700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5300cbe90 con 0x7fb53006a450 2023-08-25T03:35:29.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.083+0000 7fb536306700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb53006dd20 con 0x7fb5300b3d80 2023-08-25T03:35:29.088 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.084+0000 7fb5348a3700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fb5300b3d80 0x7fb5300d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:29.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.084+0000 7fb52f7fe700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fb53006a830 0x7fb53006d610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48648/0 (socket says 172.21.15.2:48648) 2023-08-25T03:35:29.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.084+0000 7fb52f7fe700 1 -- 172.21.15.2:0/1690533460 learned_addr learned my addr 172.21.15.2:0/1690533460 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:29.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.084+0000 7fb5348a3700 1 -- 172.21.15.2:0/1690533460 >> v1:172.21.15.31:6789/0 conn(0x7fb53006a450 legacy=0x7fb53006cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:29.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.084+0000 7fb5348a3700 1 -- 172.21.15.2:0/1690533460 >> v1:172.21.15.2:6789/0 conn(0x7fb53006a830 legacy=0x7fb53006d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:29.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.084+0000 7fb5348a3700 1 -- 172.21.15.2:0/1690533460 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5300ca3e0 con 0x7fb5300b3d80 2023-08-25T03:35:29.089 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.084+0000 7fb52effd700 1 -- 172.21.15.2:0/1690533460 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 347070790 0 0) 0x7fb5300cc470 con 0x7fb53006a830 2023-08-25T03:35:29.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.084+0000 7fb5348a3700 1 --2- 172.21.15.2:0/1690533460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5300b3d80 0x7fb5300d08b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb528009d90 tx=0x7fb52800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a4c3b209c75a73 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:29.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.084+0000 7fb52effd700 1 -- 172.21.15.2:0/1690533460 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb52800e070 con 0x7fb5300b3d80 2023-08-25T03:35:29.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.084+0000 7fb52effd700 1 -- 172.21.15.2:0/1690533460 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52800bbd0 con 0x7fb5300b3d80 2023-08-25T03:35:29.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.084+0000 7fb52effd700 1 -- 172.21.15.2:0/1690533460 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb528003a70 con 0x7fb5300b3d80 2023-08-25T03:35:29.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.085+0000 7fb536306700 1 -- 172.21.15.2:0/1690533460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5300b3d80 msgr2=0x7fb5300d08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:29.090 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.085+0000 7fb536306700 1 --2- 172.21.15.2:0/1690533460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5300b3d80 0x7fb5300d08b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb528009d90 tx=0x7fb52800b330 comp rx=0 tx=0).stop 2023-08-25T03:35:29.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.085+0000 7fb536306700 1 -- 172.21.15.2:0/1690533460 shutdown_connections 2023-08-25T03:35:29.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.085+0000 7fb536306700 1 --2- 172.21.15.2:0/1690533460 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb5300b3d80 0x7fb5300d08b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:29.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.085+0000 7fb536306700 1 -- 172.21.15.2:0/1690533460 >> 172.21.15.2:0/1690533460 conn(0x7fb5300cfd90 msgr2=0x7fb5300d01a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:29.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.085+0000 7fb536306700 1 -- 172.21.15.2:0/1690533460 shutdown_connections 2023-08-25T03:35:29.091 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.085+0000 7fb536306700 1 -- 172.21.15.2:0/1690533460 wait complete. 2023-08-25T03:35:29.092 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.086+0000 7fb536306700 1 Processor -- start 2023-08-25T03:35:29.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.086+0000 7fb536306700 1 -- start start 2023-08-25T03:35:29.095 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.087+0000 7fb536306700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb53006a450 0x7fb5300cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:29.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.087+0000 7fb536306700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb53006a830 0x7fb5301f26c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:29.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.087+0000 7fb536306700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5300b3d80 0x7fb5300cd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:29.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.087+0000 7fb536306700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb53014ceb0 con 0x7fb53006a830 2023-08-25T03:35:29.096 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.087+0000 7fb536306700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5300ce120 con 0x7fb5300b3d80 2023-08-25T03:35:29.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.087+0000 7fb536306700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5300ce400 con 0x7fb53006a450 2023-08-25T03:35:29.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.087+0000 7fb52f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb53006a830 0x7fb5301f26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:29.097 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.087+0000 7fb52f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb53006a830 0x7fb5301f26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33754/0 (socket says 172.21.15.2:33754) 2023-08-25T03:35:29.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.087+0000 7fb52f7fe700 1 -- 172.21.15.2:0/362347271 learned_addr learned my addr 172.21.15.2:0/362347271 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:29.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.087+0000 7fb5348a3700 1 --2- 172.21.15.2:0/362347271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5300b3d80 0x7fb5300cd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:29.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.087+0000 7fb52ffff700 1 --2- 172.21.15.2:0/362347271 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb53006a450 0x7fb5300cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:29.098 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.087+0000 7fb52f7fe700 1 -- 172.21.15.2:0/362347271 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb53006a450 msgr2=0x7fb5300cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:29.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.087+0000 7fb52f7fe700 1 --2- 172.21.15.2:0/362347271 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb53006a450 0x7fb5300cf1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:29.099 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.087+0000 7fb52f7fe700 1 -- 172.21.15.2:0/362347271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5300b3d80 msgr2=0x7fb5300cd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:29.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.087+0000 7fb52f7fe700 1 --2- 172.21.15.2:0/362347271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5300b3d80 0x7fb5300cd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:29.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.087+0000 7fb52f7fe700 1 -- 172.21.15.2:0/362347271 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb528009a70 con 0x7fb53006a830 2023-08-25T03:35:29.100 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.087+0000 7fb52f7fe700 1 --2- 172.21.15.2:0/362347271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb53006a830 0x7fb5301f26c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb5240011e0 tx=0x7fb52400cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:29.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.088+0000 7fb52cff9700 1 -- 172.21.15.2:0/362347271 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb524004c80 con 0x7fb53006a830 2023-08-25T03:35:29.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.088+0000 7fb52cff9700 1 -- 172.21.15.2:0/362347271 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb524004e00 con 0x7fb53006a830 2023-08-25T03:35:29.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.088+0000 7fb536306700 1 -- 172.21.15.2:0/362347271 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5300ce6e0 con 0x7fb53006a830 2023-08-25T03:35:29.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.088+0000 7fb52cff9700 1 -- 172.21.15.2:0/362347271 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb524007310 con 0x7fb53006a830 2023-08-25T03:35:29.101 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.088+0000 7fb536306700 1 -- 172.21.15.2:0/362347271 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5300ceca0 con 0x7fb53006a830 2023-08-25T03:35:29.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.089+0000 7fb536306700 1 -- 172.21.15.2:0/362347271 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5300ceec0 con 0x7fb53006a830 2023-08-25T03:35:29.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.089+0000 7fb52cff9700 1 -- 172.21.15.2:0/362347271 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fb524007470 con 0x7fb53006a830 2023-08-25T03:35:29.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.090+0000 7fb52cff9700 1 --2- 172.21.15.2:0/362347271 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb51c06a0e0 0x7fb51c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:29.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.090+0000 7fb52cff9700 1 -- 172.21.15.2:0/362347271 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fb524087fc0 con 0x7fb53006a830 2023-08-25T03:35:29.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.090+0000 7fb52ffff700 1 --2- 172.21.15.2:0/362347271 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb51c06a0e0 0x7fb51c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:29.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.090+0000 7fb52ffff700 1 --2- 172.21.15.2:0/362347271 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb51c06a0e0 0x7fb51c06c5a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb520001670 tx=0x7fb520009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:29.102 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.094+0000 7fb52cff9700 1 -- 172.21.15.2:0/362347271 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fb5240539c0 con 0x7fb53006a830 2023-08-25T03:35:29.301 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.300+0000 7fb536306700 1 -- 172.21.15.2:0/362347271 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb5300712b0 con 0x7fb53006a830 2023-08-25T03:35:29.304 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.304+0000 7fb52cff9700 1 -- 172.21.15.2:0/362347271 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb524056fe0 con 0x7fb53006a830 2023-08-25T03:35:29.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.306+0000 7fb536306700 1 -- 172.21.15.2:0/362347271 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb51c06a0e0 msgr2=0x7fb51c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:29.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.306+0000 7fb536306700 1 --2- 172.21.15.2:0/362347271 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb51c06a0e0 0x7fb51c06c5a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb520001670 tx=0x7fb520009040 comp rx=0 tx=0).stop 2023-08-25T03:35:29.307 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.306+0000 7fb536306700 1 -- 172.21.15.2:0/362347271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb53006a830 msgr2=0x7fb5301f26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:29.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.306+0000 7fb536306700 1 --2- 172.21.15.2:0/362347271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb53006a830 0x7fb5301f26c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fb5240011e0 tx=0x7fb52400cae0 comp rx=0 tx=0).stop 2023-08-25T03:35:29.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.306+0000 7fb536306700 1 -- 172.21.15.2:0/362347271 shutdown_connections 2023-08-25T03:35:29.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.306+0000 7fb536306700 1 --2- 172.21.15.2:0/362347271 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb5300b3d80 0x7fb5300cd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:29.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.306+0000 7fb536306700 1 --2- 172.21.15.2:0/362347271 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb53006a450 0x7fb5300cf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:29.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.306+0000 7fb536306700 1 --2- 172.21.15.2:0/362347271 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fb51c06a0e0 0x7fb51c06c5a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:29.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.306+0000 7fb536306700 1 --2- 172.21.15.2:0/362347271 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fb53006a830 0x7fb5301f26c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:29.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.306+0000 7fb536306700 1 -- 172.21.15.2:0/362347271 >> 172.21.15.2:0/362347271 conn(0x7fb5300cfd90 msgr2=0x7fb53015add0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:29.308 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.306+0000 7fb536306700 1 -- 172.21.15.2:0/362347271 shutdown_connections 2023-08-25T03:35:29.309 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.306+0000 7fb536306700 1 -- 172.21.15.2:0/362347271 wait complete. 2023-08-25T03:35:29.319 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values' 2023-08-25T03:35:29.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.443+0000 7f7af0878700 1 Processor -- start 2023-08-25T03:35:29.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.443+0000 7f7af0878700 1 -- start start 2023-08-25T03:35:29.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.444+0000 7f7af0878700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f7aec0b19a0 0x7f7aec15e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:29.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.444+0000 7f7af0878700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7aec0b1d80 0x7f7aec071430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:29.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.444+0000 7f7af0878700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f7aec06b2f0 0x7f7aec06b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:29.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.444+0000 7f7af0878700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aec0720b0 con 0x7f7aec06b2f0 2023-08-25T03:35:29.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.444+0000 7f7af0878700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aec071dd0 con 0x7f7aec0b19a0 2023-08-25T03:35:29.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.444+0000 7f7af0878700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec15ddb0 con 0x7f7aec0b1d80 2023-08-25T03:35:29.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.444+0000 7f7aead9d700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f7aec06b2f0 0x7f7aec06b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:29.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.445+0000 7f7aead9d700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f7aec06b2f0 0x7f7aec06b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33766/0 (socket says 172.21.15.2:33766) 2023-08-25T03:35:29.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.445+0000 7f7aead9d700 1 -- 172.21.15.2:0/3404631551 learned_addr learned my addr 172.21.15.2:0/3404631551 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:29.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.445+0000 7f7ae9d9b700 1 --2- 172.21.15.2:0/3404631551 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7aec0b1d80 0x7f7aec071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:29.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.445+0000 7f7aea59c700 1 --2- 172.21.15.2:0/3404631551 >> v2:172.21.15.31:3300/0 conn(0x7f7aec0b19a0 0x7f7aec15e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:29.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.445+0000 7f7aead9d700 1 -- 172.21.15.2:0/3404631551 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7aec0b1d80 msgr2=0x7f7aec071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:29.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.445+0000 7f7aead9d700 1 --2- 172.21.15.2:0/3404631551 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7aec0b1d80 0x7f7aec071430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:29.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.445+0000 7f7aead9d700 1 -- 172.21.15.2:0/3404631551 >> v2:172.21.15.31:3300/0 conn(0x7f7aec0b19a0 msgr2=0x7f7aec15e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:29.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.445+0000 7f7aead9d700 1 --2- 172.21.15.2:0/3404631551 >> v2:172.21.15.31:3300/0 conn(0x7f7aec0b19a0 0x7f7aec15e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:29.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.445+0000 7f7aead9d700 1 -- 172.21.15.2:0/3404631551 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec15e090 con 0x7f7aec06b2f0 2023-08-25T03:35:29.448 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.445+0000 7f7aea59c700 1 --2- 172.21.15.2:0/3404631551 >> v2:172.21.15.31:3300/0 conn(0x7f7aec0b19a0 0x7f7aec15e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:35:29.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.445+0000 7f7aead9d700 1 --2- 172.21.15.2:0/3404631551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7aec06b2f0 0x7f7aec06b780 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7aec06c390 tx=0x7f7ae400d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6aeb62e95bd2d1b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:29.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.445+0000 7f7ae959a700 1 -- 172.21.15.2:0/3404631551 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4010070 con 0x7f7aec06b2f0 2023-08-25T03:35:29.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.445+0000 7f7ae959a700 1 -- 172.21.15.2:0/3404631551 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ae400dd90 con 0x7f7aec06b2f0 2023-08-25T03:35:29.449 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.445+0000 7f7ae959a700 1 -- 172.21.15.2:0/3404631551 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae400b310 con 0x7f7aec06b2f0 2023-08-25T03:35:29.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.446+0000 7f7af0878700 1 -- 172.21.15.2:0/3404631551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7aec06b2f0 msgr2=0x7f7aec06b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:29.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.446+0000 7f7af0878700 1 --2- 172.21.15.2:0/3404631551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7aec06b2f0 0x7f7aec06b780 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7aec06c390 tx=0x7f7ae400d430 comp rx=0 tx=0).stop 2023-08-25T03:35:29.452 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.446+0000 7f7af0878700 1 -- 172.21.15.2:0/3404631551 shutdown_connections 2023-08-25T03:35:29.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.446+0000 7f7af0878700 1 --2- 172.21.15.2:0/3404631551 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7aec06b2f0 0x7f7aec06b780 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:29.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.446+0000 7f7af0878700 1 --2- 172.21.15.2:0/3404631551 >> v2:172.21.15.31:3300/0 conn(0x7f7aec0b19a0 0x7f7aec15e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:29.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.446+0000 7f7af0878700 1 --2- 172.21.15.2:0/3404631551 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7aec0b1d80 0x7f7aec071430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:29.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.446+0000 7f7af0878700 1 -- 172.21.15.2:0/3404631551 >> 172.21.15.2:0/3404631551 conn(0x7f7aec0cfea0 msgr2=0x7f7aec0d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:29.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.446+0000 7f7af0878700 1 -- 172.21.15.2:0/3404631551 shutdown_connections 2023-08-25T03:35:29.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.446+0000 7f7af0878700 1 -- 172.21.15.2:0/3404631551 wait complete. 2023-08-25T03:35:29.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.447+0000 7f7af0878700 1 Processor -- start 2023-08-25T03:35:29.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.447+0000 7f7af0878700 1 -- start start 2023-08-25T03:35:29.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.448+0000 7f7af0878700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aec06b2f0 0x7f7aec0c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:29.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.448+0000 7f7af0878700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7aec0b19a0 0x7f7aec0c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:29.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.448+0000 7f7af0878700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7aec0b1d80 0x7f7aec1559c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:29.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.448+0000 7f7af0878700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0b2e70 con 0x7f7aec0b19a0 2023-08-25T03:35:29.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.448+0000 7f7af0878700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0b3150 con 0x7f7aec06b2f0 2023-08-25T03:35:29.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.448+0000 7f7af0878700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec0b3430 con 0x7f7aec0b1d80 2023-08-25T03:35:29.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.448+0000 7f7ae9d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7aec0b19a0 0x7f7aec0c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:29.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.448+0000 7f7ae9d9b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7aec0b19a0 0x7f7aec0c7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33770/0 (socket says 172.21.15.2:33770) 2023-08-25T03:35:29.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.448+0000 7f7ae9d9b700 1 -- 172.21.15.2:0/1233592814 learned_addr learned my addr 172.21.15.2:0/1233592814 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:29.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.448+0000 7f7aea59c700 1 --2- 172.21.15.2:0/1233592814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aec06b2f0 0x7f7aec0c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:29.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.448+0000 7f7aead9d700 1 --2- 172.21.15.2:0/1233592814 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7aec0b1d80 0x7f7aec1559c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:29.455 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.448+0000 7f7ae9d9b700 1 -- 172.21.15.2:0/1233592814 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7aec0b1d80 msgr2=0x7f7aec1559c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:29.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.448+0000 7f7ae9d9b700 1 --2- 172.21.15.2:0/1233592814 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7aec0b1d80 0x7f7aec1559c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:29.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.448+0000 7f7ae9d9b700 1 -- 172.21.15.2:0/1233592814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aec06b2f0 msgr2=0x7f7aec0c6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:29.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.448+0000 7f7ae9d9b700 1 --2- 172.21.15.2:0/1233592814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aec06b2f0 0x7f7aec0c6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:29.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.448+0000 7f7ae9d9b700 1 -- 172.21.15.2:0/1233592814 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae400bd40 con 0x7f7aec0b19a0 2023-08-25T03:35:29.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.448+0000 7f7aea59c700 1 --2- 172.21.15.2:0/1233592814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aec06b2f0 0x7f7aec0c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:29.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.449+0000 7f7ae9d9b700 1 --2- 172.21.15.2:0/1233592814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7aec0b19a0 0x7f7aec0c7420 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400d800 tx=0x7f7ad400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:29.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.449+0000 7f7ae2ffd700 1 -- 172.21.15.2:0/1233592814 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad40125f0 con 0x7f7aec0b19a0 2023-08-25T03:35:29.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.449+0000 7f7ae2ffd700 1 -- 172.21.15.2:0/1233592814 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad4012750 con 0x7f7aec0b19a0 2023-08-25T03:35:29.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.449+0000 7f7af0878700 1 -- 172.21.15.2:0/1233592814 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aec0b3710 con 0x7f7aec0b19a0 2023-08-25T03:35:29.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.449+0000 7f7ae2ffd700 1 -- 172.21.15.2:0/1233592814 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4012b00 con 0x7f7aec0b19a0 2023-08-25T03:35:29.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.449+0000 7f7af0878700 1 -- 172.21.15.2:0/1233592814 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aec0b2b00 con 0x7f7aec0b19a0 2023-08-25T03:35:29.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.450+0000 7f7af0878700 1 -- 172.21.15.2:0/1233592814 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aec0b3d60 con 0x7f7aec0b19a0 2023-08-25T03:35:29.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.450+0000 7f7ae2ffd700 1 -- 172.21.15.2:0/1233592814 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7ad4012c60 con 0x7f7aec0b19a0 2023-08-25T03:35:29.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.451+0000 7f7ae2ffd700 1 --2- 172.21.15.2:0/1233592814 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f7acc06a0e0 0x7f7acc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:29.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.451+0000 7f7aea59c700 1 --2- 172.21.15.2:0/1233592814 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f7acc06a0e0 0x7f7acc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:29.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.451+0000 7f7ae2ffd700 1 -- 172.21.15.2:0/1233592814 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7ad4088670 con 0x7f7aec0b19a0 2023-08-25T03:35:29.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.451+0000 7f7aea59c700 1 --2- 172.21.15.2:0/1233592814 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f7acc06a0e0 0x7f7acc06c5a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7adc006ba0 tx=0x7f7adc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:29.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.455+0000 7f7ae2ffd700 1 -- 172.21.15.2:0/1233592814 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7ad4053fe0 con 0x7f7aec0b19a0 2023-08-25T03:35:29.739 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.738+0000 7f7af0878700 1 -- 172.21.15.2:0/1233592814 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values"]} v 0) v1 -- 0x7f7aec066f70 con 0x7f7aec0b19a0 2023-08-25T03:35:29.923 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.923+0000 7f7ae2ffd700 1 -- 172.21.15.2:0/1233592814 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_quota.TestQuota.test_disable_enable_human_readable_quota_values"]}]=0 v108) v1 ==== 160+0+0 (secure 0 0 0) 0x7f7ad4057600 con 0x7f7aec0b19a0 2023-08-25T03:35:29.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.925+0000 7f7af0878700 1 -- 172.21.15.2:0/1233592814 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f7acc06a0e0 msgr2=0x7f7acc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:29.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.925+0000 7f7af0878700 1 --2- 172.21.15.2:0/1233592814 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f7acc06a0e0 0x7f7acc06c5a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7adc006ba0 tx=0x7f7adc008040 comp rx=0 tx=0).stop 2023-08-25T03:35:29.926 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.925+0000 7f7af0878700 1 -- 172.21.15.2:0/1233592814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7aec0b19a0 msgr2=0x7f7aec0c7420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:29.927 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.925+0000 7f7af0878700 1 --2- 172.21.15.2:0/1233592814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7aec0b19a0 0x7f7aec0c7420 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400d800 tx=0x7f7ad400dbc0 comp rx=0 tx=0).stop 2023-08-25T03:35:29.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.925+0000 7f7af0878700 1 -- 172.21.15.2:0/1233592814 shutdown_connections 2023-08-25T03:35:29.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.925+0000 7f7af0878700 1 --2- 172.21.15.2:0/1233592814 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7aec0b1d80 0x7f7aec1559c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:29.928 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.925+0000 7f7af0878700 1 --2- 172.21.15.2:0/1233592814 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7aec06b2f0 0x7f7aec0c6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:29.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.925+0000 7f7af0878700 1 --2- 172.21.15.2:0/1233592814 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f7acc06a0e0 0x7f7acc06c5a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:29.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.925+0000 7f7af0878700 1 --2- 172.21.15.2:0/1233592814 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f7aec0b19a0 0x7f7aec0c7420 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:29.929 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.925+0000 7f7af0878700 1 -- 172.21.15.2:0/1233592814 >> 172.21.15.2:0/1233592814 conn(0x7f7aec0cfea0 msgr2=0x7f7aec1572b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:29.930 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.925+0000 7f7af0878700 1 -- 172.21.15.2:0/1233592814 shutdown_connections 2023-08-25T03:35:29.931 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:29.925+0000 7f7af0878700 1 -- 172.21.15.2:0/1233592814 wait complete. 2023-08-25T03:35:29.939 INFO:tasks.cephfs_test_runner:test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) ... ERROR 2023-08-25T03:35:29.940 INFO:tasks.cephfs_test_runner: 2023-08-25T03:35:29.940 INFO:tasks.cephfs_test_runner:====================================================================== 2023-08-25T03:35:29.941 INFO:tasks.cephfs_test_runner:ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-08-25T03:35:29.941 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-08-25T03:35:29.941 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-08-25T03:35:29.941 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ba6b39e4f4f77fb95c747099b53db0467d435f56/qa/tasks/cephfs/test_quota.py", line 157, in test_disable_enable_human_readable_quota_values 2023-08-25T03:35:29.941 INFO:tasks.cephfs_test_runner: self.mount_a.setfattr("./subdir", "ceph.quota.max_bytes", "1K") 2023-08-25T03:35:29.942 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ba6b39e4f4f77fb95c747099b53db0467d435f56/qa/tasks/cephfs/mount.py", line 1566, in setfattr 2023-08-25T03:35:29.942 INFO:tasks.cephfs_test_runner: self.run_shell(**kwargs) 2023-08-25T03:35:29.942 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ba6b39e4f4f77fb95c747099b53db0467d435f56/qa/tasks/cephfs/mount.py", line 780, in run_shell 2023-08-25T03:35:29.942 INFO:tasks.cephfs_test_runner: return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, 2023-08-25T03:35:29.942 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 522, in run 2023-08-25T03:35:29.943 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-08-25T03:35:29.943 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 455, in run 2023-08-25T03:35:29.943 INFO:tasks.cephfs_test_runner: r.wait() 2023-08-25T03:35:29.943 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 161, in wait 2023-08-25T03:35:29.943 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-08-25T03:35:29.944 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-08-25T03:35:29.944 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-08-25T03:35:29.944 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi002 with status 1: '(cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quota.max_bytes -v 1K ./subdir)' 2023-08-25T03:35:29.944 INFO:tasks.cephfs_test_runner: 2023-08-25T03:35:29.944 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-08-25T03:35:29.945 INFO:tasks.cephfs_test_runner:Ran 1 test in 50.030s 2023-08-25T03:35:29.945 INFO:tasks.cephfs_test_runner: 2023-08-25T03:35:29.945 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-08-25T03:35:29.945 INFO:tasks.cephfs_test_runner: 2023-08-25T03:35:29.945 INFO:tasks.cephfs_test_runner:====================================================================== 2023-08-25T03:35:29.946 INFO:tasks.cephfs_test_runner:ERROR: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-08-25T03:35:29.946 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-08-25T03:35:29.946 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-08-25T03:35:29.946 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ba6b39e4f4f77fb95c747099b53db0467d435f56/qa/tasks/cephfs/test_quota.py", line 157, in test_disable_enable_human_readable_quota_values 2023-08-25T03:35:29.946 INFO:tasks.cephfs_test_runner: self.mount_a.setfattr("./subdir", "ceph.quota.max_bytes", "1K") 2023-08-25T03:35:29.947 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ba6b39e4f4f77fb95c747099b53db0467d435f56/qa/tasks/cephfs/mount.py", line 1566, in setfattr 2023-08-25T03:35:29.947 INFO:tasks.cephfs_test_runner: self.run_shell(**kwargs) 2023-08-25T03:35:29.947 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_ba6b39e4f4f77fb95c747099b53db0467d435f56/qa/tasks/cephfs/mount.py", line 780, in run_shell 2023-08-25T03:35:29.947 INFO:tasks.cephfs_test_runner: return self.client_remote.run(args=args, cwd=cwd, timeout=timeout, 2023-08-25T03:35:29.947 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/remote.py", line 522, in run 2023-08-25T03:35:29.948 INFO:tasks.cephfs_test_runner: r = self._runner(client=self.ssh, name=self.shortname, **kwargs) 2023-08-25T03:35:29.948 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 455, in run 2023-08-25T03:35:29.948 INFO:tasks.cephfs_test_runner: r.wait() 2023-08-25T03:35:29.948 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 161, in wait 2023-08-25T03:35:29.948 INFO:tasks.cephfs_test_runner: self._raise_for_status() 2023-08-25T03:35:29.949 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/orchestra/run.py", line 181, in _raise_for_status 2023-08-25T03:35:29.949 INFO:tasks.cephfs_test_runner: raise CommandFailedError( 2023-08-25T03:35:29.949 INFO:tasks.cephfs_test_runner:teuthology.exceptions.CommandFailedError: Command failed on smithi002 with status 1: '(cd /home/ubuntu/cephtest/mnt.0 && exec setfattr -n ceph.quota.max_bytes -v 1K ./subdir)' 2023-08-25T03:35:29.949 INFO:tasks.cephfs_test_runner: 2023-08-25T03:35:29.949 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_ba6b39e4f4f77fb95c747099b53db0467d435f56/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-08-25T03:35:31.604 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=9cb51bb072ba4b52946cc12ca6f4c5ef Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_449a1bc2027504e7b3c3d7b30fa4178906581da7/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_ba6b39e4f4f77fb95c747099b53db0467d435f56/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota) 2023-08-25T03:35:31.635 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-08-25T03:35:31.645 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-08-25T03:35:31.803 INFO:tasks.kclient:Unmounting kernel clients... 2023-08-25T03:35:31.804 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:35:31.805 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:35:31.843 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18212 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-25T03:35:31.844 DEBUG:teuthology.orchestra.run.smithi002:> set -ex 2023-08-25T03:35:31.844 DEBUG:teuthology.orchestra.run.smithi002:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:35:31.904 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=44,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=18212 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-08-25T03:35:31.905 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:35:31.905 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:35:31.927 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15003 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-25T03:35:31.927 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2023-08-25T03:35:31.928 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/proc/self/mounts of=/dev/stdout 2023-08-25T03:35:31.988 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15003 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-08-25T03:35:31.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:31.990 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2023-08-25T03:35:32.012 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-08-25T03:35:32.013 INFO:teuthology.orchestra.run.smithi002.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-08-25T03:35:32.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:32.014 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-08-25T03:35:32.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:32.103 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-08-25T03:35:32.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:32.189 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link delete ceph-brx 2023-08-25T03:35:32.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:32.291 DEBUG:teuthology.orchestra.run.smithi002:> ip netns list 2023-08-25T03:35:32.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:32.312 DEBUG:teuthology.orchestra.run.smithi002:> sudo ip link delete ceph-brx 2023-08-25T03:35:32.383 INFO:teuthology.orchestra.run.smithi002.stderr:Cannot find device "ceph-brx" 2023-08-25T03:35:32.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T03:35:32.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:32.387 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-08-25T03:35:32.408 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-08-25T03:35:32.408 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-08-25T03:35:32.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:32.409 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-08-25T03:35:32.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:32.490 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-08-25T03:35:32.578 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:32.579 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2023-08-25T03:35:32.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:32.679 DEBUG:teuthology.orchestra.run.smithi031:> ip netns list 2023-08-25T03:35:32.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-08-25T03:35:32.698 DEBUG:teuthology.orchestra.run.smithi031:> sudo ip link delete ceph-brx 2023-08-25T03:35:32.768 INFO:teuthology.orchestra.run.smithi031.stderr:Cannot find device "ceph-brx" 2023-08-25T03:35:32.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T03:35:32.770 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-08-25T03:35:32.820 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-08-25T03:35:32.821 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T03:35:32.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.954+0000 7f5f71fd0700 1 Processor -- start 2023-08-25T03:35:32.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.954+0000 7f5f71fd0700 1 -- start start 2023-08-25T03:35:32.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.955+0000 7f5f71fd0700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5f6c15d390 0x7f5f6c0b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:32.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.955+0000 7f5f71fd0700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5f6c0ce4f0 0x7f5f6c0d1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:32.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.955+0000 7f5f71fd0700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f6c0d1410 con 0x7f5f6c160e00 2023-08-25T03:35:32.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.955+0000 7f5f71fd0700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f6c15d770 con 0x7f5f6c0ce4f0 2023-08-25T03:35:32.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.955+0000 7f5f71fd0700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f6c1615c0 con 0x7f5f6c15d390 2023-08-25T03:35:32.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.955+0000 7f5f6bfff700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5f6c0ce4f0 0x7f5f6c0d1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:32.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.956+0000 7f5f6affd700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f5f6c160e00 0x7f5f6c0cdde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48678/0 (socket says 172.21.15.2:48678) 2023-08-25T03:35:32.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.956+0000 7f5f6bfff700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5f6c0ce4f0 0x7f5f6c0d1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33778/0 (socket says 172.21.15.2:33778) 2023-08-25T03:35:32.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.956+0000 7f5f6affd700 1 -- 172.21.15.2:0/4015225352 learned_addr learned my addr 172.21.15.2:0/4015225352 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:32.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.956+0000 7f5f6b7fe700 1 --2- 172.21.15.2:0/4015225352 >> v2:172.21.15.31:3300/0 conn(0x7f5f6c15d390 0x7f5f6c0b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:32.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.956+0000 7f5f6bfff700 1 -- 172.21.15.2:0/4015225352 >> v2:172.21.15.31:3300/0 conn(0x7f5f6c15d390 msgr2=0x7f5f6c0b8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:32.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.956+0000 7f5f6bfff700 1 --2- 172.21.15.2:0/4015225352 >> v2:172.21.15.31:3300/0 conn(0x7f5f6c15d390 0x7f5f6c0b8d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:32.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.956+0000 7f5f6bfff700 1 -- 172.21.15.2:0/4015225352 >> v1:172.21.15.2:6789/0 conn(0x7f5f6c160e00 legacy=0x7f5f6c0cdde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:32.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.956+0000 7f5f6bfff700 1 -- 172.21.15.2:0/4015225352 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f6c1618a0 con 0x7f5f6c0ce4f0 2023-08-25T03:35:32.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.956+0000 7f5f6bfff700 1 --2- 172.21.15.2:0/4015225352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f6c0ce4f0 0x7f5f6c0d1b80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c068780 tx=0x7f5f6400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=125d15b3f103b685 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:32.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.956+0000 7f5f6a7fc700 1 -- 172.21.15.2:0/4015225352 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6400e070 con 0x7f5f6c0ce4f0 2023-08-25T03:35:32.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.956+0000 7f5f6a7fc700 1 -- 172.21.15.2:0/4015225352 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6400bb20 con 0x7f5f6c0ce4f0 2023-08-25T03:35:32.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.956+0000 7f5f6a7fc700 1 -- 172.21.15.2:0/4015225352 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f64012430 con 0x7f5f6c0ce4f0 2023-08-25T03:35:32.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.957+0000 7f5f71fd0700 1 -- 172.21.15.2:0/4015225352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f6c0ce4f0 msgr2=0x7f5f6c0d1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:32.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.957+0000 7f5f71fd0700 1 --2- 172.21.15.2:0/4015225352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f6c0ce4f0 0x7f5f6c0d1b80 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c068780 tx=0x7f5f6400b330 comp rx=0 tx=0).stop 2023-08-25T03:35:32.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.957+0000 7f5f71fd0700 1 -- 172.21.15.2:0/4015225352 shutdown_connections 2023-08-25T03:35:32.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.957+0000 7f5f71fd0700 1 --2- 172.21.15.2:0/4015225352 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f6c0ce4f0 0x7f5f6c0d1b80 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:32.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.957+0000 7f5f71fd0700 1 --2- 172.21.15.2:0/4015225352 >> v2:172.21.15.31:3300/0 conn(0x7f5f6c15d390 0x7f5f6c0b8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:32.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.957+0000 7f5f71fd0700 1 -- 172.21.15.2:0/4015225352 >> 172.21.15.2:0/4015225352 conn(0x7f5f6c06abe0 msgr2=0x7f5f6c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:32.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.957+0000 7f5f71fd0700 1 -- 172.21.15.2:0/4015225352 shutdown_connections 2023-08-25T03:35:32.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.957+0000 7f5f71fd0700 1 -- 172.21.15.2:0/4015225352 wait complete. 2023-08-25T03:35:32.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.958+0000 7f5f71fd0700 1 Processor -- start 2023-08-25T03:35:32.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.958+0000 7f5f71fd0700 1 -- start start 2023-08-25T03:35:32.980 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.959+0000 7f5f71fd0700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f6c0ce4f0 0x7f5f6c152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:32.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.959+0000 7f5f71fd0700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f6c15d390 0x7f5f6c1f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:32.981 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.959+0000 7f5f71fd0700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f6c160e00 0x7f5f6c0cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:32.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.959+0000 7f5f71fd0700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c152b50 con 0x7f5f6c0ce4f0 2023-08-25T03:35:32.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.959+0000 7f5f71fd0700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c0ccc00 con 0x7f5f6c15d390 2023-08-25T03:35:32.982 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.959+0000 7f5f71fd0700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f6c0ccee0 con 0x7f5f6c160e00 2023-08-25T03:35:32.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.959+0000 7f5f6b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f6c0ce4f0 0x7f5f6c152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:32.983 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.959+0000 7f5f6b7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f6c0ce4f0 0x7f5f6c152610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33782/0 (socket says 172.21.15.2:33782) 2023-08-25T03:35:32.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.959+0000 7f5f6b7fe700 1 -- 172.21.15.2:0/3112876054 learned_addr learned my addr 172.21.15.2:0/3112876054 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:32.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.959+0000 7f5f6affd700 1 --2- 172.21.15.2:0/3112876054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f6c15d390 0x7f5f6c1f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:32.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.959+0000 7f5f6bfff700 1 --2- 172.21.15.2:0/3112876054 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f6c160e00 0x7f5f6c0cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:32.984 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.959+0000 7f5f6b7fe700 1 -- 172.21.15.2:0/3112876054 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f6c160e00 msgr2=0x7f5f6c0cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:32.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.959+0000 7f5f6b7fe700 1 --2- 172.21.15.2:0/3112876054 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f6c160e00 0x7f5f6c0cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:32.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.959+0000 7f5f6b7fe700 1 -- 172.21.15.2:0/3112876054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f6c15d390 msgr2=0x7f5f6c1f3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:32.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.959+0000 7f5f6b7fe700 1 --2- 172.21.15.2:0/3112876054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f6c15d390 0x7f5f6c1f3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:32.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.959+0000 7f5f6b7fe700 1 -- 172.21.15.2:0/3112876054 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f64009a70 con 0x7f5f6c0ce4f0 2023-08-25T03:35:32.985 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.959+0000 7f5f6bfff700 1 --2- 172.21.15.2:0/3112876054 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f6c160e00 0x7f5f6c0cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:32.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.960+0000 7f5f6b7fe700 1 --2- 172.21.15.2:0/3112876054 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f6c0ce4f0 0x7f5f6c152610 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c00e9b0 tx=0x7f5f5c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:32.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.960+0000 7f5f63fff700 1 -- 172.21.15.2:0/3112876054 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c009710 con 0x7f5f6c0ce4f0 2023-08-25T03:35:32.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.960+0000 7f5f63fff700 1 -- 172.21.15.2:0/3112876054 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f5c00c850 con 0x7f5f6c0ce4f0 2023-08-25T03:35:32.986 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.960+0000 7f5f71fd0700 1 -- 172.21.15.2:0/3112876054 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f6c0cd1c0 con 0x7f5f6c0ce4f0 2023-08-25T03:35:32.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.960+0000 7f5f63fff700 1 -- 172.21.15.2:0/3112876054 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5c00cbc0 con 0x7f5f6c0ce4f0 2023-08-25T03:35:32.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.960+0000 7f5f71fd0700 1 -- 172.21.15.2:0/3112876054 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f6c0cc860 con 0x7f5f6c0ce4f0 2023-08-25T03:35:32.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.961+0000 7f5f71fd0700 1 -- 172.21.15.2:0/3112876054 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f6c0746d0 con 0x7f5f6c0ce4f0 2023-08-25T03:35:32.988 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.961+0000 7f5f63fff700 1 -- 172.21.15.2:0/3112876054 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5f5c00cd20 con 0x7f5f6c0ce4f0 2023-08-25T03:35:32.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.962+0000 7f5f63fff700 1 --2- 172.21.15.2:0/3112876054 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5f4c06a0e0 0x7f5f4c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:32.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.962+0000 7f5f63fff700 1 -- 172.21.15.2:0/3112876054 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5f5c088660 con 0x7f5f6c0ce4f0 2023-08-25T03:35:32.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.962+0000 7f5f6affd700 1 --2- 172.21.15.2:0/3112876054 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5f4c06a0e0 0x7f5f4c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:32.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.962+0000 7f5f6affd700 1 --2- 172.21.15.2:0/3112876054 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5f4c06a0e0 0x7f5f4c06c5a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0b1fb0 tx=0x7f5f54009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:32.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:32.966+0000 7f5f63fff700 1 -- 172.21.15.2:0/3112876054 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5f5c054050 con 0x7f5f6c0ce4f0 2023-08-25T03:35:33.105 DEBUG:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T03:35:33.108 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T03:35:33.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.165+0000 7f5f71fd0700 1 -- 172.21.15.2:0/3112876054 --> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5f6c0d1660 con 0x7f5f4c06a0e0 2023-08-25T03:35:33.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.167+0000 7f5f63fff700 1 -- 172.21.15.2:0/3112876054 <== mgr.4108 v2:172.21.15.2:6832/3528779779 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72952 (secure 0 0 0) 0x7f5f54001ce0 con 0x7f5f4c06a0e0 2023-08-25T03:35:33.170 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:35:33.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.170+0000 7f5f71fd0700 1 -- 172.21.15.2:0/3112876054 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5f4c06a0e0 msgr2=0x7f5f4c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:33.171 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.170+0000 7f5f71fd0700 1 --2- 172.21.15.2:0/3112876054 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5f4c06a0e0 0x7f5f4c06c5a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5f6c0b1fb0 tx=0x7f5f54009040 comp rx=0 tx=0).stop 2023-08-25T03:35:33.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.170+0000 7f5f71fd0700 1 -- 172.21.15.2:0/3112876054 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f6c0ce4f0 msgr2=0x7f5f6c152610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:33.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.170+0000 7f5f71fd0700 1 --2- 172.21.15.2:0/3112876054 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f6c0ce4f0 0x7f5f6c152610 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5f5c00e9b0 tx=0x7f5f5c00ecc0 comp rx=0 tx=0).stop 2023-08-25T03:35:33.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.170+0000 7f5f71fd0700 1 -- 172.21.15.2:0/3112876054 shutdown_connections 2023-08-25T03:35:33.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.170+0000 7f5f71fd0700 1 --2- 172.21.15.2:0/3112876054 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5f4c06a0e0 0x7f5f4c06c5a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:33.172 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.170+0000 7f5f71fd0700 1 --2- 172.21.15.2:0/3112876054 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5f6c0ce4f0 0x7f5f6c152610 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:33.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.170+0000 7f5f71fd0700 1 --2- 172.21.15.2:0/3112876054 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f6c160e00 0x7f5f6c0cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:33.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.170+0000 7f5f71fd0700 1 --2- 172.21.15.2:0/3112876054 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f6c15d390 0x7f5f6c1f3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:33.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.170+0000 7f5f71fd0700 1 -- 172.21.15.2:0/3112876054 >> 172.21.15.2:0/3112876054 conn(0x7f5f6c06abe0 msgr2=0x7f5f6c157400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:33.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.170+0000 7f5f71fd0700 1 -- 172.21.15.2:0/3112876054 shutdown_connections 2023-08-25T03:35:33.173 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.170+0000 7f5f71fd0700 1 -- 172.21.15.2:0/3112876054 wait complete. 2023-08-25T03:35:33.174 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-08-25T03:35:33.182 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2023-08-25T03:35:31.522531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":220848,"kb_used_data":4672,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754753872,"statfs":{"total":773094113280,"available":772867964928,"internally_reserved":0,"allocated":4784128,"data_stored":2674370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8461,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-52,"num_write_kb":-25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001216"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-25T03:35:27.921667+0000","last_change":"2023-08-25T03:33:57.570267+0000","last_active":"2023-08-25T03:35:27.921667+0000","last_peered":"2023-08-25T03:35:27.921667+0000","last_clean":"2023-08-25T03:35:27.921667+0000","last_became_active":"2023-08-25T03:33:57.569882+0000","last_became_peered":"2023-08-25T03:33:57.569882+0000","last_unstale":"2023-08-25T03:35:27.921667+0000","last_undegraded":"2023-08-25T03:35:27.921667+0000","last_fullsized":"2023-08-25T03:35:27.921667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:11:16.487327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:29.169807+0000","last_change":"2023-08-25T03:33:57.569595+0000","last_active":"2023-08-25T03:35:29.169807+0000","last_peered":"2023-08-25T03:35:29.169807+0000","last_clean":"2023-08-25T03:35:29.169807+0000","last_became_active":"2023-08-25T03:33:57.569307+0000","last_became_peered":"2023-08-25T03:33:57.569307+0000","last_unstale":"2023-08-25T03:35:29.169807+0000","last_undegraded":"2023-08-25T03:35:29.169807+0000","last_fullsized":"2023-08-25T03:35:29.169807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:28:54.310632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-08-25T03:35:25.918488+0000","last_change":"2023-08-25T03:33:57.886704+0000","last_active":"2023-08-25T03:35:25.918488+0000","last_peered":"2023-08-25T03:35:25.918488+0000","last_clean":"2023-08-25T03:35:25.918488+0000","last_became_active":"2023-08-25T03:33:57.886294+0000","last_became_peered":"2023-08-25T03:33:57.886294+0000","last_unstale":"2023-08-25T03:35:25.918488+0000","last_undegraded":"2023-08-25T03:35:25.918488+0000","last_fullsized":"2023-08-25T03:35:25.918488+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:03:43.121931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:29.173591+0000","last_change":"2023-08-25T03:33:57.570657+0000","last_active":"2023-08-25T03:35:29.173591+0000","last_peered":"2023-08-25T03:35:29.173591+0000","last_clean":"2023-08-25T03:35:29.173591+0000","last_became_active":"2023-08-25T03:33:57.570425+0000","last_became_peered":"2023-08-25T03:33:57.570425+0000","last_unstale":"2023-08-25T03:35:29.173591+0000","last_undegraded":"2023-08-25T03:35:29.173591+0000","last_fullsized":"2023-08-25T03:35:29.173591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:21:31.978492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":201,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:29.165285+0000","last_change":"2023-08-25T03:33:51.259898+0000","last_active":"2023-08-25T03:35:29.165285+0000","last_peered":"2023-08-25T03:35:29.165285+0000","last_clean":"2023-08-25T03:35:29.165285+0000","last_became_active":"2023-08-25T03:33:51.258780+0000","last_became_peered":"2023-08-25T03:33:51.258780+0000","last_unstale":"2023-08-25T03:35:29.165285+0000","last_undegraded":"2023-08-25T03:35:29.165285+0000","last_fullsized":"2023-08-25T03:35:29.165285+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:49.666858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:49.666858+0000","last_clean_scrub_stamp":"2023-08-25T03:33:49.666858+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:58:03.603885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:29.165223+0000","last_change":"2023-08-25T03:33:57.571821+0000","last_active":"2023-08-25T03:35:29.165223+0000","last_peered":"2023-08-25T03:35:29.165223+0000","last_clean":"2023-08-25T03:35:29.165223+0000","last_became_active":"2023-08-25T03:33:57.571528+0000","last_became_peered":"2023-08-25T03:33:57.571528+0000","last_unstale":"2023-08-25T03:35:29.165223+0000","last_undegraded":"2023-08-25T03:35:29.165223+0000","last_fullsized":"2023-08-25T03:35:29.165223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:11:29.277327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:29.166729+0000","last_change":"2023-08-25T03:33:57.571730+0000","last_active":"2023-08-25T03:35:29.166729+0000","last_peered":"2023-08-25T03:35:29.166729+0000","last_clean":"2023-08-25T03:35:29.166729+0000","last_became_active":"2023-08-25T03:33:57.571442+0000","last_became_peered":"2023-08-25T03:33:57.571442+0000","last_unstale":"2023-08-25T03:35:29.166729+0000","last_undegraded":"2023-08-25T03:35:29.166729+0000","last_fullsized":"2023-08-25T03:35:29.166729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:20:03.803176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-25T03:35:27.921986+0000","last_change":"2023-08-25T03:33:57.570506+0000","last_active":"2023-08-25T03:35:27.921986+0000","last_peered":"2023-08-25T03:35:27.921986+0000","last_clean":"2023-08-25T03:35:27.921986+0000","last_became_active":"2023-08-25T03:33:57.570171+0000","last_became_peered":"2023-08-25T03:33:57.570171+0000","last_unstale":"2023-08-25T03:35:27.921986+0000","last_undegraded":"2023-08-25T03:35:27.921986+0000","last_fullsized":"2023-08-25T03:35:27.921986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:30.188035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:28.924694+0000","last_change":"2023-08-25T03:33:57.570598+0000","last_active":"2023-08-25T03:35:28.924694+0000","last_peered":"2023-08-25T03:35:28.924694+0000","last_clean":"2023-08-25T03:35:28.924694+0000","last_became_active":"2023-08-25T03:33:57.570215+0000","last_became_peered":"2023-08-25T03:33:57.570215+0000","last_unstale":"2023-08-25T03:35:28.924694+0000","last_undegraded":"2023-08-25T03:35:28.924694+0000","last_fullsized":"2023-08-25T03:35:28.924694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:37:39.062456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542169,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1056768,"data_stored":782347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.56100000000000005}]},{"osd":1,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.76300000000000001}]},{"osd":2,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.629}]},{"osd":3,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.69799999999999995}]},{"osd":4,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.85299999999999998}]},{"osd":6,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.81000000000000005}]}]},{"osd":6,"up_from":14,"seq":60129542169,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":186061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.73799999999999999}]},{"osd":1,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":4.5419999999999998,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":0.40600000000000003}]},{"osd":2,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":5.5179999999999998,"5min":5.5179999999999998,"15min":5.5179999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.58999999999999997}]},{"osd":3,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.55100000000000005}]},{"osd":4,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":6.4050000000000002,"5min":6.4050000000000002,"15min":6.4050000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":7.4029999999999996,"5min":7.4029999999999996,"15min":7.4029999999999996},"last":0.71999999999999997}]},{"osd":7,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.82499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":186061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":6.6120000000000001,"5min":6.6120000000000001,"15min":6.6120000000000001},"last":0.374},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.3530000000000002,"5min":3.3530000000000002,"15min":3.3530000000000002},"last":0.28599999999999998}]},{"osd":2,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":9.0950000000000006,"5min":9.0950000000000006,"15min":9.0950000000000006},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.41599999999999998}]},{"osd":3,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":3.4820000000000002,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":6.1970000000000001,"5min":6.1970000000000001,"15min":6.1970000000000001},"last":0.309}]},{"osd":4,"last update":"Fri Aug 25 03:34:49 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":6.4400000000000004,"5min":6.4400000000000004,"15min":6.4400000000000004},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":6.5860000000000003,"5min":6.5860000000000003,"15min":6.5860000000000003},"last":0.442}]},{"osd":5,"last update":"Fri Aug 25 03:34:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":3.3900000000000001,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":6.399,"5min":6.399,"15min":6.399},"last":0.56100000000000005}]},{"osd":6,"last update":"Fri Aug 25 03:34:49 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":6.5229999999999997,"5min":6.5229999999999997,"15min":6.5229999999999997},"last":0.498},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":3.4529999999999998,"5min":3.4529999999999998,"15min":3.4529999999999998},"last":0.63}]},{"osd":7,"last update":"Fri Aug 25 03:34:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":6.5590000000000002,"5min":6.5590000000000002,"15min":6.5590000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":3.4220000000000002,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.55000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343816,"statfs":{"total":96636764160,"available":96608067584,"internally_reserved":0,"allocated":1028096,"data_stored":770107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"last":0.39200000000000002}]},{"osd":2,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.42299999999999999}]},{"osd":3,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.46700000000000003}]},{"osd":4,"last update":"Fri Aug 25 03:34:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.66700000000000004}]},{"osd":5,"last update":"Fri Aug 25 03:34:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.83399999999999996}]},{"osd":6,"last update":"Fri Aug 25 03:34:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.62},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.73699999999999999}]},{"osd":7,"last update":"Fri Aug 25 03:34:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.81799999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27444,"kb_used_data":424,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344396,"statfs":{"total":96636764160,"available":96608661504,"internally_reserved":0,"allocated":434176,"data_stored":179775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.32200000000000001}]},{"osd":1,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.34300000000000003}]},{"osd":3,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.36899999999999999}]},{"osd":4,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.46200000000000002}]},{"osd":5,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.52800000000000002}]},{"osd":6,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.45200000000000001}]},{"osd":7,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.439},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.54000000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":186061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.224},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":12.923999999999999,"5min":12.923999999999999,"15min":12.923999999999999},"last":0.311}]},{"osd":1,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":2.6680000000000001,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.28799999999999998}]},{"osd":2,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.32700000000000001}]},{"osd":4,"last update":"Fri Aug 25 03:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Aug 25 03:34:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":6.3520000000000003,"5min":6.3520000000000003,"15min":6.3520000000000003},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.63700000000000001}]},{"osd":6,"last update":"Fri Aug 25 03:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.9849999999999999,"5min":2.9849999999999999,"15min":2.9849999999999999},"last":0.72699999999999998}]},{"osd":7,"last update":"Fri Aug 25 03:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.60999999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542169,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":191979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.878},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.65100000000000002}]},{"osd":1,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":8.327,"5min":8.327,"15min":8.327},"last":0.626},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.70499999999999996}]},{"osd":2,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.76800000000000002}]},{"osd":3,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.92700000000000005}]},{"osd":5,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.67800000000000005}]},{"osd":6,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.90700000000000003}]},{"osd":7,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.95499999999999996}]}]},{"osd":5,"up_from":14,"seq":60129542168,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":191979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.39500000000000002}]},{"osd":1,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.47799999999999998}]},{"osd":2,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.59899999999999998}]},{"osd":3,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.51900000000000002}]},{"osd":4,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.71299999999999997}]},{"osd":6,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.69099999999999995}]},{"osd":7,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.62}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-25T03:35:33.186 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T03:35:33.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.346+0000 7f1016679700 1 Processor -- start 2023-08-25T03:35:33.347 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.346+0000 7f1016679700 1 -- start start 2023-08-25T03:35:33.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.347+0000 7f1016679700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f101015a050 0x7f101015db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:33.350 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.347+0000 7f1016679700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f10100b3780 0x7f101006b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:33.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.347+0000 7f1016679700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f101015a430 con 0x7f101006b730 2023-08-25T03:35:33.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.347+0000 7f1016679700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f101006bbc0 con 0x7f101015a050 2023-08-25T03:35:33.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.347+0000 7f1016679700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f101015e450 con 0x7f10100b3780 2023-08-25T03:35:33.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.347+0000 7f100ffff700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f101015a050 0x7f101015db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:33.351 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.347+0000 7f100ffff700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f101015a050 0x7f101015db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33790/0 (socket says 172.21.15.2:33790) 2023-08-25T03:35:33.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.347+0000 7f100ffff700 1 -- 172.21.15.2:0/710011101 learned_addr learned my addr 172.21.15.2:0/710011101 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:33.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.347+0000 7f100f7fe700 1 --2- 172.21.15.2:0/710011101 >> v2:172.21.15.31:3300/0 conn(0x7f10100b3780 0x7f101006b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:33.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.347+0000 7f100ffff700 1 -- 172.21.15.2:0/710011101 >> v2:172.21.15.31:3300/0 conn(0x7f10100b3780 msgr2=0x7f101006b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:33.352 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.348+0000 7f100ffff700 1 --2- 172.21.15.2:0/710011101 >> v2:172.21.15.31:3300/0 conn(0x7f10100b3780 0x7f101006b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:33.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.348+0000 7f100ffff700 1 -- 172.21.15.2:0/710011101 >> v1:172.21.15.31:6789/0 conn(0x7f101006b730 legacy=0x7f10100cdd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:33.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.348+0000 7f100ffff700 1 -- 172.21.15.2:0/710011101 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f101015e730 con 0x7f101015a050 2023-08-25T03:35:33.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.348+0000 7f100f7fe700 1 --2- 172.21.15.2:0/710011101 >> v2:172.21.15.31:3300/0 conn(0x7f10100b3780 0x7f101006b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:33.353 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.348+0000 7f100ffff700 1 --2- 172.21.15.2:0/710011101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f101015a050 0x7f101015db50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f10100b4bb0 tx=0x7f1000009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f6a17561ec965b server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:33.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.348+0000 7f100effd700 1 -- 172.21.15.2:0/710011101 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100000b870 con 0x7f101015a050 2023-08-25T03:35:33.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.348+0000 7f100effd700 1 -- 172.21.15.2:0/710011101 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100000b9d0 con 0x7f101015a050 2023-08-25T03:35:33.354 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.348+0000 7f100effd700 1 -- 172.21.15.2:0/710011101 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100000bd60 con 0x7f101015a050 2023-08-25T03:35:33.355 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.348+0000 7f1016679700 1 -- 172.21.15.2:0/710011101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f101015a050 msgr2=0x7f101015db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:33.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.348+0000 7f1016679700 1 --2- 172.21.15.2:0/710011101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f101015a050 0x7f101015db50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f10100b4bb0 tx=0x7f1000009bd0 comp rx=0 tx=0).stop 2023-08-25T03:35:33.357 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.348+0000 7f1016679700 1 -- 172.21.15.2:0/710011101 shutdown_connections 2023-08-25T03:35:33.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.348+0000 7f1016679700 1 --2- 172.21.15.2:0/710011101 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f101015a050 0x7f101015db50 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:33.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.348+0000 7f1016679700 1 --2- 172.21.15.2:0/710011101 >> v2:172.21.15.31:3300/0 conn(0x7f10100b3780 0x7f101006b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:33.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.348+0000 7f1016679700 1 -- 172.21.15.2:0/710011101 >> 172.21.15.2:0/710011101 conn(0x7f10100cfd10 msgr2=0x7f101006ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:33.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.349+0000 7f1016679700 1 -- 172.21.15.2:0/710011101 shutdown_connections 2023-08-25T03:35:33.360 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.349+0000 7f1016679700 1 -- 172.21.15.2:0/710011101 wait complete. 2023-08-25T03:35:33.364 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.350+0000 7f1016679700 1 Processor -- start 2023-08-25T03:35:33.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.350+0000 7f1016679700 1 -- start start 2023-08-25T03:35:33.365 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.351+0000 7f1016679700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f101006b730 0x7f101014c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:33.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.351+0000 7f1016679700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10100b3780 0x7f101014ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:33.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.351+0000 7f1016679700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f101015a050 0x7f101015b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:33.366 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.351+0000 7f1016679700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101014ca80 con 0x7f101006b730 2023-08-25T03:35:33.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.351+0000 7f1016679700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f101014cd60 con 0x7f10100b3780 2023-08-25T03:35:33.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.351+0000 7f1016679700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101014d040 con 0x7f101015a050 2023-08-25T03:35:33.367 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.351+0000 7f100ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f101006b730 0x7f101014c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:33.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.351+0000 7f100ffff700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f101006b730 0x7f101014c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33796/0 (socket says 172.21.15.2:33796) 2023-08-25T03:35:33.368 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.351+0000 7f100ffff700 1 -- 172.21.15.2:0/2479658080 learned_addr learned my addr 172.21.15.2:0/2479658080 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:33.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.351+0000 7f1014c16700 1 --2- 172.21.15.2:0/2479658080 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f101015a050 0x7f101015b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:33.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.351+0000 7f100f7fe700 1 --2- 172.21.15.2:0/2479658080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10100b3780 0x7f101014ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:33.369 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.351+0000 7f100ffff700 1 -- 172.21.15.2:0/2479658080 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f101015a050 msgr2=0x7f101015b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:33.370 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.351+0000 7f100ffff700 1 --2- 172.21.15.2:0/2479658080 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f101015a050 0x7f101015b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:33.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.351+0000 7f100ffff700 1 -- 172.21.15.2:0/2479658080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10100b3780 msgr2=0x7f101014ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:33.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.351+0000 7f100ffff700 1 --2- 172.21.15.2:0/2479658080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10100b3780 0x7f101014ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:33.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.351+0000 7f100ffff700 1 -- 172.21.15.2:0/2479658080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1000009550 con 0x7f101006b730 2023-08-25T03:35:33.371 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.351+0000 7f100f7fe700 1 --2- 172.21.15.2:0/2479658080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10100b3780 0x7f101014ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:33.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.351+0000 7f100ffff700 1 --2- 172.21.15.2:0/2479658080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f101006b730 0x7f101014c180 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1000000c00 tx=0x7f1000004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:33.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.351+0000 7f100cff9700 1 -- 172.21.15.2:0/2479658080 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1000004540 con 0x7f101006b730 2023-08-25T03:35:33.372 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.351+0000 7f100cff9700 1 -- 172.21.15.2:0/2479658080 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10000036a0 con 0x7f101006b730 2023-08-25T03:35:33.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.351+0000 7f1016679700 1 -- 172.21.15.2:0/2479658080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f101014d320 con 0x7f101006b730 2023-08-25T03:35:33.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.351+0000 7f100cff9700 1 -- 172.21.15.2:0/2479658080 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1000003a10 con 0x7f101006b730 2023-08-25T03:35:33.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.351+0000 7f1016679700 1 -- 172.21.15.2:0/2479658080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f101014c6c0 con 0x7f101006b730 2023-08-25T03:35:33.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.352+0000 7f1016679700 1 -- 172.21.15.2:0/2479658080 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f101014c850 con 0x7f101006b730 2023-08-25T03:35:33.373 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.353+0000 7f100cff9700 1 -- 172.21.15.2:0/2479658080 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1000021070 con 0x7f101006b730 2023-08-25T03:35:33.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.353+0000 7f100cff9700 1 --2- 172.21.15.2:0/2479658080 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f0ff006a0e0 0x7f0ff006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:33.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.353+0000 7f100f7fe700 1 --2- 172.21.15.2:0/2479658080 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f0ff006a0e0 0x7f0ff006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:33.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.353+0000 7f100cff9700 1 -- 172.21.15.2:0/2479658080 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1000089920 con 0x7f101006b730 2023-08-25T03:35:33.374 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.354+0000 7f100f7fe700 1 --2- 172.21.15.2:0/2479658080 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f0ff006a0e0 0x7f0ff006c5a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0ff80099f0 tx=0x7f0ff8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:33.375 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.358+0000 7f100cff9700 1 -- 172.21.15.2:0/2479658080 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f10000553a0 con 0x7f101006b730 2023-08-25T03:35:33.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.556+0000 7f1016679700 1 -- 172.21.15.2:0/2479658080 --> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f101015a670 con 0x7f0ff006a0e0 2023-08-25T03:35:33.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.559+0000 7f100cff9700 1 -- 172.21.15.2:0/2479658080 <== mgr.4108 v2:172.21.15.2:6832/3528779779 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72949 (secure 0 0 0) 0x7f0ff8009680 con 0x7f0ff006a0e0 2023-08-25T03:35:33.563 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:35:33.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.561+0000 7f1016679700 1 -- 172.21.15.2:0/2479658080 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f0ff006a0e0 msgr2=0x7f0ff006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:33.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.561+0000 7f1016679700 1 --2- 172.21.15.2:0/2479658080 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f0ff006a0e0 0x7f0ff006c5a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0ff80099f0 tx=0x7f0ff8008040 comp rx=0 tx=0).stop 2023-08-25T03:35:33.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.561+0000 7f1016679700 1 -- 172.21.15.2:0/2479658080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f101006b730 msgr2=0x7f101014c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:33.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.561+0000 7f1016679700 1 --2- 172.21.15.2:0/2479658080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f101006b730 0x7f101014c180 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1000000c00 tx=0x7f1000004750 comp rx=0 tx=0).stop 2023-08-25T03:35:33.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.561+0000 7f1016679700 1 -- 172.21.15.2:0/2479658080 shutdown_connections 2023-08-25T03:35:33.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.561+0000 7f1016679700 1 --2- 172.21.15.2:0/2479658080 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f0ff006a0e0 0x7f0ff006c5a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:33.568 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.561+0000 7f1016679700 1 --2- 172.21.15.2:0/2479658080 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f101006b730 0x7f101014c180 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:33.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.561+0000 7f1016679700 1 --2- 172.21.15.2:0/2479658080 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f101015a050 0x7f101015b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:33.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.561+0000 7f1016679700 1 --2- 172.21.15.2:0/2479658080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10100b3780 0x7f101014ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:33.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.561+0000 7f1016679700 1 -- 172.21.15.2:0/2479658080 >> 172.21.15.2:0/2479658080 conn(0x7f10100cfd10 msgr2=0x7f10100ce2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:33.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.561+0000 7f1016679700 1 -- 172.21.15.2:0/2479658080 shutdown_connections 2023-08-25T03:35:33.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.561+0000 7f1016679700 1 -- 172.21.15.2:0/2479658080 wait complete. 2023-08-25T03:35:33.572 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-08-25T03:35:33.574 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2023-08-25T03:35:33.523061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":108,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":220876,"kb_used_data":4700,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754753844,"statfs":{"total":773094113280,"available":772867936256,"internally_reserved":0,"allocated":4812800,"data_stored":2686574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8461,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-52,"num_write_kb":-25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.606564"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-25T03:35:27.921667+0000","last_change":"2023-08-25T03:33:57.570267+0000","last_active":"2023-08-25T03:35:27.921667+0000","last_peered":"2023-08-25T03:35:27.921667+0000","last_clean":"2023-08-25T03:35:27.921667+0000","last_became_active":"2023-08-25T03:33:57.569882+0000","last_became_peered":"2023-08-25T03:33:57.569882+0000","last_unstale":"2023-08-25T03:35:27.921667+0000","last_undegraded":"2023-08-25T03:35:27.921667+0000","last_fullsized":"2023-08-25T03:35:27.921667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:11:16.487327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:29.169807+0000","last_change":"2023-08-25T03:33:57.569595+0000","last_active":"2023-08-25T03:35:29.169807+0000","last_peered":"2023-08-25T03:35:29.169807+0000","last_clean":"2023-08-25T03:35:29.169807+0000","last_became_active":"2023-08-25T03:33:57.569307+0000","last_became_peered":"2023-08-25T03:33:57.569307+0000","last_unstale":"2023-08-25T03:35:29.169807+0000","last_undegraded":"2023-08-25T03:35:29.169807+0000","last_fullsized":"2023-08-25T03:35:29.169807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:28:54.310632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:28.927408+0000","last_change":"2023-08-25T03:33:57.886704+0000","last_active":"2023-08-25T03:35:28.927408+0000","last_peered":"2023-08-25T03:35:28.927408+0000","last_clean":"2023-08-25T03:35:28.927408+0000","last_became_active":"2023-08-25T03:33:57.886294+0000","last_became_peered":"2023-08-25T03:33:57.886294+0000","last_unstale":"2023-08-25T03:35:28.927408+0000","last_undegraded":"2023-08-25T03:35:28.927408+0000","last_fullsized":"2023-08-25T03:35:28.927408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:37:26.837165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:29.173591+0000","last_change":"2023-08-25T03:33:57.570657+0000","last_active":"2023-08-25T03:35:29.173591+0000","last_peered":"2023-08-25T03:35:29.173591+0000","last_clean":"2023-08-25T03:35:29.173591+0000","last_became_active":"2023-08-25T03:33:57.570425+0000","last_became_peered":"2023-08-25T03:33:57.570425+0000","last_unstale":"2023-08-25T03:35:29.173591+0000","last_undegraded":"2023-08-25T03:35:29.173591+0000","last_fullsized":"2023-08-25T03:35:29.173591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:21:31.978492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":201,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:29.165285+0000","last_change":"2023-08-25T03:33:51.259898+0000","last_active":"2023-08-25T03:35:29.165285+0000","last_peered":"2023-08-25T03:35:29.165285+0000","last_clean":"2023-08-25T03:35:29.165285+0000","last_became_active":"2023-08-25T03:33:51.258780+0000","last_became_peered":"2023-08-25T03:33:51.258780+0000","last_unstale":"2023-08-25T03:35:29.165285+0000","last_undegraded":"2023-08-25T03:35:29.165285+0000","last_fullsized":"2023-08-25T03:35:29.165285+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:49.666858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:49.666858+0000","last_clean_scrub_stamp":"2023-08-25T03:33:49.666858+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:58:03.603885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:29.165223+0000","last_change":"2023-08-25T03:33:57.571821+0000","last_active":"2023-08-25T03:35:29.165223+0000","last_peered":"2023-08-25T03:35:29.165223+0000","last_clean":"2023-08-25T03:35:29.165223+0000","last_became_active":"2023-08-25T03:33:57.571528+0000","last_became_peered":"2023-08-25T03:33:57.571528+0000","last_unstale":"2023-08-25T03:35:29.165223+0000","last_undegraded":"2023-08-25T03:35:29.165223+0000","last_fullsized":"2023-08-25T03:35:29.165223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:11:29.277327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:29.166729+0000","last_change":"2023-08-25T03:33:57.571730+0000","last_active":"2023-08-25T03:35:29.166729+0000","last_peered":"2023-08-25T03:35:29.166729+0000","last_clean":"2023-08-25T03:35:29.166729+0000","last_became_active":"2023-08-25T03:33:57.571442+0000","last_became_peered":"2023-08-25T03:33:57.571442+0000","last_unstale":"2023-08-25T03:35:29.166729+0000","last_undegraded":"2023-08-25T03:35:29.166729+0000","last_fullsized":"2023-08-25T03:35:29.166729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:20:03.803176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-25T03:35:27.921986+0000","last_change":"2023-08-25T03:33:57.570506+0000","last_active":"2023-08-25T03:35:27.921986+0000","last_peered":"2023-08-25T03:35:27.921986+0000","last_clean":"2023-08-25T03:35:27.921986+0000","last_became_active":"2023-08-25T03:33:57.570171+0000","last_became_peered":"2023-08-25T03:33:57.570171+0000","last_unstale":"2023-08-25T03:35:27.921986+0000","last_undegraded":"2023-08-25T03:35:27.921986+0000","last_fullsized":"2023-08-25T03:35:27.921986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:30.188035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:28.924694+0000","last_change":"2023-08-25T03:33:57.570598+0000","last_active":"2023-08-25T03:35:28.924694+0000","last_peered":"2023-08-25T03:35:28.924694+0000","last_clean":"2023-08-25T03:35:28.924694+0000","last_became_active":"2023-08-25T03:33:57.570215+0000","last_became_peered":"2023-08-25T03:33:57.570215+0000","last_unstale":"2023-08-25T03:35:28.924694+0000","last_undegraded":"2023-08-25T03:35:28.924694+0000","last_fullsized":"2023-08-25T03:35:28.924694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:37:39.062456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542169,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1056768,"data_stored":782347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.56100000000000005}]},{"osd":1,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.76300000000000001}]},{"osd":2,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.629}]},{"osd":3,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.69799999999999995}]},{"osd":4,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.85299999999999998}]},{"osd":6,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.81000000000000005}]}]},{"osd":6,"up_from":14,"seq":60129542169,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":186061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.73799999999999999}]},{"osd":1,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":4.5419999999999998,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":0.40600000000000003}]},{"osd":2,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":5.5179999999999998,"5min":5.5179999999999998,"15min":5.5179999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.58999999999999997}]},{"osd":3,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.55100000000000005}]},{"osd":4,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":6.4050000000000002,"5min":6.4050000000000002,"15min":6.4050000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":7.4029999999999996,"5min":7.4029999999999996,"15min":7.4029999999999996},"last":0.71999999999999997}]},{"osd":7,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.82499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":186061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":6.6120000000000001,"5min":6.6120000000000001,"15min":6.6120000000000001},"last":0.374},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.3530000000000002,"5min":3.3530000000000002,"15min":3.3530000000000002},"last":0.28599999999999998}]},{"osd":2,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":9.0950000000000006,"5min":9.0950000000000006,"15min":9.0950000000000006},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.41599999999999998}]},{"osd":3,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":3.4820000000000002,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":6.1970000000000001,"5min":6.1970000000000001,"15min":6.1970000000000001},"last":0.309}]},{"osd":4,"last update":"Fri Aug 25 03:34:49 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":6.4400000000000004,"5min":6.4400000000000004,"15min":6.4400000000000004},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":6.5860000000000003,"5min":6.5860000000000003,"15min":6.5860000000000003},"last":0.442}]},{"osd":5,"last update":"Fri Aug 25 03:34:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":3.3900000000000001,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":6.399,"5min":6.399,"15min":6.399},"last":0.56100000000000005}]},{"osd":6,"last update":"Fri Aug 25 03:34:49 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":6.5229999999999997,"5min":6.5229999999999997,"15min":6.5229999999999997},"last":0.498},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":3.4529999999999998,"5min":3.4529999999999998,"15min":3.4529999999999998},"last":0.63}]},{"osd":7,"last update":"Fri Aug 25 03:34:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":6.5590000000000002,"5min":6.5590000000000002,"15min":6.5590000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":3.4220000000000002,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.55000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343816,"statfs":{"total":96636764160,"available":96608067584,"internally_reserved":0,"allocated":1028096,"data_stored":770107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"last":0.39200000000000002}]},{"osd":2,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.42299999999999999}]},{"osd":3,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.46700000000000003}]},{"osd":4,"last update":"Fri Aug 25 03:34:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.66700000000000004}]},{"osd":5,"last update":"Fri Aug 25 03:34:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.83399999999999996}]},{"osd":6,"last update":"Fri Aug 25 03:34:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.62},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.73699999999999999}]},{"osd":7,"last update":"Fri Aug 25 03:34:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.81799999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":191979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.254},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.23499999999999999}]},{"osd":1,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.371},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.29399999999999998}]},{"osd":3,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.27700000000000002}]},{"osd":4,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.56000000000000005}]},{"osd":5,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.65700000000000003}]},{"osd":6,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.54900000000000004}]},{"osd":7,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.57899999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":186061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.224},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":12.923999999999999,"5min":12.923999999999999,"15min":12.923999999999999},"last":0.311}]},{"osd":1,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":2.6680000000000001,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.28799999999999998}]},{"osd":2,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.32700000000000001}]},{"osd":4,"last update":"Fri Aug 25 03:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Aug 25 03:34:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":6.3520000000000003,"5min":6.3520000000000003,"15min":6.3520000000000003},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.63700000000000001}]},{"osd":6,"last update":"Fri Aug 25 03:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.9849999999999999,"5min":2.9849999999999999,"15min":2.9849999999999999},"last":0.72699999999999998}]},{"osd":7,"last update":"Fri Aug 25 03:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.60999999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542169,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":191979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.878},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.65100000000000002}]},{"osd":1,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":8.327,"5min":8.327,"15min":8.327},"last":0.626},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.70499999999999996}]},{"osd":2,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.76800000000000002}]},{"osd":3,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.92700000000000005}]},{"osd":5,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.67800000000000005}]},{"osd":6,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.90700000000000003}]},{"osd":7,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.95499999999999996}]}]},{"osd":5,"up_from":14,"seq":60129542168,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":191979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.39500000000000002}]},{"osd":1,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.47799999999999998}]},{"osd":2,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.59899999999999998}]},{"osd":3,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.51900000000000002}]},{"osd":4,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.71299999999999997}]},{"osd":6,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.69099999999999995}]},{"osd":7,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.62}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-25T03:35:33.577 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-08-25T03:35:33.578 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T03:35:33.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.702+0000 7f77ece12700 1 Processor -- start 2023-08-25T03:35:33.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.702+0000 7f77ece12700 1 -- start start 2023-08-25T03:35:33.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.703+0000 7f77ece12700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f77e815d1f0 0x7f77e80ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:33.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.703+0000 7f77ece12700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f77e806cfe0 0x7f77e815d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:33.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.703+0000 7f77ece12700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77e80cf120 con 0x7f77e80d0720 2023-08-25T03:35:33.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.703+0000 7f77ece12700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77e80d0bb0 con 0x7f77e806cfe0 2023-08-25T03:35:33.707 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.703+0000 7f77ece12700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77e80b29f0 con 0x7f77e815d1f0 2023-08-25T03:35:33.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.703+0000 7f77e6d9d700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f77e806cfe0 0x7f77e815d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:33.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.703+0000 7f77e6d9d700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f77e806cfe0 0x7f77e815d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33808/0 (socket says 172.21.15.2:33808) 2023-08-25T03:35:33.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.703+0000 7f77e6d9d700 1 -- 172.21.15.2:0/2830537636 learned_addr learned my addr 172.21.15.2:0/2830537636 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:33.708 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.704+0000 7f77e659c700 1 --2- 172.21.15.2:0/2830537636 >> v2:172.21.15.31:3300/0 conn(0x7f77e815d1f0 0x7f77e80ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:33.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.704+0000 7f77e6d9d700 1 -- 172.21.15.2:0/2830537636 >> v2:172.21.15.31:3300/0 conn(0x7f77e815d1f0 msgr2=0x7f77e80ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:33.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.704+0000 7f77e6d9d700 1 --2- 172.21.15.2:0/2830537636 >> v2:172.21.15.31:3300/0 conn(0x7f77e815d1f0 0x7f77e80ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:33.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.704+0000 7f77e6d9d700 1 -- 172.21.15.2:0/2830537636 >> v1:172.21.15.31:6789/0 conn(0x7f77e80d0720 legacy=0x7f77e806c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:33.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.704+0000 7f77e6d9d700 1 -- 172.21.15.2:0/2830537636 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e80b2cd0 con 0x7f77e806cfe0 2023-08-25T03:35:33.709 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.704+0000 7f77e6d9d700 1 --2- 172.21.15.2:0/2830537636 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77e806cfe0 0x7f77e815d5d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f77e806fe80 tx=0x7f77e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49160ebb69eb8644 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:33.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.704+0000 7f77e559a700 1 -- 172.21.15.2:0/2830537636 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e000e070 con 0x7f77e806cfe0 2023-08-25T03:35:33.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.704+0000 7f77e559a700 1 -- 172.21.15.2:0/2830537636 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77e0003a20 con 0x7f77e806cfe0 2023-08-25T03:35:33.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.704+0000 7f77e559a700 1 -- 172.21.15.2:0/2830537636 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77e0003dd0 con 0x7f77e806cfe0 2023-08-25T03:35:33.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.705+0000 7f77ece12700 1 -- 172.21.15.2:0/2830537636 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77e806cfe0 msgr2=0x7f77e815d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:33.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.705+0000 7f77ece12700 1 --2- 172.21.15.2:0/2830537636 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77e806cfe0 0x7f77e815d5d0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f77e806fe80 tx=0x7f77e000b330 comp rx=0 tx=0).stop 2023-08-25T03:35:33.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.705+0000 7f77ece12700 1 -- 172.21.15.2:0/2830537636 shutdown_connections 2023-08-25T03:35:33.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.705+0000 7f77ece12700 1 --2- 172.21.15.2:0/2830537636 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77e806cfe0 0x7f77e815d5d0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:33.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.705+0000 7f77ece12700 1 --2- 172.21.15.2:0/2830537636 >> v2:172.21.15.31:3300/0 conn(0x7f77e815d1f0 0x7f77e80ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:33.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.705+0000 7f77ece12700 1 -- 172.21.15.2:0/2830537636 >> 172.21.15.2:0/2830537636 conn(0x7f77e80d0f70 msgr2=0x7f77e80d1350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:33.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.705+0000 7f77ece12700 1 -- 172.21.15.2:0/2830537636 shutdown_connections 2023-08-25T03:35:33.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.705+0000 7f77ece12700 1 -- 172.21.15.2:0/2830537636 wait complete. 2023-08-25T03:35:33.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.706+0000 7f77ece12700 1 Processor -- start 2023-08-25T03:35:33.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.707+0000 7f77ece12700 1 -- start start 2023-08-25T03:35:33.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.707+0000 7f77ece12700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77e806cfe0 0x7f77e815faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:33.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.707+0000 7f77ece12700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77e80d0720 0x7f77e81f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:33.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.707+0000 7f77ece12700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f77e815d1f0 0x7f77e81f6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:33.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.707+0000 7f77ece12700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e815ffe0 con 0x7f77e806cfe0 2023-08-25T03:35:33.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.707+0000 7f77ece12700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e815ea00 con 0x7f77e80d0720 2023-08-25T03:35:33.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.707+0000 7f77ece12700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77e815ece0 con 0x7f77e815d1f0 2023-08-25T03:35:33.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.708+0000 7f77e659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77e806cfe0 0x7f77e815faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:33.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.708+0000 7f77e659c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77e806cfe0 0x7f77e815faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33810/0 (socket says 172.21.15.2:33810) 2023-08-25T03:35:33.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.708+0000 7f77e659c700 1 -- 172.21.15.2:0/361060111 learned_addr learned my addr 172.21.15.2:0/361060111 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:33.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.708+0000 7f77e6d9d700 1 --2- 172.21.15.2:0/361060111 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f77e815d1f0 0x7f77e81f6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:33.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.708+0000 7f77e5d9b700 1 --2- 172.21.15.2:0/361060111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77e80d0720 0x7f77e81f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:33.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.708+0000 7f77e659c700 1 -- 172.21.15.2:0/361060111 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f77e815d1f0 msgr2=0x7f77e81f6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:33.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.708+0000 7f77e659c700 1 --2- 172.21.15.2:0/361060111 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f77e815d1f0 0x7f77e81f6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:33.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.708+0000 7f77e659c700 1 -- 172.21.15.2:0/361060111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77e80d0720 msgr2=0x7f77e81f5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:33.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.708+0000 7f77e659c700 1 --2- 172.21.15.2:0/361060111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77e80d0720 0x7f77e81f5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:33.721 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.708+0000 7f77e659c700 1 -- 172.21.15.2:0/361060111 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e0009b30 con 0x7f77e806cfe0 2023-08-25T03:35:33.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.708+0000 7f77e659c700 1 --2- 172.21.15.2:0/361060111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77e806cfe0 0x7f77e815faa0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f77d000c7f0 tx=0x7f77d000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:33.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.708+0000 7f77deffd700 1 -- 172.21.15.2:0/361060111 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d0011800 con 0x7f77e806cfe0 2023-08-25T03:35:33.722 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.708+0000 7f77deffd700 1 -- 172.21.15.2:0/361060111 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d0011960 con 0x7f77e806cfe0 2023-08-25T03:35:33.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.708+0000 7f77ece12700 1 -- 172.21.15.2:0/361060111 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e815efc0 con 0x7f77e806cfe0 2023-08-25T03:35:33.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.708+0000 7f77deffd700 1 -- 172.21.15.2:0/361060111 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d0011cf0 con 0x7f77e806cfe0 2023-08-25T03:35:33.723 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.708+0000 7f77ece12700 1 -- 172.21.15.2:0/361060111 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e815f580 con 0x7f77e806cfe0 2023-08-25T03:35:33.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.709+0000 7f77ece12700 1 -- 172.21.15.2:0/361060111 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e80746d0 con 0x7f77e806cfe0 2023-08-25T03:35:33.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.710+0000 7f77deffd700 1 -- 172.21.15.2:0/361060111 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f77d0018360 con 0x7f77e806cfe0 2023-08-25T03:35:33.724 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.710+0000 7f77deffd700 1 --2- 172.21.15.2:0/361060111 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f77c806a0e0 0x7f77c806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:33.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.710+0000 7f77deffd700 1 -- 172.21.15.2:0/361060111 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f77d00881c0 con 0x7f77e806cfe0 2023-08-25T03:35:33.725 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.710+0000 7f77e5d9b700 1 --2- 172.21.15.2:0/361060111 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f77c806a0e0 0x7f77c806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:33.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.711+0000 7f77e5d9b700 1 --2- 172.21.15.2:0/361060111 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f77c806a0e0 0x7f77c806c5a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f77e81f5d20 tx=0x7f77d8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:33.726 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.715+0000 7f77deffd700 1 -- 172.21.15.2:0/361060111 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f77d0053b30 con 0x7f77e806cfe0 2023-08-25T03:35:33.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.911+0000 7f77ece12700 1 -- 172.21.15.2:0/361060111 --> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f77e80c6610 con 0x7f77c806a0e0 2023-08-25T03:35:33.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.914+0000 7f77deffd700 1 -- 172.21.15.2:0/361060111 <== mgr.4108 v2:172.21.15.2:6832/3528779779 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72949 (secure 0 0 0) 0x7f77e80c6610 con 0x7f77c806a0e0 2023-08-25T03:35:33.916 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:35:33.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.916+0000 7f77ece12700 1 -- 172.21.15.2:0/361060111 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f77c806a0e0 msgr2=0x7f77c806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:33.917 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.916+0000 7f77ece12700 1 --2- 172.21.15.2:0/361060111 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f77c806a0e0 0x7f77c806c5a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f77e81f5d20 tx=0x7f77d8009040 comp rx=0 tx=0).stop 2023-08-25T03:35:33.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.916+0000 7f77ece12700 1 -- 172.21.15.2:0/361060111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77e806cfe0 msgr2=0x7f77e815faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:33.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.916+0000 7f77ece12700 1 --2- 172.21.15.2:0/361060111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77e806cfe0 0x7f77e815faa0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f77d000c7f0 tx=0x7f77d000cbb0 comp rx=0 tx=0).stop 2023-08-25T03:35:33.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.916+0000 7f77ece12700 1 -- 172.21.15.2:0/361060111 shutdown_connections 2023-08-25T03:35:33.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.916+0000 7f77ece12700 1 --2- 172.21.15.2:0/361060111 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f77c806a0e0 0x7f77c806c5a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:33.918 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.916+0000 7f77ece12700 1 --2- 172.21.15.2:0/361060111 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f77e806cfe0 0x7f77e815faa0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:33.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.916+0000 7f77ece12700 1 --2- 172.21.15.2:0/361060111 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f77e815d1f0 0x7f77e81f6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:33.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.916+0000 7f77ece12700 1 --2- 172.21.15.2:0/361060111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77e80d0720 0x7f77e81f5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:33.919 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.916+0000 7f77ece12700 1 -- 172.21.15.2:0/361060111 >> 172.21.15.2:0/361060111 conn(0x7f77e80d0f70 msgr2=0x7f77e806c6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:33.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.916+0000 7f77ece12700 1 -- 172.21.15.2:0/361060111 shutdown_connections 2023-08-25T03:35:33.920 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:33.916+0000 7f77ece12700 1 -- 172.21.15.2:0/361060111 wait complete. 2023-08-25T03:35:33.920 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-08-25T03:35:33.929 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":86,"stamp":"2023-08-25T03:35:33.523061+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":108,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":220876,"kb_used_data":4700,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754753844,"statfs":{"total":773094113280,"available":772867936256,"internally_reserved":0,"allocated":4812800,"data_stored":2686574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8461,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-52,"num_write_kb":-25,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.606564"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-25T03:35:27.921667+0000","last_change":"2023-08-25T03:33:57.570267+0000","last_active":"2023-08-25T03:35:27.921667+0000","last_peered":"2023-08-25T03:35:27.921667+0000","last_clean":"2023-08-25T03:35:27.921667+0000","last_became_active":"2023-08-25T03:33:57.569882+0000","last_became_peered":"2023-08-25T03:33:57.569882+0000","last_unstale":"2023-08-25T03:35:27.921667+0000","last_undegraded":"2023-08-25T03:35:27.921667+0000","last_fullsized":"2023-08-25T03:35:27.921667+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:11:16.487327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:29.169807+0000","last_change":"2023-08-25T03:33:57.569595+0000","last_active":"2023-08-25T03:35:29.169807+0000","last_peered":"2023-08-25T03:35:29.169807+0000","last_clean":"2023-08-25T03:35:29.169807+0000","last_became_active":"2023-08-25T03:33:57.569307+0000","last_became_peered":"2023-08-25T03:33:57.569307+0000","last_unstale":"2023-08-25T03:35:29.169807+0000","last_undegraded":"2023-08-25T03:35:29.169807+0000","last_fullsized":"2023-08-25T03:35:29.169807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:28:54.310632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:28.927408+0000","last_change":"2023-08-25T03:33:57.886704+0000","last_active":"2023-08-25T03:35:28.927408+0000","last_peered":"2023-08-25T03:35:28.927408+0000","last_clean":"2023-08-25T03:35:28.927408+0000","last_became_active":"2023-08-25T03:33:57.886294+0000","last_became_peered":"2023-08-25T03:33:57.886294+0000","last_unstale":"2023-08-25T03:35:28.927408+0000","last_undegraded":"2023-08-25T03:35:28.927408+0000","last_fullsized":"2023-08-25T03:35:28.927408+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T08:37:26.837165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:29.173591+0000","last_change":"2023-08-25T03:33:57.570657+0000","last_active":"2023-08-25T03:35:29.173591+0000","last_peered":"2023-08-25T03:35:29.173591+0000","last_clean":"2023-08-25T03:35:29.173591+0000","last_became_active":"2023-08-25T03:33:57.570425+0000","last_became_peered":"2023-08-25T03:33:57.570425+0000","last_unstale":"2023-08-25T03:35:29.173591+0000","last_undegraded":"2023-08-25T03:35:29.173591+0000","last_fullsized":"2023-08-25T03:35:29.173591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:21:31.978492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":201,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:29.165285+0000","last_change":"2023-08-25T03:33:51.259898+0000","last_active":"2023-08-25T03:35:29.165285+0000","last_peered":"2023-08-25T03:35:29.165285+0000","last_clean":"2023-08-25T03:35:29.165285+0000","last_became_active":"2023-08-25T03:33:51.258780+0000","last_became_peered":"2023-08-25T03:33:51.258780+0000","last_unstale":"2023-08-25T03:35:29.165285+0000","last_undegraded":"2023-08-25T03:35:29.165285+0000","last_fullsized":"2023-08-25T03:35:29.165285+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:49.666858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:49.666858+0000","last_clean_scrub_stamp":"2023-08-25T03:33:49.666858+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:58:03.603885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:29.165223+0000","last_change":"2023-08-25T03:33:57.571821+0000","last_active":"2023-08-25T03:35:29.165223+0000","last_peered":"2023-08-25T03:35:29.165223+0000","last_clean":"2023-08-25T03:35:29.165223+0000","last_became_active":"2023-08-25T03:33:57.571528+0000","last_became_peered":"2023-08-25T03:33:57.571528+0000","last_unstale":"2023-08-25T03:35:29.165223+0000","last_undegraded":"2023-08-25T03:35:29.165223+0000","last_fullsized":"2023-08-25T03:35:29.165223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:11:29.277327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:29.166729+0000","last_change":"2023-08-25T03:33:57.571730+0000","last_active":"2023-08-25T03:35:29.166729+0000","last_peered":"2023-08-25T03:35:29.166729+0000","last_clean":"2023-08-25T03:35:29.166729+0000","last_became_active":"2023-08-25T03:33:57.571442+0000","last_became_peered":"2023-08-25T03:33:57.571442+0000","last_unstale":"2023-08-25T03:35:29.166729+0000","last_undegraded":"2023-08-25T03:35:29.166729+0000","last_fullsized":"2023-08-25T03:35:29.166729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:20:03.803176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-08-25T03:35:27.921986+0000","last_change":"2023-08-25T03:33:57.570506+0000","last_active":"2023-08-25T03:35:27.921986+0000","last_peered":"2023-08-25T03:35:27.921986+0000","last_clean":"2023-08-25T03:35:27.921986+0000","last_became_active":"2023-08-25T03:33:57.570171+0000","last_became_peered":"2023-08-25T03:33:57.570171+0000","last_unstale":"2023-08-25T03:35:27.921986+0000","last_undegraded":"2023-08-25T03:35:27.921986+0000","last_fullsized":"2023-08-25T03:35:27.921986+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T11:52:30.188035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:28.924694+0000","last_change":"2023-08-25T03:33:57.570598+0000","last_active":"2023-08-25T03:35:28.924694+0000","last_peered":"2023-08-25T03:35:28.924694+0000","last_clean":"2023-08-25T03:35:28.924694+0000","last_became_active":"2023-08-25T03:33:57.570215+0000","last_became_peered":"2023-08-25T03:33:57.570215+0000","last_unstale":"2023-08-25T03:35:28.924694+0000","last_undegraded":"2023-08-25T03:35:28.924694+0000","last_fullsized":"2023-08-25T03:35:28.924694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:37:39.062456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542169,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1056768,"data_stored":782347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.56100000000000005}]},{"osd":1,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.76300000000000001}]},{"osd":2,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.629}]},{"osd":3,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.69799999999999995}]},{"osd":4,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.85299999999999998}]},{"osd":6,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.81000000000000005}]}]},{"osd":6,"up_from":14,"seq":60129542169,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":186061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.73799999999999999}]},{"osd":1,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":4.5419999999999998,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":0.40600000000000003}]},{"osd":2,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":5.5179999999999998,"5min":5.5179999999999998,"15min":5.5179999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.58999999999999997}]},{"osd":3,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.55100000000000005}]},{"osd":4,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":6.4050000000000002,"5min":6.4050000000000002,"15min":6.4050000000000002},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.64100000000000001}]},{"osd":5,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":7.4029999999999996,"5min":7.4029999999999996,"15min":7.4029999999999996},"last":0.71999999999999997}]},{"osd":7,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.82499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":186061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":6.6120000000000001,"5min":6.6120000000000001,"15min":6.6120000000000001},"last":0.374},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.3530000000000002,"5min":3.3530000000000002,"15min":3.3530000000000002},"last":0.28599999999999998}]},{"osd":2,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":9.0950000000000006,"5min":9.0950000000000006,"15min":9.0950000000000006},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.41599999999999998}]},{"osd":3,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":3.4820000000000002,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":6.1970000000000001,"5min":6.1970000000000001,"15min":6.1970000000000001},"last":0.309}]},{"osd":4,"last update":"Fri Aug 25 03:34:49 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":6.4400000000000004,"5min":6.4400000000000004,"15min":6.4400000000000004},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":6.5860000000000003,"5min":6.5860000000000003,"15min":6.5860000000000003},"last":0.442}]},{"osd":5,"last update":"Fri Aug 25 03:34:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":3.3900000000000001,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":6.399,"5min":6.399,"15min":6.399},"last":0.56100000000000005}]},{"osd":6,"last update":"Fri Aug 25 03:34:49 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":6.5229999999999997,"5min":6.5229999999999997,"15min":6.5229999999999997},"last":0.498},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":3.4529999999999998,"5min":3.4529999999999998,"15min":3.4529999999999998},"last":0.63}]},{"osd":7,"last update":"Fri Aug 25 03:34:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":6.5590000000000002,"5min":6.5590000000000002,"15min":6.5590000000000002},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":3.4220000000000002,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.55000000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28024,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343816,"statfs":{"total":96636764160,"available":96608067584,"internally_reserved":0,"allocated":1028096,"data_stored":770107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"last":0.39200000000000002}]},{"osd":2,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.42299999999999999}]},{"osd":3,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.46700000000000003}]},{"osd":4,"last update":"Fri Aug 25 03:34:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.66700000000000004}]},{"osd":5,"last update":"Fri Aug 25 03:34:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.83399999999999996}]},{"osd":6,"last update":"Fri Aug 25 03:34:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.62},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.73699999999999999}]},{"osd":7,"last update":"Fri Aug 25 03:34:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.81799999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":191979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.254},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.23499999999999999}]},{"osd":1,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.371},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.29399999999999998}]},{"osd":3,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.27700000000000002}]},{"osd":4,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.56000000000000005}]},{"osd":5,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.65700000000000003}]},{"osd":6,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.54900000000000004}]},{"osd":7,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.57899999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":446464,"data_stored":186061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.224},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":12.923999999999999,"5min":12.923999999999999,"15min":12.923999999999999},"last":0.311}]},{"osd":1,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":2.6680000000000001,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.28799999999999998}]},{"osd":2,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.32700000000000001}]},{"osd":4,"last update":"Fri Aug 25 03:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.48599999999999999}]},{"osd":5,"last update":"Fri Aug 25 03:34:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":6.3520000000000003,"5min":6.3520000000000003,"15min":6.3520000000000003},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.63700000000000001}]},{"osd":6,"last update":"Fri Aug 25 03:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.9849999999999999,"5min":2.9849999999999999,"15min":2.9849999999999999},"last":0.72699999999999998}]},{"osd":7,"last update":"Fri Aug 25 03:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.60999999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542169,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":191979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.878},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.65100000000000002}]},{"osd":1,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":8.327,"5min":8.327,"15min":8.327},"last":0.626},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.70499999999999996}]},{"osd":2,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.76800000000000002}]},{"osd":3,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.92700000000000005}]},{"osd":5,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.67800000000000005}]},{"osd":6,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.90700000000000003}]},{"osd":7,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.95499999999999996}]}]},{"osd":5,"up_from":14,"seq":60129542168,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":191979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.39500000000000002}]},{"osd":1,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.47799999999999998}]},{"osd":2,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.59899999999999998}]},{"osd":3,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.51900000000000002}]},{"osd":4,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.71299999999999997}]},{"osd":6,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.69099999999999995}]},{"osd":7,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.62}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-25T03:35:33.932 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-08-25T03:35:34.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.056+0000 7f926747d700 1 Processor -- start 2023-08-25T03:35:34.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.056+0000 7f926747d700 1 -- start start 2023-08-25T03:35:34.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.057+0000 7f926747d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f926015d370 0x7f92600d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:34.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.057+0000 7f926747d700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f92600b8aa0 0x7f92600cdde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:34.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.057+0000 7f926747d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f926015d750 con 0x7f92600d2b40 2023-08-25T03:35:34.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.057+0000 7f926747d700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92600d1aa0 con 0x7f92600b8aa0 2023-08-25T03:35:34.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.057+0000 7f926747d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92600d1d80 con 0x7f926015d370 2023-08-25T03:35:34.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.057+0000 7f9264a18700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f92600b8aa0 0x7f92600cdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:34.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.057+0000 7f9264a18700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f92600b8aa0 0x7f92600cdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33818/0 (socket says 172.21.15.2:33818) 2023-08-25T03:35:34.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.057+0000 7f9264a18700 1 -- 172.21.15.2:0/951276295 learned_addr learned my addr 172.21.15.2:0/951276295 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:34.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.058+0000 7f9265219700 1 --2- 172.21.15.2:0/951276295 >> v2:172.21.15.31:3300/0 conn(0x7f926015d370 0x7f92600d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:34.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.058+0000 7f9264a18700 1 -- 172.21.15.2:0/951276295 >> v2:172.21.15.31:3300/0 conn(0x7f926015d370 msgr2=0x7f92600d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:34.062 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.058+0000 7f9264a18700 1 --2- 172.21.15.2:0/951276295 >> v2:172.21.15.31:3300/0 conn(0x7f926015d370 0x7f92600d1250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:34.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.058+0000 7f9264a18700 1 -- 172.21.15.2:0/951276295 >> v1:172.21.15.31:6789/0 conn(0x7f92600d2b40 legacy=0x7f92600b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:34.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.058+0000 7f9264a18700 1 -- 172.21.15.2:0/951276295 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92600d2060 con 0x7f92600b8aa0 2023-08-25T03:35:34.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.058+0000 7f9265219700 1 --2- 172.21.15.2:0/951276295 >> v2:172.21.15.31:3300/0 conn(0x7f926015d370 0x7f92600d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:34.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.058+0000 7f9264a18700 1 --2- 172.21.15.2:0/951276295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92600b8aa0 0x7f92600cdde0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f92600b8860 tx=0x7f925c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21dfe04bda7e1e5e server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:34.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.058+0000 7f9257fff700 1 -- 172.21.15.2:0/951276295 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c00b870 con 0x7f92600b8aa0 2023-08-25T03:35:34.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.058+0000 7f9257fff700 1 -- 172.21.15.2:0/951276295 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925c00b9d0 con 0x7f92600b8aa0 2023-08-25T03:35:34.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.058+0000 7f9257fff700 1 -- 172.21.15.2:0/951276295 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c00bd60 con 0x7f92600b8aa0 2023-08-25T03:35:34.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.059+0000 7f926747d700 1 -- 172.21.15.2:0/951276295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92600b8aa0 msgr2=0x7f92600cdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:34.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.059+0000 7f926747d700 1 --2- 172.21.15.2:0/951276295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92600b8aa0 0x7f92600cdde0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f92600b8860 tx=0x7f925c009bd0 comp rx=0 tx=0).stop 2023-08-25T03:35:34.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.059+0000 7f926747d700 1 -- 172.21.15.2:0/951276295 shutdown_connections 2023-08-25T03:35:34.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.059+0000 7f926747d700 1 --2- 172.21.15.2:0/951276295 >> v2:172.21.15.31:3300/0 conn(0x7f926015d370 0x7f92600d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:34.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.059+0000 7f926747d700 1 --2- 172.21.15.2:0/951276295 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92600b8aa0 0x7f92600cdde0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:34.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.059+0000 7f926747d700 1 -- 172.21.15.2:0/951276295 >> 172.21.15.2:0/951276295 conn(0x7f926006ab50 msgr2=0x7f926006af30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:34.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.059+0000 7f926747d700 1 -- 172.21.15.2:0/951276295 shutdown_connections 2023-08-25T03:35:34.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.059+0000 7f926747d700 1 -- 172.21.15.2:0/951276295 wait complete. 2023-08-25T03:35:34.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.060+0000 7f926747d700 1 Processor -- start 2023-08-25T03:35:34.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.060+0000 7f926747d700 1 -- start start 2023-08-25T03:35:34.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.061+0000 7f926747d700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f92600b8aa0 0x7f92600ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:34.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.061+0000 7f926747d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92600d2b40 0x7f9260142380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:34.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.061+0000 7f926747d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f926015d370 0x7f9260153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:34.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.061+0000 7f926747d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92600c8ba0 con 0x7f92600d2b40 2023-08-25T03:35:34.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.061+0000 7f926747d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92600cd030 con 0x7f926015d370 2023-08-25T03:35:34.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.061+0000 7f926747d700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92601428c0 con 0x7f92600b8aa0 2023-08-25T03:35:34.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.061+0000 7f9264a18700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92600d2b40 0x7f9260142380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:34.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.061+0000 7f9264a18700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92600d2b40 0x7f9260142380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33826/0 (socket says 172.21.15.2:33826) 2023-08-25T03:35:34.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.061+0000 7f9264a18700 1 -- 172.21.15.2:0/3868687423 learned_addr learned my addr 172.21.15.2:0/3868687423 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:34.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.061+0000 7f9265219700 1 --2- 172.21.15.2:0/3868687423 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f92600b8aa0 0x7f92600ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:34.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.061+0000 7f9265a1a700 1 --2- 172.21.15.2:0/3868687423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f926015d370 0x7f9260153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:34.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.061+0000 7f9264a18700 1 -- 172.21.15.2:0/3868687423 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f92600b8aa0 msgr2=0x7f92600ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:34.071 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.061+0000 7f9264a18700 1 --2- 172.21.15.2:0/3868687423 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f92600b8aa0 0x7f92600ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:34.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.061+0000 7f9264a18700 1 -- 172.21.15.2:0/3868687423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f926015d370 msgr2=0x7f9260153900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:34.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.061+0000 7f9264a18700 1 --2- 172.21.15.2:0/3868687423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f926015d370 0x7f9260153900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:34.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.061+0000 7f9264a18700 1 -- 172.21.15.2:0/3868687423 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925c009550 con 0x7f92600d2b40 2023-08-25T03:35:34.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.061+0000 7f9264a18700 1 --2- 172.21.15.2:0/3868687423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92600d2b40 0x7f9260142380 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f925c000c00 tx=0x7f925c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:34.072 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.062+0000 7f9255ffb700 1 -- 172.21.15.2:0/3868687423 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c004310 con 0x7f92600d2b40 2023-08-25T03:35:34.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.062+0000 7f9255ffb700 1 -- 172.21.15.2:0/3868687423 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f925c01e070 con 0x7f92600d2b40 2023-08-25T03:35:34.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.062+0000 7f926747d700 1 -- 172.21.15.2:0/3868687423 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92601f2d30 con 0x7f92600d2b40 2023-08-25T03:35:34.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.062+0000 7f9255ffb700 1 -- 172.21.15.2:0/3868687423 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c0036a0 con 0x7f92600d2b40 2023-08-25T03:35:34.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.062+0000 7f926747d700 1 -- 172.21.15.2:0/3868687423 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92600c9090 con 0x7f92600d2b40 2023-08-25T03:35:34.073 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.063+0000 7f926747d700 1 -- 172.21.15.2:0/3868687423 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92600cd770 con 0x7f92600d2b40 2023-08-25T03:35:34.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.063+0000 7f9255ffb700 1 -- 172.21.15.2:0/3868687423 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f925c020030 con 0x7f92600d2b40 2023-08-25T03:35:34.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.064+0000 7f9255ffb700 1 --2- 172.21.15.2:0/3868687423 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f924406a0e0 0x7f924406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:34.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.064+0000 7f9255ffb700 1 -- 172.21.15.2:0/3868687423 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f925c089390 con 0x7f92600d2b40 2023-08-25T03:35:34.074 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.064+0000 7f9265219700 1 --2- 172.21.15.2:0/3868687423 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f924406a0e0 0x7f924406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:34.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.064+0000 7f9265219700 1 --2- 172.21.15.2:0/3868687423 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f924406a0e0 0x7f924406c5a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9250006f10 tx=0x7f9250008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:34.075 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.068+0000 7f9255ffb700 1 -- 172.21.15.2:0/3868687423 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f925c054d10 con 0x7f92600d2b40 2023-08-25T03:35:34.269 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.269+0000 7f926747d700 1 -- 172.21.15.2:0/3868687423 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92600d25f0 con 0x7f92600d2b40 2023-08-25T03:35:34.270 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.269+0000 7f9255ffb700 1 -- 172.21.15.2:0/3868687423 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+14624 (secure 0 0 0) 0x7f925c058330 con 0x7f92600d2b40 2023-08-25T03:35:34.271 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:35:34.271 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":40,"fsid":"dc54136d-3683-431b-8a68-994ec618505c","created":"2023-08-25T03:33:37.639781+0000","modified":"2023-08-25T03:35:28.919104+0000","last_up_change":"2023-08-25T03:33:48.665200+0000","last_in_change":"2023-08-25T03:33:40.189548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-08-25T03:33:49.560643+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2023-08-25T03:33:56.265184+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"fb6d2a67-e1ac-4aa1-a69b-8b6fc5cd4722","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6808","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6809","nonce":2226089280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6810","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6811","nonce":2226089280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6814","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6815","nonce":2226089280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6812","nonce":2226089280},{"type":"v1","addr":"172.21.15.2:6813","nonce":2226089280}]},"public_addr":"172.21.15.2:6809/2226089280","cluster_addr":"172.21.15.2:6811/2226089280","heartbeat_back_addr":"172.21.15.2:6815/2226089280","heartbeat_front_addr":"172.21.15.2:6813/2226089280","state":["exists","up"]},{"osd":1,"uuid":"c30c594a-88f7-4be9-9bd0-320bc6489fab","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6800","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6801","nonce":2145903773}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6802","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6803","nonce":2145903773}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6806","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6807","nonce":2145903773}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6804","nonce":2145903773},{"type":"v1","addr":"172.21.15.2:6805","nonce":2145903773}]},"public_addr":"172.21.15.2:6801/2145903773","cluster_addr":"172.21.15.2:6803/2145903773","heartbeat_back_addr":"172.21.15.2:6807/2145903773","heartbeat_front_addr":"172.21.15.2:6805/2145903773","state":["exists","up"]},{"osd":2,"uuid":"adab17e2-ecf8-4c8a-9c9c-26c70154856f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6816","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6817","nonce":83694260}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6818","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6819","nonce":83694260}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6823","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6825","nonce":83694260}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6820","nonce":83694260},{"type":"v1","addr":"172.21.15.2:6821","nonce":83694260}]},"public_addr":"172.21.15.2:6817/83694260","cluster_addr":"172.21.15.2:6819/83694260","heartbeat_back_addr":"172.21.15.2:6825/83694260","heartbeat_front_addr":"172.21.15.2:6821/83694260","state":["exists","up"]},{"osd":3,"uuid":"a670f4b5-ba0b-4a48-a845-afdb45e74c5f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6822","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6824","nonce":2180868296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6826","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6827","nonce":2180868296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6830","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6831","nonce":2180868296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6828","nonce":2180868296},{"type":"v1","addr":"172.21.15.2:6829","nonce":2180868296}]},"public_addr":"172.21.15.2:6824/2180868296","cluster_addr":"172.21.15.2:6827/2180868296","heartbeat_back_addr":"172.21.15.2:6831/2180868296","heartbeat_front_addr":"172.21.15.2:6829/2180868296","state":["exists","up"]},{"osd":4,"uuid":"534f7781-ab2e-4166-91fe-efd954dc1345","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6825","nonce":4389955}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6827","nonce":4389955}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6831","nonce":4389955}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":4389955},{"type":"v1","addr":"172.21.15.31:6829","nonce":4389955}]},"public_addr":"172.21.15.31:6825/4389955","cluster_addr":"172.21.15.31:6827/4389955","heartbeat_back_addr":"172.21.15.31:6831/4389955","heartbeat_front_addr":"172.21.15.31:6829/4389955","state":["exists","up"]},{"osd":5,"uuid":"c78da803-e437-4ab8-9158-e3f41948e55d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6801","nonce":1678889094}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6803","nonce":1678889094}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6807","nonce":1678889094}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1678889094},{"type":"v1","addr":"172.21.15.31:6805","nonce":1678889094}]},"public_addr":"172.21.15.31:6801/1678889094","cluster_addr":"172.21.15.31:6803/1678889094","heartbeat_back_addr":"172.21.15.31:6807/1678889094","heartbeat_front_addr":"172.21.15.31:6805/1678889094","state":["exists","up"]},{"osd":6,"uuid":"c3989d6e-cf32-4380-9c0f-be95f940c268","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6809","nonce":1680871561}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6811","nonce":1680871561}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6815","nonce":1680871561}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":1680871561},{"type":"v1","addr":"172.21.15.31:6813","nonce":1680871561}]},"public_addr":"172.21.15.31:6809/1680871561","cluster_addr":"172.21.15.31:6811/1680871561","heartbeat_back_addr":"172.21.15.31:6815/1680871561","heartbeat_front_addr":"172.21.15.31:6813/1680871561","state":["exists","up"]},{"osd":7,"uuid":"fab11910-b768-4930-addd-8cf4e2ef03dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6817","nonce":3294787191}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6819","nonce":3294787191}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6823","nonce":3294787191}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3294787191},{"type":"v1","addr":"172.21.15.31:6821","nonce":3294787191}]},"public_addr":"172.21.15.31:6817/3294787191","cluster_addr":"172.21.15.31:6819/3294787191","heartbeat_back_addr":"172.21.15.31:6823/3294787191","heartbeat_front_addr":"172.21.15.31:6821/3294787191","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.383682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.152813+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.377027+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.415786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.482421+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.331319+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.406338+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-08-25T03:33:45.438883+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.31:6835/1140461626":"2023-08-26T03:35:25.914712+0000","172.21.15.31:6833/1140461626":"2023-08-26T03:35:25.914712+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-08-25T03:35:34.272 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.271+0000 7f926747d700 1 -- 172.21.15.2:0/3868687423 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f924406a0e0 msgr2=0x7f924406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:34.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.271+0000 7f926747d700 1 --2- 172.21.15.2:0/3868687423 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f924406a0e0 0x7f924406c5a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9250006f10 tx=0x7f9250008040 comp rx=0 tx=0).stop 2023-08-25T03:35:34.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.271+0000 7f926747d700 1 -- 172.21.15.2:0/3868687423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92600d2b40 msgr2=0x7f9260142380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:34.273 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.271+0000 7f926747d700 1 --2- 172.21.15.2:0/3868687423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92600d2b40 0x7f9260142380 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f925c000c00 tx=0x7f925c004750 comp rx=0 tx=0).stop 2023-08-25T03:35:34.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.271+0000 7f926747d700 1 -- 172.21.15.2:0/3868687423 shutdown_connections 2023-08-25T03:35:34.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.271+0000 7f926747d700 1 --2- 172.21.15.2:0/3868687423 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f926015d370 0x7f9260153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:34.274 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.271+0000 7f926747d700 1 --2- 172.21.15.2:0/3868687423 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f92600b8aa0 0x7f92600ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:34.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.272+0000 7f926747d700 1 --2- 172.21.15.2:0/3868687423 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f924406a0e0 0x7f924406c5a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:34.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.272+0000 7f926747d700 1 --2- 172.21.15.2:0/3868687423 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f92600d2b40 0x7f9260142380 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:34.275 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.272+0000 7f926747d700 1 -- 172.21.15.2:0/3868687423 >> 172.21.15.2:0/3868687423 conn(0x7f926006ab50 msgr2=0x7f92600cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:34.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.272+0000 7f926747d700 1 -- 172.21.15.2:0/3868687423 shutdown_connections 2023-08-25T03:35:34.276 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:34.272+0000 7f926747d700 1 -- 172.21.15.2:0/3868687423 wait complete. 2023-08-25T03:35:35.286 INFO:tasks.ceph:Scrubbing osd.0 2023-08-25T03:35:35.286 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-08-25T03:35:35.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.429+0000 7f0a09735700 1 Processor -- start 2023-08-25T03:35:35.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.429+0000 7f0a09735700 1 -- start start 2023-08-25T03:35:35.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.430+0000 7f0a09735700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f0a04160ce0 0x7f0a040d15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:35.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.430+0000 7f0a09735700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a040d0ab0 con 0x7f0a040b4da0 2023-08-25T03:35:35.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.430+0000 7f0a09735700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a040d07d0 con 0x7f0a040b5180 2023-08-25T03:35:35.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.430+0000 7f0a09735700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a040ce450 con 0x7f0a04160ce0 2023-08-25T03:35:35.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.430+0000 7f0a037fe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f0a04160ce0 0x7f0a040d15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:35.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.430+0000 7f0a02ffd700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f0a040b4da0 0x7f0a041605d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48730/0 (socket says 172.21.15.2:48730) 2023-08-25T03:35:35.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.430+0000 7f0a02ffd700 1 -- 172.21.15.2:0/2442198274 learned_addr learned my addr 172.21.15.2:0/2442198274 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:35.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.430+0000 7f0a037fe700 1 -- 172.21.15.2:0/2442198274 >> v1:172.21.15.31:6789/0 conn(0x7f0a040b5180 legacy=0x7f0a040d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:35.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.430+0000 7f0a037fe700 1 -- 172.21.15.2:0/2442198274 >> v1:172.21.15.2:6789/0 conn(0x7f0a040b4da0 legacy=0x7f0a041605d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:35.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.430+0000 7f0a037fe700 1 -- 172.21.15.2:0/2442198274 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a040ce730 con 0x7f0a04160ce0 2023-08-25T03:35:35.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.430+0000 7f0a01ffb700 1 -- 172.21.15.2:0/2442198274 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1007380112 0 0) 0x7f0a040d0ab0 con 0x7f0a040b4da0 2023-08-25T03:35:35.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.430+0000 7f0a037fe700 1 --2- 172.21.15.2:0/2442198274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a04160ce0 0x7f0a040d15d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009d90 tx=0x7f09fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebea208c8059123d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:35.434 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.431+0000 7f0a01ffb700 1 -- 172.21.15.2:0/2442198274 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc00e070 con 0x7f0a04160ce0 2023-08-25T03:35:35.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.431+0000 7f0a01ffb700 1 -- 172.21.15.2:0/2442198274 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09fc00bb90 con 0x7f0a04160ce0 2023-08-25T03:35:35.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.431+0000 7f0a01ffb700 1 -- 172.21.15.2:0/2442198274 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc003a20 con 0x7f0a04160ce0 2023-08-25T03:35:35.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.431+0000 7f0a09735700 1 -- 172.21.15.2:0/2442198274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a04160ce0 msgr2=0x7f0a040d15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:35.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.431+0000 7f0a09735700 1 --2- 172.21.15.2:0/2442198274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a04160ce0 0x7f0a040d15d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009d90 tx=0x7f09fc00b330 comp rx=0 tx=0).stop 2023-08-25T03:35:35.435 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.431+0000 7f0a09735700 1 -- 172.21.15.2:0/2442198274 shutdown_connections 2023-08-25T03:35:35.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.431+0000 7f0a09735700 1 --2- 172.21.15.2:0/2442198274 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a04160ce0 0x7f0a040d15d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:35.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.431+0000 7f0a09735700 1 -- 172.21.15.2:0/2442198274 >> 172.21.15.2:0/2442198274 conn(0x7f0a040cfb30 msgr2=0x7f0a040cff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:35.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.431+0000 7f0a09735700 1 -- 172.21.15.2:0/2442198274 shutdown_connections 2023-08-25T03:35:35.436 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.431+0000 7f0a09735700 1 -- 172.21.15.2:0/2442198274 wait complete. 2023-08-25T03:35:35.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.433+0000 7f0a09735700 1 Processor -- start 2023-08-25T03:35:35.439 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.433+0000 7f0a09735700 1 -- start start 2023-08-25T03:35:35.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.433+0000 7f0a09735700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a040b4da0 0x7f0a041f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:35.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.433+0000 7f0a09735700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a040b5180 0x7f0a040c9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:35.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.433+0000 7f0a09735700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0a04160ce0 0x7f0a040c9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:35.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.434+0000 7f0a09735700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a040b14e0 con 0x7f0a040b5180 2023-08-25T03:35:35.440 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.434+0000 7f0a09735700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a040c9a80 con 0x7f0a040b4da0 2023-08-25T03:35:35.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.434+0000 7f0a09735700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a041f3be0 con 0x7f0a04160ce0 2023-08-25T03:35:35.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.434+0000 7f0a027fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a040b5180 0x7f0a040c9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:35.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.434+0000 7f0a027fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a040b5180 0x7f0a040c9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33840/0 (socket says 172.21.15.2:33840) 2023-08-25T03:35:35.441 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.434+0000 7f0a027fc700 1 -- 172.21.15.2:0/3484636197 learned_addr learned my addr 172.21.15.2:0/3484636197 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:35.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.434+0000 7f0a02ffd700 1 --2- 172.21.15.2:0/3484636197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a040b4da0 0x7f0a041f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:35.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.434+0000 7f0a037fe700 1 --2- 172.21.15.2:0/3484636197 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0a04160ce0 0x7f0a040c9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:35.442 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.434+0000 7f0a027fc700 1 -- 172.21.15.2:0/3484636197 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0a04160ce0 msgr2=0x7f0a040c9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:35.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.434+0000 7f0a027fc700 1 --2- 172.21.15.2:0/3484636197 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0a04160ce0 0x7f0a040c9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:35.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.434+0000 7f0a027fc700 1 -- 172.21.15.2:0/3484636197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a040b4da0 msgr2=0x7f0a041f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:35.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.434+0000 7f0a027fc700 1 --2- 172.21.15.2:0/3484636197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a040b4da0 0x7f0a041f2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:35.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.434+0000 7f0a027fc700 1 -- 172.21.15.2:0/3484636197 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc009a70 con 0x7f0a040b5180 2023-08-25T03:35:35.443 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.434+0000 7f0a02ffd700 1 --2- 172.21.15.2:0/3484636197 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a040b4da0 0x7f0a041f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:35:35.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.434+0000 7f0a027fc700 1 --2- 172.21.15.2:0/3484636197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a040b5180 0x7f0a040c9000 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f09f400b450 tx=0x7f09f400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:35.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.434+0000 7f09eb7fe700 1 -- 172.21.15.2:0/3484636197 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f4011830 con 0x7f0a040b5180 2023-08-25T03:35:35.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.434+0000 7f09eb7fe700 1 -- 172.21.15.2:0/3484636197 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09f4011990 con 0x7f0a040b5180 2023-08-25T03:35:35.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.434+0000 7f0a09735700 1 -- 172.21.15.2:0/3484636197 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a040cc0c0 con 0x7f0a040b5180 2023-08-25T03:35:35.444 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.434+0000 7f09eb7fe700 1 -- 172.21.15.2:0/3484636197 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09f4011d20 con 0x7f0a040b5180 2023-08-25T03:35:35.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.434+0000 7f0a09735700 1 -- 172.21.15.2:0/3484636197 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a040cc680 con 0x7f0a040b5180 2023-08-25T03:35:35.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.435+0000 7f0a09735700 1 -- 172.21.15.2:0/3484636197 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0a040b6350 con 0x7f0a040b5180 2023-08-25T03:35:35.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.436+0000 7f09eb7fe700 1 -- 172.21.15.2:0/3484636197 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f09f401a400 con 0x7f0a040b5180 2023-08-25T03:35:35.445 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.436+0000 7f09eb7fe700 1 --2- 172.21.15.2:0/3484636197 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f09f006a0e0 0x7f09f006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:35.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.436+0000 7f09eb7fe700 1 -- 172.21.15.2:0/3484636197 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f09f4088940 con 0x7f0a040b5180 2023-08-25T03:35:35.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.436+0000 7f0a02ffd700 1 --2- 172.21.15.2:0/3484636197 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f09f006a0e0 0x7f09f006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:35.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.436+0000 7f09eb7fe700 1 --2- 172.21.15.2:0/3484636197 >> [v2:172.21.15.2:6808/2226089280,v1:172.21.15.2:6809/2226089280] conn(0x7f09f0070560 0x7f09f0072980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:35.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.436+0000 7f09eb7fe700 1 -- 172.21.15.2:0/3484636197 --> [v2:172.21.15.2:6808/2226089280,v1:172.21.15.2:6809/2226089280] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09f0073050 con 0x7f09f0070560 2023-08-25T03:35:35.446 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.436+0000 7f09eb7fe700 1 -- 172.21.15.2:0/3484636197 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09f4088d70 con 0x7f0a040b5180 2023-08-25T03:35:35.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.437+0000 7f0a037fe700 1 --2- 172.21.15.2:0/3484636197 >> [v2:172.21.15.2:6808/2226089280,v1:172.21.15.2:6809/2226089280] conn(0x7f09f0070560 0x7f09f0072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:35.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.437+0000 7f0a02ffd700 1 --2- 172.21.15.2:0/3484636197 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f09f006a0e0 0x7f09f006c5a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f09f8000f60 tx=0x7f09f8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:35.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.437+0000 7f0a037fe700 1 --2- 172.21.15.2:0/3484636197 >> [v2:172.21.15.2:6808/2226089280,v1:172.21.15.2:6809/2226089280] conn(0x7f09f0070560 0x7f09f0072980 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:35.447 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.438+0000 7f09eb7fe700 1 -- 172.21.15.2:0/3484636197 <== osd.0 v2:172.21.15.2:6808/2226089280 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f09f0073050 con 0x7f09f0070560 2023-08-25T03:35:35.453 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.452+0000 7f0a09735700 1 -- 172.21.15.2:0/3484636197 --> [v2:172.21.15.2:6808/2226089280,v1:172.21.15.2:6809/2226089280] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0a041f3da0 con 0x7f09f0070560 2023-08-25T03:35:35.454 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.453+0000 7f09eb7fe700 1 -- 172.21.15.2:0/3484636197 <== osd.0 v2:172.21.15.2:6808/2226089280 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f0a041f3da0 con 0x7f09f0070560 2023-08-25T03:35:35.454 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-08-25T03:35:35.454 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-08-25T03:35:35.455 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-08-25T03:35:35.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.453+0000 7f0a09735700 1 -- 172.21.15.2:0/3484636197 >> [v2:172.21.15.2:6808/2226089280,v1:172.21.15.2:6809/2226089280] conn(0x7f09f0070560 msgr2=0x7f09f0072980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:35.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.453+0000 7f0a09735700 1 --2- 172.21.15.2:0/3484636197 >> [v2:172.21.15.2:6808/2226089280,v1:172.21.15.2:6809/2226089280] conn(0x7f09f0070560 0x7f09f0072980 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:35.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.453+0000 7f0a09735700 1 -- 172.21.15.2:0/3484636197 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f09f006a0e0 msgr2=0x7f09f006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:35.456 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.453+0000 7f0a09735700 1 --2- 172.21.15.2:0/3484636197 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f09f006a0e0 0x7f09f006c5a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f09f8000f60 tx=0x7f09f8009040 comp rx=0 tx=0).stop 2023-08-25T03:35:35.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.453+0000 7f0a09735700 1 -- 172.21.15.2:0/3484636197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a040b5180 msgr2=0x7f0a040c9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:35.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.453+0000 7f0a09735700 1 --2- 172.21.15.2:0/3484636197 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f0a040b5180 0x7f0a040c9000 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f09f400b450 tx=0x7f09f400b810 comp rx=0 tx=0).stop 2023-08-25T03:35:35.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.453+0000 7f0a037fe700 1 -- 172.21.15.2:0/3484636197 reap_dead start 2023-08-25T03:35:35.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.453+0000 7f0a09735700 1 -- 172.21.15.2:0/3484636197 shutdown_connections 2023-08-25T03:35:35.457 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.453+0000 7f0a09735700 1 -- 172.21.15.2:0/3484636197 >> 172.21.15.2:0/3484636197 conn(0x7f0a040cfb30 msgr2=0x7f0a040cd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:35.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.453+0000 7f0a09735700 1 -- 172.21.15.2:0/3484636197 shutdown_connections 2023-08-25T03:35:35.458 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.453+0000 7f0a09735700 1 -- 172.21.15.2:0/3484636197 wait complete. 2023-08-25T03:35:35.466 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-08-25T03:35:35.592 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.592+0000 7f5bf3d3f700 1 Processor -- start 2023-08-25T03:35:35.593 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.592+0000 7f5bf3d3f700 1 -- start start 2023-08-25T03:35:35.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.593+0000 7f5bf3d3f700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5bec0b20b0 0x7f5bec0cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:35.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.593+0000 7f5bf3d3f700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f5bec0716a0 0x7f5bec15b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:35.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.593+0000 7f5bf3d3f700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bec06de40 con 0x7f5bec15f770 2023-08-25T03:35:35.595 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.593+0000 7f5bf3d3f700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bec15edd0 con 0x7f5bec0b20b0 2023-08-25T03:35:35.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.593+0000 7f5bf3d3f700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bec071b10 con 0x7f5bec0716a0 2023-08-25T03:35:35.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.593+0000 7f5bf12da700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f5bec0b20b0 0x7f5bec0cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:35.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.593+0000 7f5bf1adb700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f5bec15f770 0x7f5bec0b19a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48746/0 (socket says 172.21.15.2:48746) 2023-08-25T03:35:35.596 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.593+0000 7f5bf1adb700 1 -- 172.21.15.2:0/2742398751 learned_addr learned my addr 172.21.15.2:0/2742398751 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:35.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.593+0000 7f5bf12da700 1 -- 172.21.15.2:0/2742398751 >> v2:172.21.15.31:3300/0 conn(0x7f5bec0716a0 msgr2=0x7f5bec15b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:35.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.593+0000 7f5bf12da700 1 --2- 172.21.15.2:0/2742398751 >> v2:172.21.15.31:3300/0 conn(0x7f5bec0716a0 0x7f5bec15b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:35.597 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.593+0000 7f5bf12da700 1 -- 172.21.15.2:0/2742398751 >> v1:172.21.15.2:6789/0 conn(0x7f5bec15f770 legacy=0x7f5bec0b19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:35.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.593+0000 7f5bf0ad9700 1 -- 172.21.15.2:0/2742398751 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3956229888 0 0) 0x7f5bec06de40 con 0x7f5bec15f770 2023-08-25T03:35:35.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.593+0000 7f5bf12da700 1 -- 172.21.15.2:0/2742398751 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bec06d080 con 0x7f5bec0b20b0 2023-08-25T03:35:35.598 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.593+0000 7f5bf12da700 1 --2- 172.21.15.2:0/2742398751 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5bec0b20b0 0x7f5bec0cc2c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5bec0ca8f0 tx=0x7f5bdc009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f11f5c22c12b91 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:35.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.593+0000 7f5bf0ad9700 1 -- 172.21.15.2:0/2742398751 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc011830 con 0x7f5bec0b20b0 2023-08-25T03:35:35.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.593+0000 7f5bf0ad9700 1 -- 172.21.15.2:0/2742398751 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bdc011990 con 0x7f5bec0b20b0 2023-08-25T03:35:35.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.593+0000 7f5bf0ad9700 1 -- 172.21.15.2:0/2742398751 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc011d20 con 0x7f5bec0b20b0 2023-08-25T03:35:35.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.594+0000 7f5bf3d3f700 1 -- 172.21.15.2:0/2742398751 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5bec0b20b0 msgr2=0x7f5bec0cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:35.599 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.594+0000 7f5bf3d3f700 1 --2- 172.21.15.2:0/2742398751 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5bec0b20b0 0x7f5bec0cc2c0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5bec0ca8f0 tx=0x7f5bdc009b20 comp rx=0 tx=0).stop 2023-08-25T03:35:35.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.594+0000 7f5bf3d3f700 1 -- 172.21.15.2:0/2742398751 shutdown_connections 2023-08-25T03:35:35.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.594+0000 7f5bf3d3f700 1 --2- 172.21.15.2:0/2742398751 >> v2:172.21.15.31:3300/0 conn(0x7f5bec0716a0 0x7f5bec15b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:35.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.594+0000 7f5bf3d3f700 1 --2- 172.21.15.2:0/2742398751 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5bec0b20b0 0x7f5bec0cc2c0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:35.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.594+0000 7f5bf3d3f700 1 -- 172.21.15.2:0/2742398751 >> 172.21.15.2:0/2742398751 conn(0x7f5bec0cfea0 msgr2=0x7f5bec0d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:35.600 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.594+0000 7f5bf3d3f700 1 -- 172.21.15.2:0/2742398751 shutdown_connections 2023-08-25T03:35:35.601 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.594+0000 7f5bf3d3f700 1 -- 172.21.15.2:0/2742398751 wait complete. 2023-08-25T03:35:35.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.595+0000 7f5bf3d3f700 1 Processor -- start 2023-08-25T03:35:35.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.595+0000 7f5bf3d3f700 1 -- start start 2023-08-25T03:35:35.603 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.596+0000 7f5bf3d3f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bec0716a0 0x7f5bec146070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:35.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.596+0000 7f5bf3d3f700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5bec0b20b0 0x7f5bec0c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:35.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.596+0000 7f5bf3d3f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5bec15f770 0x7f5bec158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:35.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.596+0000 7f5bf3d3f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec156ce0 con 0x7f5bec15f770 2023-08-25T03:35:35.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.596+0000 7f5bf3d3f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec156060 con 0x7f5bec0716a0 2023-08-25T03:35:35.604 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.596+0000 7f5bf3d3f700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bec156340 con 0x7f5bec0b20b0 2023-08-25T03:35:35.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.596+0000 7f5bf22dc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5bec15f770 0x7f5bec158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:35.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.596+0000 7f5bf22dc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5bec15f770 0x7f5bec158650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33858/0 (socket says 172.21.15.2:33858) 2023-08-25T03:35:35.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.596+0000 7f5bf22dc700 1 -- 172.21.15.2:0/1175477684 learned_addr learned my addr 172.21.15.2:0/1175477684 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:35.605 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.596+0000 7f5bf1adb700 1 --2- 172.21.15.2:0/1175477684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bec0716a0 0x7f5bec146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:35.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.596+0000 7f5bf12da700 1 --2- 172.21.15.2:0/1175477684 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5bec0b20b0 0x7f5bec0c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:35.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.596+0000 7f5bf22dc700 1 -- 172.21.15.2:0/1175477684 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5bec0b20b0 msgr2=0x7f5bec0c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:35.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.596+0000 7f5bf22dc700 1 --2- 172.21.15.2:0/1175477684 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5bec0b20b0 0x7f5bec0c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:35.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.596+0000 7f5bf22dc700 1 -- 172.21.15.2:0/1175477684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bec0716a0 msgr2=0x7f5bec146070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:35.606 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.596+0000 7f5bf22dc700 1 --2- 172.21.15.2:0/1175477684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bec0716a0 0x7f5bec146070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:35.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.596+0000 7f5bf22dc700 1 -- 172.21.15.2:0/1175477684 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bdc0094a0 con 0x7f5bec15f770 2023-08-25T03:35:35.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.596+0000 7f5bf22dc700 1 --2- 172.21.15.2:0/1175477684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5bec15f770 0x7f5bec158650 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5be400b970 tx=0x7f5be400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:35.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.597+0000 7f5be27fc700 1 -- 172.21.15.2:0/1175477684 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be400c9b0 con 0x7f5bec15f770 2023-08-25T03:35:35.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.597+0000 7f5be27fc700 1 -- 172.21.15.2:0/1175477684 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5be400cb30 con 0x7f5bec15f770 2023-08-25T03:35:35.607 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.597+0000 7f5bf3d3f700 1 -- 172.21.15.2:0/1175477684 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bec156620 con 0x7f5bec15f770 2023-08-25T03:35:35.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.597+0000 7f5be27fc700 1 -- 172.21.15.2:0/1175477684 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5be4014440 con 0x7f5bec15f770 2023-08-25T03:35:35.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.597+0000 7f5bf3d3f700 1 -- 172.21.15.2:0/1175477684 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bec1465b0 con 0x7f5bec15f770 2023-08-25T03:35:35.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.598+0000 7f5bf3d3f700 1 -- 172.21.15.2:0/1175477684 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bec066f70 con 0x7f5bec15f770 2023-08-25T03:35:35.608 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.598+0000 7f5be27fc700 1 -- 172.21.15.2:0/1175477684 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f5be401d3b0 con 0x7f5bec15f770 2023-08-25T03:35:35.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.598+0000 7f5be27fc700 1 --2- 172.21.15.2:0/1175477684 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5bd006a0e0 0x7f5bd006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:35.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.599+0000 7f5be27fc700 1 -- 172.21.15.2:0/1175477684 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5be4006e40 con 0x7f5bec15f770 2023-08-25T03:35:35.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.599+0000 7f5bf1adb700 1 --2- 172.21.15.2:0/1175477684 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5bd006a0e0 0x7f5bd006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:35.609 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.599+0000 7f5bf1adb700 1 --2- 172.21.15.2:0/1175477684 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5bd006a0e0 0x7f5bd006c5a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5be8000f60 tx=0x7f5be8007c90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:35.610 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.603+0000 7f5be27fc700 1 -- 172.21.15.2:0/1175477684 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f5be4053a20 con 0x7f5bec15f770 2023-08-25T03:35:35.806 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.806+0000 7f5bf3d3f700 1 -- 172.21.15.2:0/1175477684 --> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bec06d640 con 0x7f5bd006a0e0 2023-08-25T03:35:35.807 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.806+0000 7f5be27fc700 1 -- 172.21.15.2:0/1175477684 <== mgr.4108 v2:172.21.15.2:6832/3528779779 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5be8013420 con 0x7f5bd006a0e0 2023-08-25T03:35:35.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.808+0000 7f5bf3d3f700 1 -- 172.21.15.2:0/1175477684 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5bd006a0e0 msgr2=0x7f5bd006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:35.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.808+0000 7f5bf3d3f700 1 --2- 172.21.15.2:0/1175477684 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5bd006a0e0 0x7f5bd006c5a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5be8000f60 tx=0x7f5be8007c90 comp rx=0 tx=0).stop 2023-08-25T03:35:35.809 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.808+0000 7f5bf3d3f700 1 -- 172.21.15.2:0/1175477684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5bec15f770 msgr2=0x7f5bec158650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:35.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.808+0000 7f5bf3d3f700 1 --2- 172.21.15.2:0/1175477684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5bec15f770 0x7f5bec158650 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f5be400b970 tx=0x7f5be400bd30 comp rx=0 tx=0).stop 2023-08-25T03:35:35.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.808+0000 7f5bf3d3f700 1 -- 172.21.15.2:0/1175477684 shutdown_connections 2023-08-25T03:35:35.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.808+0000 7f5bf3d3f700 1 --2- 172.21.15.2:0/1175477684 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f5bd006a0e0 0x7f5bd006c5a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:35.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.808+0000 7f5bf3d3f700 1 --2- 172.21.15.2:0/1175477684 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f5bec15f770 0x7f5bec158650 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:35.810 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.808+0000 7f5bf3d3f700 1 --2- 172.21.15.2:0/1175477684 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5bec0716a0 0x7f5bec146070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:35.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.808+0000 7f5bf3d3f700 1 --2- 172.21.15.2:0/1175477684 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5bec0b20b0 0x7f5bec0c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:35.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.808+0000 7f5bf3d3f700 1 -- 172.21.15.2:0/1175477684 >> 172.21.15.2:0/1175477684 conn(0x7f5bec0cfea0 msgr2=0x7f5bec0cbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:35.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.808+0000 7f5bf3d3f700 1 -- 172.21.15.2:0/1175477684 shutdown_connections 2023-08-25T03:35:35.811 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.808+0000 7f5bf3d3f700 1 -- 172.21.15.2:0/1175477684 wait complete. 2023-08-25T03:35:35.812 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 0 to deep-scrub 2023-08-25T03:35:35.821 INFO:tasks.ceph:Scrubbing osd.1 2023-08-25T03:35:35.822 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-08-25T03:35:35.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.950+0000 7f1605658700 1 Processor -- start 2023-08-25T03:35:35.951 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.950+0000 7f1605658700 1 -- start start 2023-08-25T03:35:35.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.951+0000 7f1605658700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f16000b4da0 0x7f16001605d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:35.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.951+0000 7f1605658700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16000b5180 0x7f1600160b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:35.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.951+0000 7f1605658700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f16000d21e0 0x7f1600161280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:35.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.951+0000 7f1605658700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16000d0ab0 con 0x7f16000b4da0 2023-08-25T03:35:35.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.951+0000 7f1605658700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1600161050 con 0x7f16000d21e0 2023-08-25T03:35:35.954 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.951+0000 7f1605658700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16000d07d0 con 0x7f16000b5180 2023-08-25T03:35:35.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.951+0000 7f15feffd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f16000b4da0 0x7f16001605d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:35.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.951+0000 7f15feffd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f16000b4da0 0x7f16001605d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33862/0 (socket says 172.21.15.2:33862) 2023-08-25T03:35:35.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.951+0000 7f15feffd700 1 -- 172.21.15.2:0/2655934054 learned_addr learned my addr 172.21.15.2:0/2655934054 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:35.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.951+0000 7f15ff7fe700 1 --2- 172.21.15.2:0/2655934054 >> v2:172.21.15.31:3300/0 conn(0x7f16000d21e0 0x7f1600161280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:35.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.951+0000 7f15fe7fc700 1 --2- 172.21.15.2:0/2655934054 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16000b5180 0x7f1600160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:35.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.951+0000 7f15feffd700 1 -- 172.21.15.2:0/2655934054 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16000b5180 msgr2=0x7f1600160b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:35.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.951+0000 7f15feffd700 1 --2- 172.21.15.2:0/2655934054 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16000b5180 0x7f1600160b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:35.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.951+0000 7f15feffd700 1 -- 172.21.15.2:0/2655934054 >> v2:172.21.15.31:3300/0 conn(0x7f16000d21e0 msgr2=0x7f1600161280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:35.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.951+0000 7f15feffd700 1 --2- 172.21.15.2:0/2655934054 >> v2:172.21.15.31:3300/0 conn(0x7f16000d21e0 0x7f1600161280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:35.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.951+0000 7f15feffd700 1 -- 172.21.15.2:0/2655934054 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16000ce450 con 0x7f16000b4da0 2023-08-25T03:35:35.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.951+0000 7f15ff7fe700 1 --2- 172.21.15.2:0/2655934054 >> v2:172.21.15.31:3300/0 conn(0x7f16000d21e0 0x7f1600161280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:35.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.952+0000 7f15feffd700 1 --2- 172.21.15.2:0/2655934054 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16000b4da0 0x7f16001605d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f16000b2aa0 tx=0x7f15f4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f845e8390a0e1c47 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:35.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.952+0000 7f15fdffb700 1 -- 172.21.15.2:0/2655934054 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f4011830 con 0x7f16000b4da0 2023-08-25T03:35:35.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.952+0000 7f15fdffb700 1 -- 172.21.15.2:0/2655934054 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f4011990 con 0x7f16000b4da0 2023-08-25T03:35:35.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.952+0000 7f15fdffb700 1 -- 172.21.15.2:0/2655934054 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f4011d20 con 0x7f16000b4da0 2023-08-25T03:35:35.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.952+0000 7f1605658700 1 -- 172.21.15.2:0/2655934054 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16000b4da0 msgr2=0x7f16001605d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:35.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.952+0000 7f1605658700 1 --2- 172.21.15.2:0/2655934054 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16000b4da0 0x7f16001605d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f16000b2aa0 tx=0x7f15f4009bd0 comp rx=0 tx=0).stop 2023-08-25T03:35:35.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.952+0000 7f1605658700 1 -- 172.21.15.2:0/2655934054 shutdown_connections 2023-08-25T03:35:35.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.952+0000 7f1605658700 1 --2- 172.21.15.2:0/2655934054 >> v2:172.21.15.31:3300/0 conn(0x7f16000d21e0 0x7f1600161280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:35.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.952+0000 7f1605658700 1 --2- 172.21.15.2:0/2655934054 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16000b4da0 0x7f16001605d0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:35.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.952+0000 7f1605658700 1 --2- 172.21.15.2:0/2655934054 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16000b5180 0x7f1600160b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:35.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.952+0000 7f1605658700 1 -- 172.21.15.2:0/2655934054 >> 172.21.15.2:0/2655934054 conn(0x7f16000cfb30 msgr2=0x7f16000cff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:35.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.952+0000 7f1605658700 1 -- 172.21.15.2:0/2655934054 shutdown_connections 2023-08-25T03:35:35.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.952+0000 7f1605658700 1 -- 172.21.15.2:0/2655934054 wait complete. 2023-08-25T03:35:35.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.954+0000 7f1605658700 1 Processor -- start 2023-08-25T03:35:35.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.954+0000 7f1605658700 1 -- start start 2023-08-25T03:35:35.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.954+0000 7f1605658700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16000b4da0 0x7f16000ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:35.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.955+0000 7f1605658700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16000b5180 0x7f160015a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:35.963 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.955+0000 7f1605658700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16000d21e0 0x7f16000cb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:35.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.955+0000 7f1605658700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16000cbbb0 con 0x7f16000d21e0 2023-08-25T03:35:35.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.955+0000 7f1605658700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16000cbe90 con 0x7f16000b5180 2023-08-25T03:35:35.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.955+0000 7f1605658700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16000cc170 con 0x7f16000b4da0 2023-08-25T03:35:35.964 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.955+0000 7f15ff7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16000d21e0 0x7f16000cb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:35.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.955+0000 7f15ff7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16000d21e0 0x7f16000cb1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33872/0 (socket says 172.21.15.2:33872) 2023-08-25T03:35:35.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.955+0000 7f15ff7fe700 1 -- 172.21.15.2:0/670430485 learned_addr learned my addr 172.21.15.2:0/670430485 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:35.965 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.955+0000 7f15feffd700 1 --2- 172.21.15.2:0/670430485 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16000b4da0 0x7f16000ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:35.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.955+0000 7f15fe7fc700 1 --2- 172.21.15.2:0/670430485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16000b5180 0x7f160015a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:35.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.955+0000 7f15ff7fe700 1 -- 172.21.15.2:0/670430485 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16000b4da0 msgr2=0x7f16000ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:35.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.955+0000 7f15ff7fe700 1 --2- 172.21.15.2:0/670430485 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16000b4da0 0x7f16000ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:35.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.955+0000 7f15ff7fe700 1 -- 172.21.15.2:0/670430485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16000b5180 msgr2=0x7f160015a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:35.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.955+0000 7f15ff7fe700 1 --2- 172.21.15.2:0/670430485 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16000b5180 0x7f160015a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:35.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.955+0000 7f15ff7fe700 1 -- 172.21.15.2:0/670430485 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f4009550 con 0x7f16000d21e0 2023-08-25T03:35:35.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.955+0000 7f15feffd700 1 --2- 172.21.15.2:0/670430485 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16000b4da0 0x7f16000ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:35.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.955+0000 7f15ff7fe700 1 --2- 172.21.15.2:0/670430485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16000d21e0 0x7f16000cb1d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f15f8007d40 tx=0x7f15f800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:35.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.955+0000 7f15e77fe700 1 -- 172.21.15.2:0/670430485 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f800ddd0 con 0x7f16000d21e0 2023-08-25T03:35:35.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.955+0000 7f15e77fe700 1 -- 172.21.15.2:0/670430485 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f800f040 con 0x7f16000d21e0 2023-08-25T03:35:35.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.955+0000 7f1605658700 1 -- 172.21.15.2:0/670430485 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16000cc450 con 0x7f16000d21e0 2023-08-25T03:35:35.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.955+0000 7f15e77fe700 1 -- 172.21.15.2:0/670430485 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f8003c60 con 0x7f16000d21e0 2023-08-25T03:35:35.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.955+0000 7f1605658700 1 -- 172.21.15.2:0/670430485 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f160015ae20 con 0x7f16000d21e0 2023-08-25T03:35:35.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.956+0000 7f1605658700 1 -- 172.21.15.2:0/670430485 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f16000b6350 con 0x7f16000d21e0 2023-08-25T03:35:35.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.957+0000 7f15e77fe700 1 -- 172.21.15.2:0/670430485 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f15f8004030 con 0x7f16000d21e0 2023-08-25T03:35:35.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.957+0000 7f15e77fe700 1 --2- 172.21.15.2:0/670430485 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f15ec06a0e0 0x7f15ec06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:35.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.957+0000 7f15e77fe700 1 -- 172.21.15.2:0/670430485 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f15f8088960 con 0x7f16000d21e0 2023-08-25T03:35:35.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.957+0000 7f15feffd700 1 --2- 172.21.15.2:0/670430485 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f15ec06a0e0 0x7f15ec06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:35.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.957+0000 7f15e77fe700 1 --2- 172.21.15.2:0/670430485 >> [v2:172.21.15.2:6800/2145903773,v1:172.21.15.2:6801/2145903773] conn(0x7f15ec070560 0x7f15ec072980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:35.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.957+0000 7f15e77fe700 1 -- 172.21.15.2:0/670430485 --> [v2:172.21.15.2:6800/2145903773,v1:172.21.15.2:6801/2145903773] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f15ec073050 con 0x7f15ec070560 2023-08-25T03:35:35.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.957+0000 7f15e77fe700 1 -- 172.21.15.2:0/670430485 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15f8088d90 con 0x7f16000d21e0 2023-08-25T03:35:35.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.958+0000 7f15fe7fc700 1 --2- 172.21.15.2:0/670430485 >> [v2:172.21.15.2:6800/2145903773,v1:172.21.15.2:6801/2145903773] conn(0x7f15ec070560 0x7f15ec072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:35.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.958+0000 7f15feffd700 1 --2- 172.21.15.2:0/670430485 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f15ec06a0e0 0x7f15ec06c5a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f15f4014040 tx=0x7f15f4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:35.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.958+0000 7f15fe7fc700 1 --2- 172.21.15.2:0/670430485 >> [v2:172.21.15.2:6800/2145903773,v1:172.21.15.2:6801/2145903773] conn(0x7f15ec070560 0x7f15ec072980 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:35.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.958+0000 7f15e77fe700 1 -- 172.21.15.2:0/670430485 <== osd.1 v2:172.21.15.2:6800/2145903773 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f15ec073050 con 0x7f15ec070560 2023-08-25T03:35:35.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.973+0000 7f1605658700 1 -- 172.21.15.2:0/670430485 --> [v2:172.21.15.2:6800/2145903773,v1:172.21.15.2:6801/2145903773] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f16000ce730 con 0x7f15ec070560 2023-08-25T03:35:35.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.974+0000 7f15e77fe700 1 -- 172.21.15.2:0/670430485 <== osd.1 v2:172.21.15.2:6800/2145903773 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f15f0009ae0 con 0x7f15ec070560 2023-08-25T03:35:35.975 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-08-25T03:35:35.975 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-08-25T03:35:35.975 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-08-25T03:35:35.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.974+0000 7f1605658700 1 -- 172.21.15.2:0/670430485 >> [v2:172.21.15.2:6800/2145903773,v1:172.21.15.2:6801/2145903773] conn(0x7f15ec070560 msgr2=0x7f15ec072980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:35.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.974+0000 7f1605658700 1 --2- 172.21.15.2:0/670430485 >> [v2:172.21.15.2:6800/2145903773,v1:172.21.15.2:6801/2145903773] conn(0x7f15ec070560 0x7f15ec072980 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:35.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.974+0000 7f1605658700 1 -- 172.21.15.2:0/670430485 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f15ec06a0e0 msgr2=0x7f15ec06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:35.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.974+0000 7f1605658700 1 --2- 172.21.15.2:0/670430485 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f15ec06a0e0 0x7f15ec06c5a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f15f4014040 tx=0x7f15f4008030 comp rx=0 tx=0).stop 2023-08-25T03:35:35.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.974+0000 7f1605658700 1 -- 172.21.15.2:0/670430485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16000d21e0 msgr2=0x7f16000cb1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:35.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.974+0000 7f1605658700 1 --2- 172.21.15.2:0/670430485 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f16000d21e0 0x7f16000cb1d0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f15f8007d40 tx=0x7f15f800d3b0 comp rx=0 tx=0).stop 2023-08-25T03:35:35.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.974+0000 7f15ff7fe700 1 -- 172.21.15.2:0/670430485 reap_dead start 2023-08-25T03:35:35.977 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.974+0000 7f1605658700 1 -- 172.21.15.2:0/670430485 shutdown_connections 2023-08-25T03:35:35.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.974+0000 7f1605658700 1 -- 172.21.15.2:0/670430485 >> 172.21.15.2:0/670430485 conn(0x7f16000cfb30 msgr2=0x7f16000b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:35.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.974+0000 7f1605658700 1 -- 172.21.15.2:0/670430485 shutdown_connections 2023-08-25T03:35:35.978 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:35.974+0000 7f1605658700 1 -- 172.21.15.2:0/670430485 wait complete. 2023-08-25T03:35:35.986 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-08-25T03:35:36.112 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.112+0000 7f384505d700 1 Processor -- start 2023-08-25T03:35:36.113 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.112+0000 7f384505d700 1 -- start start 2023-08-25T03:35:36.114 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.112+0000 7f384505d700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f38400b1fb0 0x7f3840160f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:36.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.112+0000 7f384505d700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f3840161450 0x7f38400cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:36.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.112+0000 7f384505d700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f384006d450 con 0x7f38400ce0b0 2023-08-25T03:35:36.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.112+0000 7f384505d700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f384006d730 con 0x7f3840161450 2023-08-25T03:35:36.115 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.112+0000 7f384505d700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f384006da10 con 0x7f38400b1fb0 2023-08-25T03:35:36.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.113+0000 7f383e59c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f3840161450 0x7f38400cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:36.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.113+0000 7f383e59c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f3840161450 0x7f38400cdb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33878/0 (socket says 172.21.15.2:33878) 2023-08-25T03:35:36.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.113+0000 7f383e59c700 1 -- 172.21.15.2:0/2089561751 learned_addr learned my addr 172.21.15.2:0/2089561751 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:36.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.113+0000 7f383ed9d700 1 --2- 172.21.15.2:0/2089561751 >> v2:172.21.15.31:3300/0 conn(0x7f38400b1fb0 0x7f3840160f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:36.116 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.113+0000 7f383e59c700 1 -- 172.21.15.2:0/2089561751 >> v2:172.21.15.31:3300/0 conn(0x7f38400b1fb0 msgr2=0x7f3840160f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:36.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.113+0000 7f383e59c700 1 --2- 172.21.15.2:0/2089561751 >> v2:172.21.15.31:3300/0 conn(0x7f38400b1fb0 0x7f3840160f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.113+0000 7f383e59c700 1 -- 172.21.15.2:0/2089561751 >> v1:172.21.15.31:6789/0 conn(0x7f38400ce0b0 legacy=0x7f384006b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:36.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.113+0000 7f383e59c700 1 -- 172.21.15.2:0/2089561751 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384006dcf0 con 0x7f3840161450 2023-08-25T03:35:36.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.113+0000 7f383e59c700 1 --2- 172.21.15.2:0/2089561751 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3840161450 0x7f38400cdb70 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f384006e0d0 tx=0x7f3834009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db522d6272fe392 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:36.117 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.113+0000 7f383dd9b700 1 -- 172.21.15.2:0/2089561751 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3834004230 con 0x7f3840161450 2023-08-25T03:35:36.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.113+0000 7f383dd9b700 1 -- 172.21.15.2:0/2089561751 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3834004390 con 0x7f3840161450 2023-08-25T03:35:36.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.113+0000 7f383dd9b700 1 -- 172.21.15.2:0/2089561751 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38340036a0 con 0x7f3840161450 2023-08-25T03:35:36.118 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.114+0000 7f384505d700 1 -- 172.21.15.2:0/2089561751 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3840161450 msgr2=0x7f38400cdb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:36.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.114+0000 7f384505d700 1 --2- 172.21.15.2:0/2089561751 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3840161450 0x7f38400cdb70 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f384006e0d0 tx=0x7f3834009bd0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.114+0000 7f384505d700 1 -- 172.21.15.2:0/2089561751 shutdown_connections 2023-08-25T03:35:36.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.114+0000 7f384505d700 1 --2- 172.21.15.2:0/2089561751 >> v2:172.21.15.31:3300/0 conn(0x7f38400b1fb0 0x7f3840160f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.119 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.114+0000 7f384505d700 1 --2- 172.21.15.2:0/2089561751 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3840161450 0x7f38400cdb70 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.114+0000 7f384505d700 1 -- 172.21.15.2:0/2089561751 >> 172.21.15.2:0/2089561751 conn(0x7f38400cfea0 msgr2=0x7f38400d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:36.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.114+0000 7f384505d700 1 -- 172.21.15.2:0/2089561751 shutdown_connections 2023-08-25T03:35:36.120 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.114+0000 7f384505d700 1 -- 172.21.15.2:0/2089561751 wait complete. 2023-08-25T03:35:36.122 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.115+0000 7f384505d700 1 Processor -- start 2023-08-25T03:35:36.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.115+0000 7f384505d700 1 -- start start 2023-08-25T03:35:36.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.116+0000 7f384505d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38400b1fb0 0x7f38400b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:36.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.116+0000 7f384505d700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f38400ce0b0 0x7f38400b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:36.123 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.116+0000 7f384505d700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38401f21c0 0x7f38400c7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:36.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.116+0000 7f384505d700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38400b53d0 con 0x7f38400b1fb0 2023-08-25T03:35:36.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.116+0000 7f384505d700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3840151e50 con 0x7f38401f21c0 2023-08-25T03:35:36.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.116+0000 7f384505d700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38400b4480 con 0x7f38400ce0b0 2023-08-25T03:35:36.124 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.116+0000 7f383ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38400b1fb0 0x7f38400b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:36.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.116+0000 7f383ed9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38400b1fb0 0x7f38400b3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33886/0 (socket says 172.21.15.2:33886) 2023-08-25T03:35:36.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.116+0000 7f383ed9d700 1 -- 172.21.15.2:0/1239400570 learned_addr learned my addr 172.21.15.2:0/1239400570 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:36.125 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.116+0000 7f383e59c700 1 --2- 172.21.15.2:0/1239400570 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f38400ce0b0 0x7f38400b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:36.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.116+0000 7f383f59e700 1 --2- 172.21.15.2:0/1239400570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38401f21c0 0x7f38400c7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:36.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.117+0000 7f383ed9d700 1 -- 172.21.15.2:0/1239400570 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f38400ce0b0 msgr2=0x7f38400b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:36.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.117+0000 7f383ed9d700 1 --2- 172.21.15.2:0/1239400570 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f38400ce0b0 0x7f38400b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.126 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.117+0000 7f383ed9d700 1 -- 172.21.15.2:0/1239400570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38401f21c0 msgr2=0x7f38400c7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:36.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.117+0000 7f383ed9d700 1 --2- 172.21.15.2:0/1239400570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38401f21c0 0x7f38400c7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.117+0000 7f383ed9d700 1 -- 172.21.15.2:0/1239400570 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3834009550 con 0x7f38400b1fb0 2023-08-25T03:35:36.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.117+0000 7f383ed9d700 1 --2- 172.21.15.2:0/1239400570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38400b1fb0 0x7f38400b3780 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f383000c820 tx=0x7f383000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:36.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.117+0000 7f38277fe700 1 -- 172.21.15.2:0/1239400570 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3830011800 con 0x7f38400b1fb0 2023-08-25T03:35:36.127 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.117+0000 7f38277fe700 1 -- 172.21.15.2:0/1239400570 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3830011960 con 0x7f38400b1fb0 2023-08-25T03:35:36.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.117+0000 7f384505d700 1 -- 172.21.15.2:0/1239400570 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38400b4750 con 0x7f38400b1fb0 2023-08-25T03:35:36.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.117+0000 7f38277fe700 1 -- 172.21.15.2:0/1239400570 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3830011cd0 con 0x7f38400b1fb0 2023-08-25T03:35:36.128 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.117+0000 7f384505d700 1 -- 172.21.15.2:0/1239400570 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38400b4d10 con 0x7f38400b1fb0 2023-08-25T03:35:36.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.118+0000 7f384505d700 1 -- 172.21.15.2:0/1239400570 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3840066f70 con 0x7f38400b1fb0 2023-08-25T03:35:36.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.118+0000 7f38277fe700 1 -- 172.21.15.2:0/1239400570 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3830004020 con 0x7f38400b1fb0 2023-08-25T03:35:36.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.119+0000 7f38277fe700 1 --2- 172.21.15.2:0/1239400570 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f382c06a0e0 0x7f382c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:36.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.119+0000 7f383e59c700 1 --2- 172.21.15.2:0/1239400570 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f382c06a0e0 0x7f382c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:36.129 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.119+0000 7f38277fe700 1 -- 172.21.15.2:0/1239400570 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3830087fe0 con 0x7f38400b1fb0 2023-08-25T03:35:36.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.119+0000 7f383e59c700 1 --2- 172.21.15.2:0/1239400570 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f382c06a0e0 0x7f382c06c5a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f384006e0d0 tx=0x7f3834008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:36.130 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.124+0000 7f38277fe700 1 -- 172.21.15.2:0/1239400570 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3830053a60 con 0x7f38400b1fb0 2023-08-25T03:35:36.324 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.323+0000 7f384505d700 1 -- 172.21.15.2:0/1239400570 --> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f384015d920 con 0x7f382c06a0e0 2023-08-25T03:35:36.325 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.323+0000 7f38277fe700 1 -- 172.21.15.2:0/1239400570 <== mgr.4108 v2:172.21.15.2:6832/3528779779 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3834004580 con 0x7f382c06a0e0 2023-08-25T03:35:36.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.325+0000 7f384505d700 1 -- 172.21.15.2:0/1239400570 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f382c06a0e0 msgr2=0x7f382c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:36.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.325+0000 7f384505d700 1 --2- 172.21.15.2:0/1239400570 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f382c06a0e0 0x7f382c06c5a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f384006e0d0 tx=0x7f3834008030 comp rx=0 tx=0).stop 2023-08-25T03:35:36.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.325+0000 7f384505d700 1 -- 172.21.15.2:0/1239400570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38400b1fb0 msgr2=0x7f38400b3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:36.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.325+0000 7f384505d700 1 --2- 172.21.15.2:0/1239400570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38400b1fb0 0x7f38400b3780 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f383000c820 tx=0x7f383000cb30 comp rx=0 tx=0).stop 2023-08-25T03:35:36.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.325+0000 7f384505d700 1 -- 172.21.15.2:0/1239400570 shutdown_connections 2023-08-25T03:35:36.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.325+0000 7f384505d700 1 --2- 172.21.15.2:0/1239400570 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f382c06a0e0 0x7f382c06c5a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.325+0000 7f384505d700 1 --2- 172.21.15.2:0/1239400570 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f38400b1fb0 0x7f38400b3780 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.325+0000 7f384505d700 1 --2- 172.21.15.2:0/1239400570 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f38401f21c0 0x7f38400c7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.325+0000 7f384505d700 1 --2- 172.21.15.2:0/1239400570 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f38400ce0b0 0x7f38400b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.325+0000 7f384505d700 1 -- 172.21.15.2:0/1239400570 >> 172.21.15.2:0/1239400570 conn(0x7f38400cfea0 msgr2=0x7f38400cd070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:36.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.326+0000 7f384505d700 1 -- 172.21.15.2:0/1239400570 shutdown_connections 2023-08-25T03:35:36.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.326+0000 7f384505d700 1 -- 172.21.15.2:0/1239400570 wait complete. 2023-08-25T03:35:36.332 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 1 to deep-scrub 2023-08-25T03:35:36.338 INFO:tasks.ceph:Scrubbing osd.2 2023-08-25T03:35:36.339 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-08-25T03:35:36.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.467+0000 7f82a83fe700 1 Processor -- start 2023-08-25T03:35:36.468 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.467+0000 7f82a83fe700 1 -- start start 2023-08-25T03:35:36.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.468+0000 7f82a83fe700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f82a015d4d0 0x7f82a0161de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:36.478 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.468+0000 7f82a83fe700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f82a015d8b0 0x7f82a006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:36.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.468+0000 7f82a83fe700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a006c5c0 con 0x7f82a00cd9e0 2023-08-25T03:35:36.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.468+0000 7f82a83fe700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a00ce1c0 con 0x7f82a015d4d0 2023-08-25T03:35:36.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.468+0000 7f82a83fe700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a006b5a0 con 0x7f82a015d8b0 2023-08-25T03:35:36.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.468+0000 7f82a619a700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f82a015d4d0 0x7f82a0161de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:36.479 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.468+0000 7f82a619a700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f82a015d4d0 0x7f82a0161de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33892/0 (socket says 172.21.15.2:33892) 2023-08-25T03:35:36.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.468+0000 7f82a619a700 1 -- 172.21.15.2:0/13650951 learned_addr learned my addr 172.21.15.2:0/13650951 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:36.480 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.468+0000 7f82a5999700 1 --2- 172.21.15.2:0/13650951 >> v2:172.21.15.31:3300/0 conn(0x7f82a015d8b0 0x7f82a006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:36.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.468+0000 7f82a619a700 1 -- 172.21.15.2:0/13650951 >> v2:172.21.15.31:3300/0 conn(0x7f82a015d8b0 msgr2=0x7f82a006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:36.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.468+0000 7f82a619a700 1 --2- 172.21.15.2:0/13650951 >> v2:172.21.15.31:3300/0 conn(0x7f82a015d8b0 0x7f82a006c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.468+0000 7f82a619a700 1 -- 172.21.15.2:0/13650951 >> v1:172.21.15.31:6789/0 conn(0x7f82a00cd9e0 legacy=0x7f82a00cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:36.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.468+0000 7f82a619a700 1 -- 172.21.15.2:0/13650951 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a006b880 con 0x7f82a015d4d0 2023-08-25T03:35:36.481 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.469+0000 7f82a619a700 1 --2- 172.21.15.2:0/13650951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82a015d4d0 0x7f82a0161de0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f82a00cebe0 tx=0x7f8290009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4327e9bf562799b1 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:36.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.469+0000 7f82a5198700 1 -- 172.21.15.2:0/13650951 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290004230 con 0x7f82a015d4d0 2023-08-25T03:35:36.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.469+0000 7f82a5198700 1 -- 172.21.15.2:0/13650951 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8290004390 con 0x7f82a015d4d0 2023-08-25T03:35:36.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.469+0000 7f82a5198700 1 -- 172.21.15.2:0/13650951 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82900036a0 con 0x7f82a015d4d0 2023-08-25T03:35:36.482 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.469+0000 7f82a83fe700 1 -- 172.21.15.2:0/13650951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82a015d4d0 msgr2=0x7f82a0161de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:36.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.469+0000 7f82a83fe700 1 --2- 172.21.15.2:0/13650951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82a015d4d0 0x7f82a0161de0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f82a00cebe0 tx=0x7f8290009bd0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.469+0000 7f82a83fe700 1 -- 172.21.15.2:0/13650951 shutdown_connections 2023-08-25T03:35:36.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.469+0000 7f82a83fe700 1 --2- 172.21.15.2:0/13650951 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82a015d4d0 0x7f82a0161de0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.469+0000 7f82a83fe700 1 --2- 172.21.15.2:0/13650951 >> v2:172.21.15.31:3300/0 conn(0x7f82a015d8b0 0x7f82a006c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.483 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.469+0000 7f82a83fe700 1 -- 172.21.15.2:0/13650951 >> 172.21.15.2:0/13650951 conn(0x7f82a0065fe0 msgr2=0x7f82a00663c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:36.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.469+0000 7f82a83fe700 1 -- 172.21.15.2:0/13650951 shutdown_connections 2023-08-25T03:35:36.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.469+0000 7f82a83fe700 1 -- 172.21.15.2:0/13650951 wait complete. 2023-08-25T03:35:36.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.471+0000 7f82a83fe700 1 Processor -- start 2023-08-25T03:35:36.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.471+0000 7f82a83fe700 1 -- start start 2023-08-25T03:35:36.484 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.471+0000 7f82a83fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82a00cd9e0 0x7f82a01578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:36.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.471+0000 7f82a83fe700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a015d4d0 0x7f82a00b61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:36.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.471+0000 7f82a83fe700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f82a015d8b0 0x7f82a00c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:36.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.471+0000 7f82a83fe700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a00b3ca0 con 0x7f82a00cd9e0 2023-08-25T03:35:36.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.471+0000 7f82a83fe700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a00b3ef0 con 0x7f82a015d4d0 2023-08-25T03:35:36.485 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.471+0000 7f82a83fe700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a00b41d0 con 0x7f82a015d8b0 2023-08-25T03:35:36.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.472+0000 7f82a619a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82a00cd9e0 0x7f82a01578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:36.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.472+0000 7f82a619a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82a00cd9e0 0x7f82a01578b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33898/0 (socket says 172.21.15.2:33898) 2023-08-25T03:35:36.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.472+0000 7f82a619a700 1 -- 172.21.15.2:0/3559268940 learned_addr learned my addr 172.21.15.2:0/3559268940 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:36.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.472+0000 7f82a5999700 1 --2- 172.21.15.2:0/3559268940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a015d4d0 0x7f82a00b61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:36.486 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.472+0000 7f82a699b700 1 --2- 172.21.15.2:0/3559268940 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f82a015d8b0 0x7f82a00c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:36.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.472+0000 7f82a619a700 1 -- 172.21.15.2:0/3559268940 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f82a015d8b0 msgr2=0x7f82a00c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:36.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.472+0000 7f82a619a700 1 --2- 172.21.15.2:0/3559268940 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f82a015d8b0 0x7f82a00c8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.472+0000 7f82a619a700 1 -- 172.21.15.2:0/3559268940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a015d4d0 msgr2=0x7f82a00b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:36.487 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.472+0000 7f82a619a700 1 --2- 172.21.15.2:0/3559268940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a015d4d0 0x7f82a00b61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.472+0000 7f82a619a700 1 -- 172.21.15.2:0/3559268940 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8290009550 con 0x7f82a00cd9e0 2023-08-25T03:35:36.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.472+0000 7f82a5999700 1 --2- 172.21.15.2:0/3559268940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82a015d4d0 0x7f82a00b61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:36.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.472+0000 7f82a619a700 1 --2- 172.21.15.2:0/3559268940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82a00cd9e0 0x7f82a01578b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8290009130 tx=0x7f8290004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:36.488 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.472+0000 7f8296ffd700 1 -- 172.21.15.2:0/3559268940 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8290003e20 con 0x7f82a00cd9e0 2023-08-25T03:35:36.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.472+0000 7f8296ffd700 1 -- 172.21.15.2:0/3559268940 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8290011630 con 0x7f82a00cd9e0 2023-08-25T03:35:36.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.472+0000 7f82a83fe700 1 -- 172.21.15.2:0/3559268940 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a00b44b0 con 0x7f82a00cd9e0 2023-08-25T03:35:36.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.472+0000 7f8296ffd700 1 -- 172.21.15.2:0/3559268940 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82900119a0 con 0x7f82a00cd9e0 2023-08-25T03:35:36.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.472+0000 7f82a83fe700 1 -- 172.21.15.2:0/3559268940 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a00c9080 con 0x7f82a00cd9e0 2023-08-25T03:35:36.489 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.473+0000 7f82a83fe700 1 -- 172.21.15.2:0/3559268940 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f82a00ce4a0 con 0x7f82a00cd9e0 2023-08-25T03:35:36.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.474+0000 7f8296ffd700 1 -- 172.21.15.2:0/3559268940 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8290021070 con 0x7f82a00cd9e0 2023-08-25T03:35:36.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.474+0000 7f8296ffd700 1 --2- 172.21.15.2:0/3559268940 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f828406a0e0 0x7f828406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:36.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.474+0000 7f8296ffd700 1 -- 172.21.15.2:0/3559268940 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8290089c50 con 0x7f82a00cd9e0 2023-08-25T03:35:36.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.474+0000 7f82a5999700 1 --2- 172.21.15.2:0/3559268940 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f828406a0e0 0x7f828406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:36.490 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.474+0000 7f8296ffd700 1 --2- 172.21.15.2:0/3559268940 >> [v2:172.21.15.2:6816/83694260,v1:172.21.15.2:6817/83694260] conn(0x7f8284070560 0x7f8284072980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:36.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.474+0000 7f8296ffd700 1 -- 172.21.15.2:0/3559268940 --> [v2:172.21.15.2:6816/83694260,v1:172.21.15.2:6817/83694260] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8284073050 con 0x7f8284070560 2023-08-25T03:35:36.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.474+0000 7f8296ffd700 1 -- 172.21.15.2:0/3559268940 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f829008a080 con 0x7f82a00cd9e0 2023-08-25T03:35:36.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.474+0000 7f82a699b700 1 --2- 172.21.15.2:0/3559268940 >> [v2:172.21.15.2:6816/83694260,v1:172.21.15.2:6817/83694260] conn(0x7f8284070560 0x7f8284072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:36.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.475+0000 7f82a5999700 1 --2- 172.21.15.2:0/3559268940 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f828406a0e0 0x7f828406c5a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f82a01f4490 tx=0x7f829c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:36.491 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.475+0000 7f82a699b700 1 --2- 172.21.15.2:0/3559268940 >> [v2:172.21.15.2:6816/83694260,v1:172.21.15.2:6817/83694260] conn(0x7f8284070560 0x7f8284072980 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:36.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.475+0000 7f8296ffd700 1 -- 172.21.15.2:0/3559268940 <== osd.2 v2:172.21.15.2:6816/83694260 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f8284073050 con 0x7f8284070560 2023-08-25T03:35:36.492 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.490+0000 7f82a83fe700 1 -- 172.21.15.2:0/3559268940 --> [v2:172.21.15.2:6816/83694260,v1:172.21.15.2:6817/83694260] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f82a006bb60 con 0x7f8284070560 2023-08-25T03:35:36.493 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.490+0000 7f8296ffd700 1 -- 172.21.15.2:0/3559268940 <== osd.2 v2:172.21.15.2:6816/83694260 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f8298005040 con 0x7f8284070560 2023-08-25T03:35:36.493 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-08-25T03:35:36.493 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-08-25T03:35:36.494 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-08-25T03:35:36.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.490+0000 7f82a83fe700 1 -- 172.21.15.2:0/3559268940 >> [v2:172.21.15.2:6816/83694260,v1:172.21.15.2:6817/83694260] conn(0x7f8284070560 msgr2=0x7f8284072980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:36.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.490+0000 7f82a83fe700 1 --2- 172.21.15.2:0/3559268940 >> [v2:172.21.15.2:6816/83694260,v1:172.21.15.2:6817/83694260] conn(0x7f8284070560 0x7f8284072980 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.491+0000 7f82a83fe700 1 -- 172.21.15.2:0/3559268940 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f828406a0e0 msgr2=0x7f828406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:36.495 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.491+0000 7f82a83fe700 1 --2- 172.21.15.2:0/3559268940 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f828406a0e0 0x7f828406c5a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f82a01f4490 tx=0x7f829c008040 comp rx=0 tx=0).stop 2023-08-25T03:35:36.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.491+0000 7f82a83fe700 1 -- 172.21.15.2:0/3559268940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82a00cd9e0 msgr2=0x7f82a01578b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:36.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.491+0000 7f82a83fe700 1 --2- 172.21.15.2:0/3559268940 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f82a00cd9e0 0x7f82a01578b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f8290009130 tx=0x7f8290004750 comp rx=0 tx=0).stop 2023-08-25T03:35:36.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.491+0000 7f82a699b700 1 -- 172.21.15.2:0/3559268940 reap_dead start 2023-08-25T03:35:36.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.491+0000 7f82a83fe700 1 -- 172.21.15.2:0/3559268940 shutdown_connections 2023-08-25T03:35:36.496 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.491+0000 7f82a83fe700 1 -- 172.21.15.2:0/3559268940 >> 172.21.15.2:0/3559268940 conn(0x7f82a0065fe0 msgr2=0x7f82a0161150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:36.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.491+0000 7f82a83fe700 1 -- 172.21.15.2:0/3559268940 shutdown_connections 2023-08-25T03:35:36.497 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.491+0000 7f82a83fe700 1 -- 172.21.15.2:0/3559268940 wait complete. 2023-08-25T03:35:36.504 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-08-25T03:35:36.630 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.630+0000 7fd2b195f700 1 Processor -- start 2023-08-25T03:35:36.631 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.630+0000 7fd2b195f700 1 -- start start 2023-08-25T03:35:36.633 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.631+0000 7fd2b195f700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd2ac0d1cb0 0x7fd2ac0b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:36.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.631+0000 7fd2b195f700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2ac0b5760 con 0x7fd2ac0d21b0 2023-08-25T03:35:36.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.631+0000 7fd2b195f700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2ac0b5a40 con 0x7fd2ac160a10 2023-08-25T03:35:36.634 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.631+0000 7fd2b195f700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2ac0d2fd0 con 0x7fd2ac0d1cb0 2023-08-25T03:35:36.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.631+0000 7fd2ab7fe700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fd2ac0d21b0 0x7fd2ac0b6270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48810/0 (socket says 172.21.15.2:48810) 2023-08-25T03:35:36.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.631+0000 7fd2ab7fe700 1 -- 172.21.15.2:0/529767316 learned_addr learned my addr 172.21.15.2:0/529767316 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:36.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.631+0000 7fd2aa7fc700 1 --2- 172.21.15.2:0/529767316 >> v2:172.21.15.31:3300/0 conn(0x7fd2ac0d1cb0 0x7fd2ac0b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:36.635 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.631+0000 7fd2a9ffb700 1 -- 172.21.15.2:0/529767316 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1913755591 0 0) 0x7fd2ac0b5760 con 0x7fd2ac0d21b0 2023-08-25T03:35:36.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.631+0000 7fd2a9ffb700 1 -- 172.21.15.2:0/529767316 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2940035b0 con 0x7fd2ac0d21b0 2023-08-25T03:35:36.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.631+0000 7fd2a9ffb700 1 -- 172.21.15.2:0/529767316 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4193576238 0 0) 0x7fd2940035b0 con 0x7fd2ac0d21b0 2023-08-25T03:35:36.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.631+0000 7fd2a9ffb700 1 -- 172.21.15.2:0/529767316 >> v2:172.21.15.31:3300/0 conn(0x7fd2ac0d1cb0 msgr2=0x7fd2ac0b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:36.636 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.631+0000 7fd2a9ffb700 1 --2- 172.21.15.2:0/529767316 >> v2:172.21.15.31:3300/0 conn(0x7fd2ac0d1cb0 0x7fd2ac0b4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.631+0000 7fd2a9ffb700 1 -- 172.21.15.2:0/529767316 >> v1:172.21.15.31:6789/0 conn(0x7fd2ac160a10 legacy=0x7fd2ac0d28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:36.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.631+0000 7fd2a9ffb700 1 -- 172.21.15.2:0/529767316 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ac0d0750 con 0x7fd2ac0d21b0 2023-08-25T03:35:36.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.631+0000 7fd2a9ffb700 1 -- 172.21.15.2:0/529767316 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7fd2a40019f0 con 0x7fd2ac0d21b0 2023-08-25T03:35:36.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.631+0000 7fd2a9ffb700 1 -- 172.21.15.2:0/529767316 >> v1:172.21.15.2:6789/0 conn(0x7fd2ac0d21b0 legacy=0x7fd2ac0b6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:36.637 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.631+0000 7fd2a9ffb700 1 --2- 172.21.15.2:0/529767316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd294003a00 0x7fd294003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:36.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.631+0000 7fd2a9ffb700 1 --2- 172.21.15.2:0/529767316 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd294004850 0x7fd294004cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:36.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.631+0000 7fd2a9ffb700 1 --2- 172.21.15.2:0/529767316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd294005210 0x7fd29400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:36.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.631+0000 7fd2a9ffb700 1 -- 172.21.15.2:0/529767316 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac0d2fd0 con 0x7fd294005210 2023-08-25T03:35:36.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.632+0000 7fd2a9ffb700 1 -- 172.21.15.2:0/529767316 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29400c0c0 con 0x7fd294003a00 2023-08-25T03:35:36.638 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.632+0000 7fd2a9ffb700 1 -- 172.21.15.2:0/529767316 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd29400c280 con 0x7fd294004850 2023-08-25T03:35:36.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.632+0000 7fd2ab7fe700 1 --2- 172.21.15.2:0/529767316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd294005210 0x7fd29400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:36.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.632+0000 7fd2aaffd700 1 --2- 172.21.15.2:0/529767316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd294003a00 0x7fd294003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:36.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.632+0000 7fd2aa7fc700 1 --2- 172.21.15.2:0/529767316 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd294004850 0x7fd294004cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:36.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.632+0000 7fd2ab7fe700 1 -- 172.21.15.2:0/529767316 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd294004850 msgr2=0x7fd294004cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:36.639 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.632+0000 7fd2ab7fe700 1 --2- 172.21.15.2:0/529767316 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd294004850 0x7fd294004cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.632+0000 7fd2ab7fe700 1 -- 172.21.15.2:0/529767316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd294003a00 msgr2=0x7fd294003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:36.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.632+0000 7fd2ab7fe700 1 --2- 172.21.15.2:0/529767316 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd294003a00 0x7fd294003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.632+0000 7fd2ab7fe700 1 -- 172.21.15.2:0/529767316 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2940035b0 con 0x7fd294005210 2023-08-25T03:35:36.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.632+0000 7fd2ab7fe700 1 -- 172.21.15.2:0/529767316 reap_dead start 2023-08-25T03:35:36.640 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.632+0000 7fd2ab7fe700 1 --2- 172.21.15.2:0/529767316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd294005210 0x7fd29400ba10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd2a400a830 tx=0x7fd294005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef83e3e3b9dc40d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:36.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.632+0000 7fd2a9ffb700 1 -- 172.21.15.2:0/529767316 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a400a2f0 con 0x7fd294005210 2023-08-25T03:35:36.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.632+0000 7fd2a9ffb700 1 -- 172.21.15.2:0/529767316 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a4012620 con 0x7fd294005210 2023-08-25T03:35:36.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.632+0000 7fd2a9ffb700 1 -- 172.21.15.2:0/529767316 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a40129d0 con 0x7fd294005210 2023-08-25T03:35:36.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.633+0000 7fd2b195f700 1 -- 172.21.15.2:0/529767316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd294005210 msgr2=0x7fd29400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:36.641 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.633+0000 7fd2b195f700 1 --2- 172.21.15.2:0/529767316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd294005210 0x7fd29400ba10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd2a400a830 tx=0x7fd294005880 comp rx=0 tx=0).stop 2023-08-25T03:35:36.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.633+0000 7fd2b195f700 1 -- 172.21.15.2:0/529767316 shutdown_connections 2023-08-25T03:35:36.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.633+0000 7fd2b195f700 1 --2- 172.21.15.2:0/529767316 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd294005210 0x7fd29400ba10 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.633+0000 7fd2b195f700 1 -- 172.21.15.2:0/529767316 >> 172.21.15.2:0/529767316 conn(0x7fd2ac0cfb90 msgr2=0x7fd2ac0cffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:36.642 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.633+0000 7fd2b195f700 1 -- 172.21.15.2:0/529767316 shutdown_connections 2023-08-25T03:35:36.643 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.633+0000 7fd2b195f700 1 -- 172.21.15.2:0/529767316 wait complete. 2023-08-25T03:35:36.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.634+0000 7fd2b195f700 1 Processor -- start 2023-08-25T03:35:36.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.634+0000 7fd2b195f700 1 -- start start 2023-08-25T03:35:36.645 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.635+0000 7fd2b195f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd294005210 0x7fd2ac152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:36.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.635+0000 7fd2b195f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ac1513f0 0x7fd2ac0b7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:36.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.635+0000 7fd2b195f700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd2ac149230 0x7fd2ac1496c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:36.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.635+0000 7fd2b195f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac151b60 con 0x7fd294005210 2023-08-25T03:35:36.646 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.635+0000 7fd2b195f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac152c70 con 0x7fd2ac1513f0 2023-08-25T03:35:36.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.635+0000 7fd2b195f700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ac149c00 con 0x7fd2ac149230 2023-08-25T03:35:36.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.635+0000 7fd2aaffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd294005210 0x7fd2ac152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:36.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.635+0000 7fd2aaffd700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd294005210 0x7fd2ac152730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33920/0 (socket says 172.21.15.2:33920) 2023-08-25T03:35:36.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.635+0000 7fd2aaffd700 1 -- 172.21.15.2:0/2015466579 learned_addr learned my addr 172.21.15.2:0/2015466579 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:36.647 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.635+0000 7fd2aa7fc700 1 --2- 172.21.15.2:0/2015466579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ac1513f0 0x7fd2ac0b7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:36.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.635+0000 7fd2ab7fe700 1 --2- 172.21.15.2:0/2015466579 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd2ac149230 0x7fd2ac1496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:36.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.635+0000 7fd2aaffd700 1 -- 172.21.15.2:0/2015466579 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd2ac149230 msgr2=0x7fd2ac1496c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:36.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.635+0000 7fd2aaffd700 1 --2- 172.21.15.2:0/2015466579 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd2ac149230 0x7fd2ac1496c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.635+0000 7fd2aaffd700 1 -- 172.21.15.2:0/2015466579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ac1513f0 msgr2=0x7fd2ac0b7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:36.648 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.635+0000 7fd2aaffd700 1 --2- 172.21.15.2:0/2015466579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ac1513f0 0x7fd2ac0b7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.635+0000 7fd2aaffd700 1 -- 172.21.15.2:0/2015466579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ac156260 con 0x7fd294005210 2023-08-25T03:35:36.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.635+0000 7fd2aa7fc700 1 --2- 172.21.15.2:0/2015466579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ac1513f0 0x7fd2ac0b7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:36.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.635+0000 7fd2ab7fe700 1 --2- 172.21.15.2:0/2015466579 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd2ac149230 0x7fd2ac1496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:36.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.635+0000 7fd2aaffd700 1 --2- 172.21.15.2:0/2015466579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd294005210 0x7fd2ac152730 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000f850 tx=0x7fd2a000fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:36.649 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.636+0000 7fd293fff700 1 -- 172.21.15.2:0/2015466579 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a00050d0 con 0x7fd294005210 2023-08-25T03:35:36.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.636+0000 7fd293fff700 1 -- 172.21.15.2:0/2015466579 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a0005230 con 0x7fd294005210 2023-08-25T03:35:36.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.636+0000 7fd2b195f700 1 -- 172.21.15.2:0/2015466579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2ac156540 con 0x7fd294005210 2023-08-25T03:35:36.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.636+0000 7fd293fff700 1 -- 172.21.15.2:0/2015466579 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a0003680 con 0x7fd294005210 2023-08-25T03:35:36.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.636+0000 7fd2b195f700 1 -- 172.21.15.2:0/2015466579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ac156b00 con 0x7fd294005210 2023-08-25T03:35:36.650 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.637+0000 7fd2b195f700 1 -- 172.21.15.2:0/2015466579 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2ac156cc0 con 0x7fd294005210 2023-08-25T03:35:36.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.637+0000 7fd293fff700 1 -- 172.21.15.2:0/2015466579 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd2a00037e0 con 0x7fd294005210 2023-08-25T03:35:36.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.637+0000 7fd293fff700 1 --2- 172.21.15.2:0/2015466579 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd29806a0e0 0x7fd29806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:36.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.638+0000 7fd293fff700 1 -- 172.21.15.2:0/2015466579 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd2a0087f70 con 0x7fd294005210 2023-08-25T03:35:36.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.638+0000 7fd2aa7fc700 1 --2- 172.21.15.2:0/2015466579 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd29806a0e0 0x7fd29806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:36.651 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.638+0000 7fd2aa7fc700 1 --2- 172.21.15.2:0/2015466579 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd29806a0e0 0x7fd29806c5a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0d2160 tx=0x7fd29c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:36.652 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.642+0000 7fd293fff700 1 -- 172.21.15.2:0/2015466579 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fd2a00ba990 con 0x7fd294005210 2023-08-25T03:35:36.843 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.842+0000 7fd2b195f700 1 -- 172.21.15.2:0/2015466579 --> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2ac0d0a30 con 0x7fd29806a0e0 2023-08-25T03:35:36.844 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.843+0000 7fd293fff700 1 -- 172.21.15.2:0/2015466579 <== mgr.4108 v2:172.21.15.2:6832/3528779779 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd29c009830 con 0x7fd29806a0e0 2023-08-25T03:35:36.845 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.844+0000 7fd2b195f700 1 -- 172.21.15.2:0/2015466579 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd29806a0e0 msgr2=0x7fd29806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:36.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.844+0000 7fd2b195f700 1 --2- 172.21.15.2:0/2015466579 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd29806a0e0 0x7fd29806c5a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0d2160 tx=0x7fd29c008040 comp rx=0 tx=0).stop 2023-08-25T03:35:36.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.844+0000 7fd2b195f700 1 -- 172.21.15.2:0/2015466579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd294005210 msgr2=0x7fd2ac152730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:36.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.844+0000 7fd2b195f700 1 --2- 172.21.15.2:0/2015466579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd294005210 0x7fd2ac152730 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd2a000f850 tx=0x7fd2a000fc10 comp rx=0 tx=0).stop 2023-08-25T03:35:36.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.845+0000 7fd2b195f700 1 -- 172.21.15.2:0/2015466579 shutdown_connections 2023-08-25T03:35:36.846 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.845+0000 7fd2b195f700 1 --2- 172.21.15.2:0/2015466579 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd29806a0e0 0x7fd29806c5a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.845+0000 7fd2b195f700 1 --2- 172.21.15.2:0/2015466579 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd294005210 0x7fd2ac152730 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.845+0000 7fd2b195f700 1 --2- 172.21.15.2:0/2015466579 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd2ac149230 0x7fd2ac1496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.845+0000 7fd2b195f700 1 --2- 172.21.15.2:0/2015466579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2ac1513f0 0x7fd2ac0b7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.847 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.845+0000 7fd2b195f700 1 -- 172.21.15.2:0/2015466579 >> 172.21.15.2:0/2015466579 conn(0x7fd2ac0cfb90 msgr2=0x7fd2ac0b6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:36.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.845+0000 7fd2b195f700 1 -- 172.21.15.2:0/2015466579 shutdown_connections 2023-08-25T03:35:36.848 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.845+0000 7fd2b195f700 1 -- 172.21.15.2:0/2015466579 wait complete. 2023-08-25T03:35:36.848 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 2 to deep-scrub 2023-08-25T03:35:36.857 INFO:tasks.ceph:Scrubbing osd.3 2023-08-25T03:35:36.858 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-08-25T03:35:36.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.986+0000 7f1daa58e700 1 Processor -- start 2023-08-25T03:35:36.987 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.986+0000 7f1daa58e700 1 -- start start 2023-08-25T03:35:36.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.987+0000 7f1daa58e700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f1da406c760 0x7f1da406cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:36.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.987+0000 7f1daa58e700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f1da406d2b0 0x7f1da406aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:36.989 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.987+0000 7f1daa58e700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da406a660 con 0x7f1da415b590 2023-08-25T03:35:36.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.987+0000 7f1daa58e700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da406d740 con 0x7f1da406d2b0 2023-08-25T03:35:36.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.987+0000 7f1daa58e700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da40cb630 con 0x7f1da406c760 2023-08-25T03:35:36.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.987+0000 7f1da8b2b700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f1da406d2b0 0x7f1da406aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:36.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.987+0000 7f1da3fff700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f1da415b590 0x7f1da40caf20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48826/0 (socket says 172.21.15.2:48826) 2023-08-25T03:35:36.990 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.987+0000 7f1da3fff700 1 -- 172.21.15.2:0/4025309350 learned_addr learned my addr 172.21.15.2:0/4025309350 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:36.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.987+0000 7f1da37fe700 1 --2- 172.21.15.2:0/4025309350 >> v2:172.21.15.31:3300/0 conn(0x7f1da406c760 0x7f1da406cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:36.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.987+0000 7f1da8b2b700 1 -- 172.21.15.2:0/4025309350 >> v2:172.21.15.31:3300/0 conn(0x7f1da406c760 msgr2=0x7f1da406cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:36.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.987+0000 7f1da8b2b700 1 --2- 172.21.15.2:0/4025309350 >> v2:172.21.15.31:3300/0 conn(0x7f1da406c760 0x7f1da406cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.987+0000 7f1da8b2b700 1 -- 172.21.15.2:0/4025309350 >> v1:172.21.15.2:6789/0 conn(0x7f1da415b590 legacy=0x7f1da40caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:36.991 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.987+0000 7f1da8b2b700 1 -- 172.21.15.2:0/4025309350 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da40cbe50 con 0x7f1da406d2b0 2023-08-25T03:35:36.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.987+0000 7f1da2ffd700 1 -- 172.21.15.2:0/4025309350 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2088408337 0 0) 0x7f1da406a660 con 0x7f1da415b590 2023-08-25T03:35:36.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.987+0000 7f1da8b2b700 1 --2- 172.21.15.2:0/4025309350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1da406d2b0 0x7f1da406aaf0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1da406bda0 tx=0x7f1d9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a97a2ae3014549a6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:36.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.988+0000 7f1da2ffd700 1 -- 172.21.15.2:0/4025309350 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c00e070 con 0x7f1da406d2b0 2023-08-25T03:35:36.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.988+0000 7f1da2ffd700 1 -- 172.21.15.2:0/4025309350 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9c00bb20 con 0x7f1da406d2b0 2023-08-25T03:35:36.992 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.988+0000 7f1da2ffd700 1 -- 172.21.15.2:0/4025309350 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c012430 con 0x7f1da406d2b0 2023-08-25T03:35:36.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.988+0000 7f1daa58e700 1 -- 172.21.15.2:0/4025309350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1da406d2b0 msgr2=0x7f1da406aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:36.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.988+0000 7f1daa58e700 1 --2- 172.21.15.2:0/4025309350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1da406d2b0 0x7f1da406aaf0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1da406bda0 tx=0x7f1d9c00b330 comp rx=0 tx=0).stop 2023-08-25T03:35:36.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.988+0000 7f1daa58e700 1 -- 172.21.15.2:0/4025309350 shutdown_connections 2023-08-25T03:35:36.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.988+0000 7f1daa58e700 1 --2- 172.21.15.2:0/4025309350 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1da406d2b0 0x7f1da406aaf0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.993 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.988+0000 7f1daa58e700 1 --2- 172.21.15.2:0/4025309350 >> v2:172.21.15.31:3300/0 conn(0x7f1da406c760 0x7f1da406cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:36.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.988+0000 7f1daa58e700 1 -- 172.21.15.2:0/4025309350 >> 172.21.15.2:0/4025309350 conn(0x7f1da40cc020 msgr2=0x7f1da40cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:36.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.988+0000 7f1daa58e700 1 -- 172.21.15.2:0/4025309350 shutdown_connections 2023-08-25T03:35:36.994 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.988+0000 7f1daa58e700 1 -- 172.21.15.2:0/4025309350 wait complete. 2023-08-25T03:35:36.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.990+0000 7f1daa58e700 1 Processor -- start 2023-08-25T03:35:36.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.990+0000 7f1daa58e700 1 -- start start 2023-08-25T03:35:36.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.991+0000 7f1daa58e700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1da406c760 0x7f1da40c6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:36.997 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.991+0000 7f1daa58e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da406d2b0 0x7f1da4152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:36.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.991+0000 7f1daa58e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1da415b590 0x7f1da40b2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:36.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.991+0000 7f1daa58e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da40b3fc0 con 0x7f1da415b590 2023-08-25T03:35:36.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.991+0000 7f1daa58e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da40ce1f0 con 0x7f1da406d2b0 2023-08-25T03:35:36.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.991+0000 7f1daa58e700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da40cd950 con 0x7f1da406c760 2023-08-25T03:35:36.998 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.991+0000 7f1da8b2b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1da415b590 0x7f1da40b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:36.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.991+0000 7f1da8b2b700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1da415b590 0x7f1da40b2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33938/0 (socket says 172.21.15.2:33938) 2023-08-25T03:35:36.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.991+0000 7f1da8b2b700 1 -- 172.21.15.2:0/3882216310 learned_addr learned my addr 172.21.15.2:0/3882216310 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:36.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.991+0000 7f1da3fff700 1 --2- 172.21.15.2:0/3882216310 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1da406c760 0x7f1da40c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:36.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.991+0000 7f1da37fe700 1 --2- 172.21.15.2:0/3882216310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da406d2b0 0x7f1da4152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:36.999 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.991+0000 7f1da8b2b700 1 -- 172.21.15.2:0/3882216310 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1da406c760 msgr2=0x7f1da40c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:37.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.992+0000 7f1da8b2b700 1 --2- 172.21.15.2:0/3882216310 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1da406c760 0x7f1da40c6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.992+0000 7f1da8b2b700 1 -- 172.21.15.2:0/3882216310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da406d2b0 msgr2=0x7f1da4152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:37.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.992+0000 7f1da8b2b700 1 --2- 172.21.15.2:0/3882216310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1da406d2b0 0x7f1da4152b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.992+0000 7f1da8b2b700 1 -- 172.21.15.2:0/3882216310 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d9c009a70 con 0x7f1da415b590 2023-08-25T03:35:37.000 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.992+0000 7f1da8b2b700 1 --2- 172.21.15.2:0/3882216310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1da415b590 0x7f1da40b2440 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c014040 tx=0x7f1d9c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:37.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.992+0000 7f1da0ff9700 1 -- 172.21.15.2:0/3882216310 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c012430 con 0x7f1da415b590 2023-08-25T03:35:37.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.992+0000 7f1da0ff9700 1 -- 172.21.15.2:0/3882216310 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9c020070 con 0x7f1da415b590 2023-08-25T03:35:37.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.992+0000 7f1daa58e700 1 -- 172.21.15.2:0/3882216310 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da40cdc30 con 0x7f1da415b590 2023-08-25T03:35:37.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.992+0000 7f1da0ff9700 1 -- 172.21.15.2:0/3882216310 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c004280 con 0x7f1da415b590 2023-08-25T03:35:37.001 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.992+0000 7f1daa58e700 1 -- 172.21.15.2:0/3882216310 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da40cd170 con 0x7f1da415b590 2023-08-25T03:35:37.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.993+0000 7f1daa58e700 1 -- 172.21.15.2:0/3882216310 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1da40cb890 con 0x7f1da415b590 2023-08-25T03:35:37.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.994+0000 7f1da0ff9700 1 -- 172.21.15.2:0/3882216310 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f1d9c0043e0 con 0x7f1da415b590 2023-08-25T03:35:37.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.994+0000 7f1da0ff9700 1 --2- 172.21.15.2:0/3882216310 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1d8406a0e0 0x7f1d8406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:37.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.994+0000 7f1da0ff9700 1 -- 172.21.15.2:0/3882216310 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f1d9c022070 con 0x7f1da415b590 2023-08-25T03:35:37.002 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.994+0000 7f1da3fff700 1 --2- 172.21.15.2:0/3882216310 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1d8406a0e0 0x7f1d8406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:37.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.994+0000 7f1da0ff9700 1 --2- 172.21.15.2:0/3882216310 >> [v2:172.21.15.2:6822/2180868296,v1:172.21.15.2:6824/2180868296] conn(0x7f1d84070560 0x7f1d84072980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:37.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.994+0000 7f1da0ff9700 1 -- 172.21.15.2:0/3882216310 --> [v2:172.21.15.2:6822/2180868296,v1:172.21.15.2:6824/2180868296] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1d84073050 con 0x7f1d84070560 2023-08-25T03:35:37.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.994+0000 7f1da0ff9700 1 -- 172.21.15.2:0/3882216310 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d9c08a8b0 con 0x7f1da415b590 2023-08-25T03:35:37.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.994+0000 7f1da37fe700 1 --2- 172.21.15.2:0/3882216310 >> [v2:172.21.15.2:6822/2180868296,v1:172.21.15.2:6824/2180868296] conn(0x7f1d84070560 0x7f1d84072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:37.003 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.994+0000 7f1da3fff700 1 --2- 172.21.15.2:0/3882216310 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1d8406a0e0 0x7f1d8406c5a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c000f60 tx=0x7f1d8c007e70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:37.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.995+0000 7f1da37fe700 1 --2- 172.21.15.2:0/3882216310 >> [v2:172.21.15.2:6822/2180868296,v1:172.21.15.2:6824/2180868296] conn(0x7f1d84070560 0x7f1d84072980 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:37.004 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:36.995+0000 7f1da0ff9700 1 -- 172.21.15.2:0/3882216310 <== osd.3 v2:172.21.15.2:6822/2180868296 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f1d84073050 con 0x7f1d84070560 2023-08-25T03:35:37.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.010+0000 7f1daa58e700 1 -- 172.21.15.2:0/3882216310 --> [v2:172.21.15.2:6822/2180868296,v1:172.21.15.2:6824/2180868296] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f1da415b970 con 0x7f1d84070560 2023-08-25T03:35:37.011 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.010+0000 7f1da0ff9700 1 -- 172.21.15.2:0/3882216310 <== osd.3 v2:172.21.15.2:6822/2180868296 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f1d94009d30 con 0x7f1d84070560 2023-08-25T03:35:37.011 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-08-25T03:35:37.012 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-08-25T03:35:37.012 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-08-25T03:35:37.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.011+0000 7f1daa58e700 1 -- 172.21.15.2:0/3882216310 >> [v2:172.21.15.2:6822/2180868296,v1:172.21.15.2:6824/2180868296] conn(0x7f1d84070560 msgr2=0x7f1d84072980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:37.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.011+0000 7f1daa58e700 1 --2- 172.21.15.2:0/3882216310 >> [v2:172.21.15.2:6822/2180868296,v1:172.21.15.2:6824/2180868296] conn(0x7f1d84070560 0x7f1d84072980 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.013 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.011+0000 7f1daa58e700 1 -- 172.21.15.2:0/3882216310 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1d8406a0e0 msgr2=0x7f1d8406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:37.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.011+0000 7f1daa58e700 1 --2- 172.21.15.2:0/3882216310 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f1d8406a0e0 0x7f1d8406c5a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1d8c000f60 tx=0x7f1d8c007e70 comp rx=0 tx=0).stop 2023-08-25T03:35:37.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.011+0000 7f1daa58e700 1 -- 172.21.15.2:0/3882216310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1da415b590 msgr2=0x7f1da40b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:37.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.011+0000 7f1daa58e700 1 --2- 172.21.15.2:0/3882216310 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f1da415b590 0x7f1da40b2440 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c014040 tx=0x7f1d9c004ad0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.011+0000 7f1da8b2b700 1 -- 172.21.15.2:0/3882216310 reap_dead start 2023-08-25T03:35:37.014 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.011+0000 7f1daa58e700 1 -- 172.21.15.2:0/3882216310 shutdown_connections 2023-08-25T03:35:37.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.011+0000 7f1daa58e700 1 -- 172.21.15.2:0/3882216310 >> 172.21.15.2:0/3882216310 conn(0x7f1da40cc020 msgr2=0x7f1da40aefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:37.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.011+0000 7f1daa58e700 1 -- 172.21.15.2:0/3882216310 shutdown_connections 2023-08-25T03:35:37.015 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.011+0000 7f1daa58e700 1 -- 172.21.15.2:0/3882216310 wait complete. 2023-08-25T03:35:37.023 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-08-25T03:35:37.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.148+0000 7f7735594700 1 Processor -- start 2023-08-25T03:35:37.149 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.148+0000 7f7735594700 1 -- start start 2023-08-25T03:35:37.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.149+0000 7f7735594700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f773015d430 0x7f77300d0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:37.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.149+0000 7f7735594700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f77300d1e10 0x7f77300b0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:37.151 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.149+0000 7f7735594700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77300cf290 con 0x7f77300d1050 2023-08-25T03:35:37.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.149+0000 7f7735594700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77300d3de0 con 0x7f773015d430 2023-08-25T03:35:37.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.149+0000 7f7735594700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77300afe90 con 0x7f77300d1e10 2023-08-25T03:35:37.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.149+0000 7f772effd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f773015d430 0x7f77300d0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:37.152 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.149+0000 7f772effd700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f773015d430 0x7f77300d0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33944/0 (socket says 172.21.15.2:33944) 2023-08-25T03:35:37.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.149+0000 7f772effd700 1 -- 172.21.15.2:0/1524849759 learned_addr learned my addr 172.21.15.2:0/1524849759 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:37.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.149+0000 7f772f7fe700 1 --2- 172.21.15.2:0/1524849759 >> v2:172.21.15.31:3300/0 conn(0x7f77300d1e10 0x7f77300b0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:37.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.149+0000 7f772effd700 1 -- 172.21.15.2:0/1524849759 >> v2:172.21.15.31:3300/0 conn(0x7f77300d1e10 msgr2=0x7f77300b0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:37.153 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.149+0000 7f772effd700 1 --2- 172.21.15.2:0/1524849759 >> v2:172.21.15.31:3300/0 conn(0x7f77300d1e10 0x7f77300b0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.149+0000 7f772effd700 1 -- 172.21.15.2:0/1524849759 >> v1:172.21.15.31:6789/0 conn(0x7f77300d1050 legacy=0x7f77300d1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:37.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.149+0000 7f772effd700 1 -- 172.21.15.2:0/1524849759 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77300d40c0 con 0x7f773015d430 2023-08-25T03:35:37.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.149+0000 7f772f7fe700 1 --2- 172.21.15.2:0/1524849759 >> v2:172.21.15.31:3300/0 conn(0x7f77300d1e10 0x7f77300b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:37.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.149+0000 7f772effd700 1 --2- 172.21.15.2:0/1524849759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f773015d430 0x7f77300d0160 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f773006aa60 tx=0x7f7724009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f7a95c734167f3d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:37.154 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.149+0000 7f772dffb700 1 -- 172.21.15.2:0/1524849759 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772400b870 con 0x7f773015d430 2023-08-25T03:35:37.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.150+0000 7f772dffb700 1 -- 172.21.15.2:0/1524849759 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772400b9d0 con 0x7f773015d430 2023-08-25T03:35:37.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.150+0000 7f772dffb700 1 -- 172.21.15.2:0/1524849759 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772400bd60 con 0x7f773015d430 2023-08-25T03:35:37.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.150+0000 7f7735594700 1 -- 172.21.15.2:0/1524849759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f773015d430 msgr2=0x7f77300d0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:37.155 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.150+0000 7f7735594700 1 --2- 172.21.15.2:0/1524849759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f773015d430 0x7f77300d0160 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f773006aa60 tx=0x7f7724009bd0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.150+0000 7f7735594700 1 -- 172.21.15.2:0/1524849759 shutdown_connections 2023-08-25T03:35:37.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.150+0000 7f7735594700 1 --2- 172.21.15.2:0/1524849759 >> v2:172.21.15.31:3300/0 conn(0x7f77300d1e10 0x7f77300b0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.150+0000 7f7735594700 1 --2- 172.21.15.2:0/1524849759 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f773015d430 0x7f77300d0160 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.150+0000 7f7735594700 1 -- 172.21.15.2:0/1524849759 >> 172.21.15.2:0/1524849759 conn(0x7f77300d3120 msgr2=0x7f77300d3530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:37.156 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.150+0000 7f7735594700 1 -- 172.21.15.2:0/1524849759 shutdown_connections 2023-08-25T03:35:37.157 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.150+0000 7f7735594700 1 -- 172.21.15.2:0/1524849759 wait complete. 2023-08-25T03:35:37.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.151+0000 7f7735594700 1 Processor -- start 2023-08-25T03:35:37.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.152+0000 7f7735594700 1 -- start start 2023-08-25T03:35:37.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.152+0000 7f7735594700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77300d1050 0x7f773015fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:37.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.152+0000 7f7735594700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f77300d1e10 0x7f77300ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:37.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.152+0000 7f7735594700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f773015d430 0x7f77300b32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:37.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.152+0000 7f7735594700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77300cc0f0 con 0x7f773015d430 2023-08-25T03:35:37.160 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.152+0000 7f7735594700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77300cb240 con 0x7f77300d1050 2023-08-25T03:35:37.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.152+0000 7f7735594700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77300cb4e0 con 0x7f77300d1e10 2023-08-25T03:35:37.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.152+0000 7f772f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f773015d430 0x7f77300b32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:37.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.152+0000 7f772f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f773015d430 0x7f77300b32e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33952/0 (socket says 172.21.15.2:33952) 2023-08-25T03:35:37.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.152+0000 7f772f7fe700 1 -- 172.21.15.2:0/4015234494 learned_addr learned my addr 172.21.15.2:0/4015234494 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:37.161 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.152+0000 7f772e7fc700 1 --2- 172.21.15.2:0/4015234494 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f77300d1e10 0x7f77300ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:37.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.153+0000 7f772effd700 1 --2- 172.21.15.2:0/4015234494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77300d1050 0x7f773015fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:37.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.153+0000 7f772f7fe700 1 -- 172.21.15.2:0/4015234494 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f77300d1e10 msgr2=0x7f77300ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:37.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.153+0000 7f772f7fe700 1 --2- 172.21.15.2:0/4015234494 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f77300d1e10 0x7f77300ccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.153+0000 7f772f7fe700 1 -- 172.21.15.2:0/4015234494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77300d1050 msgr2=0x7f773015fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:37.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.153+0000 7f772f7fe700 1 --2- 172.21.15.2:0/4015234494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77300d1050 0x7f773015fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.153+0000 7f772f7fe700 1 -- 172.21.15.2:0/4015234494 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7724009550 con 0x7f773015d430 2023-08-25T03:35:37.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.153+0000 7f772effd700 1 --2- 172.21.15.2:0/4015234494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77300d1050 0x7f773015fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:37.163 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.153+0000 7f772f7fe700 1 --2- 172.21.15.2:0/4015234494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f773015d430 0x7f77300b32e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7728007ec0 tx=0x7f772800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:37.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.153+0000 7f77177fe700 1 -- 172.21.15.2:0/4015234494 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f772800de30 con 0x7f773015d430 2023-08-25T03:35:37.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.153+0000 7f77177fe700 1 -- 172.21.15.2:0/4015234494 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f772800f040 con 0x7f773015d430 2023-08-25T03:35:37.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.153+0000 7f7735594700 1 -- 172.21.15.2:0/4015234494 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77300cb7c0 con 0x7f773015d430 2023-08-25T03:35:37.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.153+0000 7f77177fe700 1 -- 172.21.15.2:0/4015234494 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7728003c60 con 0x7f773015d430 2023-08-25T03:35:37.164 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.153+0000 7f7735594700 1 -- 172.21.15.2:0/4015234494 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77300cbd80 con 0x7f773015d430 2023-08-25T03:35:37.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.154+0000 7f7735594700 1 -- 172.21.15.2:0/4015234494 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77300cc260 con 0x7f773015d430 2023-08-25T03:35:37.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.154+0000 7f77177fe700 1 -- 172.21.15.2:0/4015234494 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f7728004030 con 0x7f773015d430 2023-08-25T03:35:37.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.155+0000 7f77177fe700 1 --2- 172.21.15.2:0/4015234494 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f771c06a0e0 0x7f771c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:37.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.155+0000 7f77177fe700 1 -- 172.21.15.2:0/4015234494 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f7728088430 con 0x7f773015d430 2023-08-25T03:35:37.165 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.155+0000 7f772effd700 1 --2- 172.21.15.2:0/4015234494 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f771c06a0e0 0x7f771c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:37.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.155+0000 7f772effd700 1 --2- 172.21.15.2:0/4015234494 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f771c06a0e0 0x7f771c06c5a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7724014040 tx=0x7f7724008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:37.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.160+0000 7f77177fe700 1 -- 172.21.15.2:0/4015234494 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f7728053da0 con 0x7f773015d430 2023-08-25T03:35:37.358 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.358+0000 7f7735594700 1 -- 172.21.15.2:0/4015234494 --> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f77300b0170 con 0x7f771c06a0e0 2023-08-25T03:35:37.359 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.358+0000 7f77177fe700 1 -- 172.21.15.2:0/4015234494 <== mgr.4108 v2:172.21.15.2:6832/3528779779 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f772400bbc0 con 0x7f771c06a0e0 2023-08-25T03:35:37.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.360+0000 7f7735594700 1 -- 172.21.15.2:0/4015234494 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f771c06a0e0 msgr2=0x7f771c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:37.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.360+0000 7f7735594700 1 --2- 172.21.15.2:0/4015234494 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f771c06a0e0 0x7f771c06c5a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7724014040 tx=0x7f7724008030 comp rx=0 tx=0).stop 2023-08-25T03:35:37.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.360+0000 7f7735594700 1 -- 172.21.15.2:0/4015234494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f773015d430 msgr2=0x7f77300b32e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:37.361 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.360+0000 7f7735594700 1 --2- 172.21.15.2:0/4015234494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f773015d430 0x7f77300b32e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7728007ec0 tx=0x7f772800d3b0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.360+0000 7f7735594700 1 -- 172.21.15.2:0/4015234494 shutdown_connections 2023-08-25T03:35:37.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.360+0000 7f7735594700 1 --2- 172.21.15.2:0/4015234494 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f771c06a0e0 0x7f771c06c5a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.360+0000 7f7735594700 1 --2- 172.21.15.2:0/4015234494 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f773015d430 0x7f77300b32e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.360+0000 7f7735594700 1 --2- 172.21.15.2:0/4015234494 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f77300d1050 0x7f773015fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.362 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.360+0000 7f7735594700 1 --2- 172.21.15.2:0/4015234494 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f77300d1e10 0x7f77300ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.360+0000 7f7735594700 1 -- 172.21.15.2:0/4015234494 >> 172.21.15.2:0/4015234494 conn(0x7f77300d3120 msgr2=0x7f77300b7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:37.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.360+0000 7f7735594700 1 -- 172.21.15.2:0/4015234494 shutdown_connections 2023-08-25T03:35:37.363 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.360+0000 7f7735594700 1 -- 172.21.15.2:0/4015234494 wait complete. 2023-08-25T03:35:37.363 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 3 to deep-scrub 2023-08-25T03:35:37.374 INFO:tasks.ceph:Scrubbing osd.4 2023-08-25T03:35:37.374 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-08-25T03:35:37.513 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.502+0000 7f2c66f41700 1 Processor -- start 2023-08-25T03:35:37.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.502+0000 7f2c66f41700 1 -- start start 2023-08-25T03:35:37.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.503+0000 7f2c66f41700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2c6015d4d0 0x7f2c60161de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:37.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.503+0000 7f2c66f41700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c600ce1c0 con 0x7f2c6015d8b0 2023-08-25T03:35:37.514 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.503+0000 7f2c66f41700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c6006b5a0 con 0x7f2c600cd9e0 2023-08-25T03:35:37.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.503+0000 7f2c66f41700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6006b880 con 0x7f2c6015d4d0 2023-08-25T03:35:37.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.503+0000 7f2c5ffff700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f2c6015d8b0 0x7f2c6006c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48856/0 (socket says 172.21.15.2:48856) 2023-08-25T03:35:37.515 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.503+0000 7f2c5ffff700 1 -- 172.21.15.2:0/2907193108 learned_addr learned my addr 172.21.15.2:0/2907193108 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:37.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.503+0000 7f2c64cdd700 1 --2- 172.21.15.2:0/2907193108 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2c6015d4d0 0x7f2c60161de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:37.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.503+0000 7f2c5f7fe700 1 -- 172.21.15.2:0/2907193108 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1129762351 0 0) 0x7f2c600ce1c0 con 0x7f2c6015d8b0 2023-08-25T03:35:37.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.503+0000 7f2c5f7fe700 1 -- 172.21.15.2:0/2907193108 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c500035b0 con 0x7f2c6015d8b0 2023-08-25T03:35:37.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.504+0000 7f2c5f7fe700 1 -- 172.21.15.2:0/2907193108 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1474943292 0 0) 0x7f2c500035b0 con 0x7f2c6015d8b0 2023-08-25T03:35:37.516 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.504+0000 7f2c5f7fe700 1 -- 172.21.15.2:0/2907193108 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2c6015d4d0 msgr2=0x7f2c60161de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:37.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.504+0000 7f2c5f7fe700 1 --2- 172.21.15.2:0/2907193108 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2c6015d4d0 0x7f2c60161de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.504+0000 7f2c5f7fe700 1 -- 172.21.15.2:0/2907193108 >> v1:172.21.15.31:6789/0 conn(0x7f2c600cd9e0 legacy=0x7f2c600cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:37.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.504+0000 7f2c5f7fe700 1 -- 172.21.15.2:0/2907193108 --> v1:172.21.15.2:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c6006bb60 con 0x7f2c6015d8b0 2023-08-25T03:35:37.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.504+0000 7f2c5f7fe700 1 -- 172.21.15.2:0/2907193108 <== mon.0 v1:172.21.15.2:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1603474726 0 0) 0x7f2c4c001670 con 0x7f2c6015d8b0 2023-08-25T03:35:37.517 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.504+0000 7f2c5f7fe700 1 -- 172.21.15.2:0/2907193108 >> v1:172.21.15.2:6789/0 conn(0x7f2c6015d8b0 legacy=0x7f2c6006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:37.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.504+0000 7f2c5f7fe700 1 --2- 172.21.15.2:0/2907193108 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c50003a30 0x7f2c50003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:37.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.504+0000 7f2c5f7fe700 1 --2- 172.21.15.2:0/2907193108 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2c6015d4d0 0x7f2c50004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:37.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.504+0000 7f2c5f7fe700 1 --2- 172.21.15.2:0/2907193108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c50004d90 0x7f2c50005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:37.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.504+0000 7f2c5f7fe700 1 -- 172.21.15.2:0/2907193108 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6006b880 con 0x7f2c50003a30 2023-08-25T03:35:37.518 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.504+0000 7f2c5f7fe700 1 -- 172.21.15.2:0/2907193108 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5000bac0 con 0x7f2c50004d90 2023-08-25T03:35:37.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.504+0000 7f2c5f7fe700 1 -- 172.21.15.2:0/2907193108 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5000bc80 con 0x7f2c6015d4d0 2023-08-25T03:35:37.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.504+0000 7f2c64cdd700 1 --2- 172.21.15.2:0/2907193108 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c50003a30 0x7f2c50003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:37.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.504+0000 7f2c5ffff700 1 --2- 172.21.15.2:0/2907193108 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2c6015d4d0 0x7f2c50004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:37.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.504+0000 7f2c654de700 1 --2- 172.21.15.2:0/2907193108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c50004d90 0x7f2c50005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:37.519 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.504+0000 7f2c64cdd700 1 -- 172.21.15.2:0/2907193108 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2c6015d4d0 msgr2=0x7f2c50004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:37.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.504+0000 7f2c64cdd700 1 --2- 172.21.15.2:0/2907193108 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2c6015d4d0 0x7f2c50004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.504+0000 7f2c64cdd700 1 -- 172.21.15.2:0/2907193108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c50004d90 msgr2=0x7f2c50005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:37.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.504+0000 7f2c64cdd700 1 --2- 172.21.15.2:0/2907193108 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c50004d90 0x7f2c50005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.504+0000 7f2c64cdd700 1 -- 172.21.15.2:0/2907193108 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c500035b0 con 0x7f2c50003a30 2023-08-25T03:35:37.520 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.504+0000 7f2c5ffff700 1 --2- 172.21.15.2:0/2907193108 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2c6015d4d0 0x7f2c50004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:37.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.504+0000 7f2c64cdd700 1 --2- 172.21.15.2:0/2907193108 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c50003a30 0x7f2c50003e50 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2c54009b30 tx=0x7f2c50005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e63ea4ee9ca95d9 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:37.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.505+0000 7f2c5f7fe700 1 -- 172.21.15.2:0/2907193108 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5400e6c0 con 0x7f2c50003a30 2023-08-25T03:35:37.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.505+0000 7f2c5f7fe700 1 -- 172.21.15.2:0/2907193108 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5400e820 con 0x7f2c50003a30 2023-08-25T03:35:37.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.505+0000 7f2c5f7fe700 1 -- 172.21.15.2:0/2907193108 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5400eb90 con 0x7f2c50003a30 2023-08-25T03:35:37.521 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.505+0000 7f2c66f41700 1 -- 172.21.15.2:0/2907193108 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c50003a30 msgr2=0x7f2c50003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:37.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.505+0000 7f2c66f41700 1 --2- 172.21.15.2:0/2907193108 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c50003a30 0x7f2c50003e50 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2c54009b30 tx=0x7f2c50005880 comp rx=0 tx=0).stop 2023-08-25T03:35:37.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.505+0000 7f2c654de700 1 -- 172.21.15.2:0/2907193108 reap_dead start 2023-08-25T03:35:37.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.505+0000 7f2c66f41700 1 -- 172.21.15.2:0/2907193108 shutdown_connections 2023-08-25T03:35:37.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.505+0000 7f2c66f41700 1 -- 172.21.15.2:0/2907193108 >> 172.21.15.2:0/2907193108 conn(0x7f2c60065fe0 msgr2=0x7f2c600663c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:37.522 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.505+0000 7f2c66f41700 1 -- 172.21.15.2:0/2907193108 shutdown_connections 2023-08-25T03:35:37.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.505+0000 7f2c66f41700 1 -- 172.21.15.2:0/2907193108 wait complete. 2023-08-25T03:35:37.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.506+0000 7f2c66f41700 1 Processor -- start 2023-08-25T03:35:37.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.506+0000 7f2c66f41700 1 -- start start 2023-08-25T03:35:37.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.507+0000 7f2c66f41700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2c50003a30 0x7f2c600ad440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:37.523 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.507+0000 7f2c66f41700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c60151800 0x7f2c600c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:37.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.507+0000 7f2c66f41700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c600afd10 0x7f2c600b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:37.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.507+0000 7f2c66f41700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c600b0700 con 0x7f2c600afd10 2023-08-25T03:35:37.524 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.507+0000 7f2c66f41700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c600ca110 con 0x7f2c60151800 2023-08-25T03:35:37.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.507+0000 7f2c66f41700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c600b1d30 con 0x7f2c50003a30 2023-08-25T03:35:37.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.507+0000 7f2c654de700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c600afd10 0x7f2c600b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:37.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.507+0000 7f2c654de700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c600afd10 0x7f2c600b01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33972/0 (socket says 172.21.15.2:33972) 2023-08-25T03:35:37.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.507+0000 7f2c654de700 1 -- 172.21.15.2:0/66784043 learned_addr learned my addr 172.21.15.2:0/66784043 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:37.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.508+0000 7f2c654de700 1 -- 172.21.15.2:0/66784043 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2c50003a30 msgr2=0x7f2c600ad440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:37.525 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.508+0000 7f2c64cdd700 1 --2- 172.21.15.2:0/66784043 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2c50003a30 0x7f2c600ad440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:37.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.508+0000 7f2c5ffff700 1 --2- 172.21.15.2:0/66784043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c60151800 0x7f2c600c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:37.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.508+0000 7f2c654de700 1 --2- 172.21.15.2:0/66784043 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2c50003a30 0x7f2c600ad440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.508+0000 7f2c654de700 1 -- 172.21.15.2:0/66784043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c60151800 msgr2=0x7f2c600c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:37.526 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.508+0000 7f2c654de700 1 --2- 172.21.15.2:0/66784043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c60151800 0x7f2c600c9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.508+0000 7f2c654de700 1 -- 172.21.15.2:0/66784043 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c600b1fb0 con 0x7f2c600afd10 2023-08-25T03:35:37.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.508+0000 7f2c654de700 1 --2- 172.21.15.2:0/66784043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c600afd10 0x7f2c600b01c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2c600cebe0 tx=0x7f2c5800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:37.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.508+0000 7f2c5d7fa700 1 -- 172.21.15.2:0/66784043 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c58003c10 con 0x7f2c600afd10 2023-08-25T03:35:37.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.508+0000 7f2c5d7fa700 1 -- 172.21.15.2:0/66784043 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c58003d90 con 0x7f2c600afd10 2023-08-25T03:35:37.527 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.508+0000 7f2c66f41700 1 -- 172.21.15.2:0/66784043 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c600ca340 con 0x7f2c600afd10 2023-08-25T03:35:37.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.508+0000 7f2c5d7fa700 1 -- 172.21.15.2:0/66784043 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c58005310 con 0x7f2c600afd10 2023-08-25T03:35:37.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.508+0000 7f2c66f41700 1 -- 172.21.15.2:0/66784043 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c600ca900 con 0x7f2c600afd10 2023-08-25T03:35:37.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.509+0000 7f2c66f41700 1 -- 172.21.15.2:0/66784043 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2c600cde50 con 0x7f2c600afd10 2023-08-25T03:35:37.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.509+0000 7f2c5d7fa700 1 -- 172.21.15.2:0/66784043 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f2c58005470 con 0x7f2c600afd10 2023-08-25T03:35:37.528 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.510+0000 7f2c5d7fa700 1 --2- 172.21.15.2:0/66784043 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2c4406a0e0 0x7f2c4406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:37.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.510+0000 7f2c5d7fa700 1 -- 172.21.15.2:0/66784043 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f2c58088270 con 0x7f2c600afd10 2023-08-25T03:35:37.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.510+0000 7f2c64cdd700 1 --2- 172.21.15.2:0/66784043 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2c4406a0e0 0x7f2c4406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:37.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.510+0000 7f2c5d7fa700 1 --2- 172.21.15.2:0/66784043 >> [v2:172.21.15.31:6824/4389955,v1:172.21.15.31:6825/4389955] conn(0x7f2c44070560 0x7f2c44072980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:37.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.510+0000 7f2c5d7fa700 1 -- 172.21.15.2:0/66784043 --> [v2:172.21.15.31:6824/4389955,v1:172.21.15.31:6825/4389955] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c44073050 con 0x7f2c44070560 2023-08-25T03:35:37.529 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.510+0000 7f2c5d7fa700 1 -- 172.21.15.2:0/66784043 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c5808a910 con 0x7f2c600afd10 2023-08-25T03:35:37.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.510+0000 7f2c5ffff700 1 --2- 172.21.15.2:0/66784043 >> [v2:172.21.15.31:6824/4389955,v1:172.21.15.31:6825/4389955] conn(0x7f2c44070560 0x7f2c44072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:37.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.510+0000 7f2c64cdd700 1 --2- 172.21.15.2:0/66784043 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2c4406a0e0 0x7f2c4406c5a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2c54000c00 tx=0x7f2c54008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:37.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.511+0000 7f2c5ffff700 1 --2- 172.21.15.2:0/66784043 >> [v2:172.21.15.31:6824/4389955,v1:172.21.15.31:6825/4389955] conn(0x7f2c44070560 0x7f2c44072980 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:37.530 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.511+0000 7f2c5d7fa700 1 -- 172.21.15.2:0/66784043 <== osd.4 v2:172.21.15.31:6824/4389955 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f2c44073050 con 0x7f2c44070560 2023-08-25T03:35:37.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.526+0000 7f2c66f41700 1 -- 172.21.15.2:0/66784043 --> [v2:172.21.15.31:6824/4389955,v1:172.21.15.31:6825/4389955] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2c600ce4a0 con 0x7f2c44070560 2023-08-25T03:35:37.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.526+0000 7f2c5d7fa700 1 -- 172.21.15.2:0/66784043 <== osd.4 v2:172.21.15.31:6824/4389955 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f2c4c001540 con 0x7f2c44070560 2023-08-25T03:35:37.532 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-08-25T03:35:37.532 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-08-25T03:35:37.532 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-08-25T03:35:37.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.527+0000 7f2c66f41700 1 -- 172.21.15.2:0/66784043 >> [v2:172.21.15.31:6824/4389955,v1:172.21.15.31:6825/4389955] conn(0x7f2c44070560 msgr2=0x7f2c44072980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:37.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.527+0000 7f2c66f41700 1 --2- 172.21.15.2:0/66784043 >> [v2:172.21.15.31:6824/4389955,v1:172.21.15.31:6825/4389955] conn(0x7f2c44070560 0x7f2c44072980 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.527+0000 7f2c66f41700 1 -- 172.21.15.2:0/66784043 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2c4406a0e0 msgr2=0x7f2c4406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:37.534 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.527+0000 7f2c66f41700 1 --2- 172.21.15.2:0/66784043 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f2c4406a0e0 0x7f2c4406c5a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2c54000c00 tx=0x7f2c54008030 comp rx=0 tx=0).stop 2023-08-25T03:35:37.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.527+0000 7f2c66f41700 1 -- 172.21.15.2:0/66784043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c600afd10 msgr2=0x7f2c600b01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:37.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.527+0000 7f2c66f41700 1 --2- 172.21.15.2:0/66784043 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f2c600afd10 0x7f2c600b01c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2c600cebe0 tx=0x7f2c5800ed50 comp rx=0 tx=0).stop 2023-08-25T03:35:37.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.527+0000 7f2c654de700 1 -- 172.21.15.2:0/66784043 reap_dead start 2023-08-25T03:35:37.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.527+0000 7f2c66f41700 1 -- 172.21.15.2:0/66784043 shutdown_connections 2023-08-25T03:35:37.535 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.527+0000 7f2c66f41700 1 -- 172.21.15.2:0/66784043 >> 172.21.15.2:0/66784043 conn(0x7f2c60065fe0 msgr2=0x7f2c6015c680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:37.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.527+0000 7f2c66f41700 1 -- 172.21.15.2:0/66784043 shutdown_connections 2023-08-25T03:35:37.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.527+0000 7f2c66f41700 1 -- 172.21.15.2:0/66784043 wait complete. 2023-08-25T03:35:37.539 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-08-25T03:35:37.666 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.664+0000 7ff3ba8cb700 1 Processor -- start 2023-08-25T03:35:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.664+0000 7ff3ba8cb700 1 -- start start 2023-08-25T03:35:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.665+0000 7ff3ba8cb700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff3b415a410 0x7ff3b40ce170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.665+0000 7ff3ba8cb700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b4071dc0 con 0x7ff3b40ce6b0 2023-08-25T03:35:37.667 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.665+0000 7ff3ba8cb700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b40720a0 con 0x7ff3b415a030 2023-08-25T03:35:37.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.665+0000 7ff3ba8cb700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b40ceb00 con 0x7ff3b415a410 2023-08-25T03:35:37.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.665+0000 7ff3b8e68700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7ff3b40ce6b0 0x7ff3b40ccd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48880/0 (socket says 172.21.15.2:48880) 2023-08-25T03:35:37.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.665+0000 7ff3b8e68700 1 -- 172.21.15.2:0/2919460702 learned_addr learned my addr 172.21.15.2:0/2919460702 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:37.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.665+0000 7ff3b37fe700 1 --2- 172.21.15.2:0/2919460702 >> v2:172.21.15.31:3300/0 conn(0x7ff3b415a410 0x7ff3b40ce170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:37.668 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.666+0000 7ff3b2ffd700 1 -- 172.21.15.2:0/2919460702 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1891220194 0 0) 0x7ff3b4071dc0 con 0x7ff3b40ce6b0 2023-08-25T03:35:37.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.666+0000 7ff3b2ffd700 1 -- 172.21.15.2:0/2919460702 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3a00035b0 con 0x7ff3b40ce6b0 2023-08-25T03:35:37.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.666+0000 7ff3b2ffd700 1 -- 172.21.15.2:0/2919460702 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3056190892 0 0) 0x7ff3b40720a0 con 0x7ff3b415a030 2023-08-25T03:35:37.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.666+0000 7ff3b2ffd700 1 -- 172.21.15.2:0/2919460702 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3b4071dc0 con 0x7ff3b415a030 2023-08-25T03:35:37.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.666+0000 7ff3b37fe700 1 -- 172.21.15.2:0/2919460702 >> v1:172.21.15.31:6789/0 conn(0x7ff3b415a030 legacy=0x7ff3b40cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:37.669 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.666+0000 7ff3b37fe700 1 -- 172.21.15.2:0/2919460702 >> v1:172.21.15.2:6789/0 conn(0x7ff3b40ce6b0 legacy=0x7ff3b40ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:37.670 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.666+0000 7ff3b37fe700 1 -- 172.21.15.2:0/2919460702 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b40cb4b0 con 0x7ff3b415a410 2023-08-25T03:35:37.672 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.666+0000 7ff3b37fe700 1 --2- 172.21.15.2:0/2919460702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b415a410 0x7ff3b40ce170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009840 tx=0x7ff3a4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f55b5590a97001 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:37.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.666+0000 7ff3b2ffd700 1 -- 172.21.15.2:0/2919460702 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4011830 con 0x7ff3b415a410 2023-08-25T03:35:37.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.666+0000 7ff3b2ffd700 1 -- 172.21.15.2:0/2919460702 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a4011990 con 0x7ff3b415a410 2023-08-25T03:35:37.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.666+0000 7ff3b2ffd700 1 -- 172.21.15.2:0/2919460702 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4011d20 con 0x7ff3b415a410 2023-08-25T03:35:37.673 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.667+0000 7ff3ba8cb700 1 -- 172.21.15.2:0/2919460702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b415a410 msgr2=0x7ff3b40ce170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:37.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.667+0000 7ff3ba8cb700 1 --2- 172.21.15.2:0/2919460702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b415a410 0x7ff3b40ce170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009840 tx=0x7ff3a4009b50 comp rx=0 tx=0).stop 2023-08-25T03:35:37.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.667+0000 7ff3ba8cb700 1 -- 172.21.15.2:0/2919460702 shutdown_connections 2023-08-25T03:35:37.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.667+0000 7ff3ba8cb700 1 --2- 172.21.15.2:0/2919460702 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b415a410 0x7ff3b40ce170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.667+0000 7ff3ba8cb700 1 -- 172.21.15.2:0/2919460702 >> 172.21.15.2:0/2919460702 conn(0x7ff3b40b1cf0 msgr2=0x7ff3b40b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:37.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.667+0000 7ff3ba8cb700 1 -- 172.21.15.2:0/2919460702 shutdown_connections 2023-08-25T03:35:37.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.667+0000 7ff3ba8cb700 1 -- 172.21.15.2:0/2919460702 wait complete. 2023-08-25T03:35:37.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.668+0000 7ff3ba8cb700 1 Processor -- start 2023-08-25T03:35:37.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.668+0000 7ff3ba8cb700 1 -- start start 2023-08-25T03:35:37.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.669+0000 7ff3ba8cb700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b40ce6b0 0x7ff3b4156710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:37.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.669+0000 7ff3ba8cb700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3b415a030 0x7ff3b4155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:37.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.669+0000 7ff3ba8cb700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff3b415a410 0x7ff3b4158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:37.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.669+0000 7ff3ba8cb700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b4156cf0 con 0x7ff3b415a030 2023-08-25T03:35:37.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.669+0000 7ff3ba8cb700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b4156530 con 0x7ff3b40ce6b0 2023-08-25T03:35:37.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.669+0000 7ff3ba8cb700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b41589a0 con 0x7ff3b415a410 2023-08-25T03:35:37.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.669+0000 7ff3b37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3b415a030 0x7ff3b4155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:37.676 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.669+0000 7ff3b37fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3b415a030 0x7ff3b4155ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:33986/0 (socket says 172.21.15.2:33986) 2023-08-25T03:35:37.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.669+0000 7ff3b37fe700 1 -- 172.21.15.2:0/2313897755 learned_addr learned my addr 172.21.15.2:0/2313897755 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:37.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.669+0000 7ff3b3fff700 1 --2- 172.21.15.2:0/2313897755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b40ce6b0 0x7ff3b4156710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:37.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.669+0000 7ff3b8e68700 1 --2- 172.21.15.2:0/2313897755 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff3b415a410 0x7ff3b4158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:37.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.670+0000 7ff3b37fe700 1 -- 172.21.15.2:0/2313897755 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff3b415a410 msgr2=0x7ff3b4158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:37.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.670+0000 7ff3b37fe700 1 --2- 172.21.15.2:0/2313897755 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff3b415a410 0x7ff3b4158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.670+0000 7ff3b37fe700 1 -- 172.21.15.2:0/2313897755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b40ce6b0 msgr2=0x7ff3b4156710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:37.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.670+0000 7ff3b37fe700 1 --2- 172.21.15.2:0/2313897755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b40ce6b0 0x7ff3b4156710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.670+0000 7ff3b37fe700 1 -- 172.21.15.2:0/2313897755 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a40094a0 con 0x7ff3b415a030 2023-08-25T03:35:37.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.670+0000 7ff3b8e68700 1 --2- 172.21.15.2:0/2313897755 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff3b415a410 0x7ff3b4158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:35:37.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.670+0000 7ff3b3fff700 1 --2- 172.21.15.2:0/2313897755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b40ce6b0 0x7ff3b4156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:37.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.670+0000 7ff3b37fe700 1 --2- 172.21.15.2:0/2313897755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3b415a030 0x7ff3b4155ff0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009fd0 tx=0x7ff3a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:37.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.670+0000 7ff3b0ff9700 1 -- 172.21.15.2:0/2313897755 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a401b870 con 0x7ff3b415a030 2023-08-25T03:35:37.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.670+0000 7ff3b0ff9700 1 -- 172.21.15.2:0/2313897755 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a401b9d0 con 0x7ff3b415a030 2023-08-25T03:35:37.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.670+0000 7ff3ba8cb700 1 -- 172.21.15.2:0/2313897755 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b40c3c10 con 0x7ff3b415a030 2023-08-25T03:35:37.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.670+0000 7ff3b0ff9700 1 -- 172.21.15.2:0/2313897755 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a401bd80 con 0x7ff3b415a030 2023-08-25T03:35:37.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.670+0000 7ff3ba8cb700 1 -- 172.21.15.2:0/2313897755 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b40c41d0 con 0x7ff3b415a030 2023-08-25T03:35:37.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.671+0000 7ff3ba8cb700 1 -- 172.21.15.2:0/2313897755 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b4066f70 con 0x7ff3b415a030 2023-08-25T03:35:37.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.671+0000 7ff3b0ff9700 1 -- 172.21.15.2:0/2313897755 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff3a4023070 con 0x7ff3b415a030 2023-08-25T03:35:37.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.672+0000 7ff3b0ff9700 1 --2- 172.21.15.2:0/2313897755 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff39406a110 0x7ff39406c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:37.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.672+0000 7ff3b0ff9700 1 -- 172.21.15.2:0/2313897755 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff3a4089380 con 0x7ff3b415a030 2023-08-25T03:35:37.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.672+0000 7ff3b3fff700 1 --2- 172.21.15.2:0/2313897755 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff39406a110 0x7ff39406c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:37.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.672+0000 7ff3b3fff700 1 --2- 172.21.15.2:0/2313897755 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff39406a110 0x7ff39406c5d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff39c0011e0 tx=0x7ff39c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:37.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.676+0000 7ff3b0ff9700 1 -- 172.21.15.2:0/2313897755 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7ff3a4054ea0 con 0x7ff3b415a030 2023-08-25T03:35:37.874 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.874+0000 7ff3ba8cb700 1 -- 172.21.15.2:0/2313897755 --> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7ff3b40cc3c0 con 0x7ff39406a110 2023-08-25T03:35:37.875 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.874+0000 7ff3b0ff9700 1 -- 172.21.15.2:0/2313897755 <== mgr.4108 v2:172.21.15.2:6832/3528779779 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff39c001cf0 con 0x7ff39406a110 2023-08-25T03:35:37.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.876+0000 7ff3ba8cb700 1 -- 172.21.15.2:0/2313897755 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff39406a110 msgr2=0x7ff39406c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:37.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.876+0000 7ff3ba8cb700 1 --2- 172.21.15.2:0/2313897755 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff39406a110 0x7ff39406c5d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ff39c0011e0 tx=0x7ff39c009040 comp rx=0 tx=0).stop 2023-08-25T03:35:37.877 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.876+0000 7ff3ba8cb700 1 -- 172.21.15.2:0/2313897755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3b415a030 msgr2=0x7ff3b4155ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:37.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.876+0000 7ff3ba8cb700 1 --2- 172.21.15.2:0/2313897755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3b415a030 0x7ff3b4155ff0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009fd0 tx=0x7ff3a4004750 comp rx=0 tx=0).stop 2023-08-25T03:35:37.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.876+0000 7ff3ba8cb700 1 -- 172.21.15.2:0/2313897755 shutdown_connections 2023-08-25T03:35:37.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.876+0000 7ff3ba8cb700 1 --2- 172.21.15.2:0/2313897755 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff3b415a410 0x7ff3b4158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.876+0000 7ff3ba8cb700 1 --2- 172.21.15.2:0/2313897755 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff3b40ce6b0 0x7ff3b4156710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.878 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.876+0000 7ff3ba8cb700 1 --2- 172.21.15.2:0/2313897755 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7ff39406a110 0x7ff39406c5d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.876+0000 7ff3ba8cb700 1 --2- 172.21.15.2:0/2313897755 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff3b415a030 0x7ff3b4155ff0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:37.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.876+0000 7ff3ba8cb700 1 -- 172.21.15.2:0/2313897755 >> 172.21.15.2:0/2313897755 conn(0x7ff3b40b1cf0 msgr2=0x7ff3b406d420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:37.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.876+0000 7ff3ba8cb700 1 -- 172.21.15.2:0/2313897755 shutdown_connections 2023-08-25T03:35:37.879 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:37.876+0000 7ff3ba8cb700 1 -- 172.21.15.2:0/2313897755 wait complete. 2023-08-25T03:35:37.880 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 4 to deep-scrub 2023-08-25T03:35:37.889 INFO:tasks.ceph:Scrubbing osd.5 2023-08-25T03:35:37.889 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-08-25T03:35:38.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.017+0000 7fd023d87700 1 Processor -- start 2023-08-25T03:35:38.018 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.017+0000 7fd023d87700 1 -- start start 2023-08-25T03:35:38.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.018+0000 7fd023d87700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fd01c0716a0 0x7fd01c06e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:38.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.018+0000 7fd023d87700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd01c15a490 con 0x7fd01c15dbb0 2023-08-25T03:35:38.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.018+0000 7fd023d87700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd01c06e6c0 con 0x7fd01c15a0b0 2023-08-25T03:35:38.020 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.018+0000 7fd023d87700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd01c15e120 con 0x7fd01c0716a0 2023-08-25T03:35:38.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.018+0000 7fd022324700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fd01c15dbb0 0x7fd01c0ca9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48894/0 (socket says 172.21.15.2:48894) 2023-08-25T03:35:38.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.018+0000 7fd022324700 1 -- 172.21.15.2:0/1408002994 learned_addr learned my addr 172.21.15.2:0/1408002994 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:38.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.018+0000 7fd021322700 1 --2- 172.21.15.2:0/1408002994 >> v2:172.21.15.31:3300/0 conn(0x7fd01c0716a0 0x7fd01c06e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:38.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.019+0000 7fd020b21700 1 -- 172.21.15.2:0/1408002994 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 342886005 0 0) 0x7fd014001490 con 0x7fd01c15dbb0 2023-08-25T03:35:38.021 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.019+0000 7fd020b21700 1 -- 172.21.15.2:0/1408002994 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0080035b0 con 0x7fd01c15dbb0 2023-08-25T03:35:38.022 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.019+0000 7fd020b21700 1 -- 172.21.15.2:0/1408002994 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1980589221 0 0) 0x7fd01c06e6c0 con 0x7fd01c15a0b0 2023-08-25T03:35:38.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.019+0000 7fd020b21700 1 -- 172.21.15.2:0/1408002994 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd014001490 con 0x7fd01c15a0b0 2023-08-25T03:35:38.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.019+0000 7fd021322700 1 -- 172.21.15.2:0/1408002994 >> v1:172.21.15.31:6789/0 conn(0x7fd01c15a0b0 legacy=0x7fd01c070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.023 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.019+0000 7fd020b21700 1 -- 172.21.15.2:0/1408002994 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 386405599 0 0) 0x7fd0080035b0 con 0x7fd01c15dbb0 2023-08-25T03:35:38.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.019+0000 7fd021322700 1 -- 172.21.15.2:0/1408002994 >> v1:172.21.15.2:6789/0 conn(0x7fd01c15dbb0 legacy=0x7fd01c0ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.019+0000 7fd021322700 1 -- 172.21.15.2:0/1408002994 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd01c15e400 con 0x7fd01c0716a0 2023-08-25T03:35:38.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.019+0000 7fd021322700 1 --2- 172.21.15.2:0/1408002994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd01c0716a0 0x7fd01c06e180 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd018009840 tx=0x7fd018009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52d9f320fa6b861f server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:38.024 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.019+0000 7fd020b21700 1 -- 172.21.15.2:0/1408002994 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd014001b80 con 0x7fd01c0716a0 2023-08-25T03:35:38.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.019+0000 7fd020b21700 1 -- 172.21.15.2:0/1408002994 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd018004030 con 0x7fd01c0716a0 2023-08-25T03:35:38.025 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.019+0000 7fd020b21700 1 -- 172.21.15.2:0/1408002994 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0180043c0 con 0x7fd01c0716a0 2023-08-25T03:35:38.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.020+0000 7fd023d87700 1 -- 172.21.15.2:0/1408002994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd01c0716a0 msgr2=0x7fd01c06e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.020+0000 7fd023d87700 1 --2- 172.21.15.2:0/1408002994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd01c0716a0 0x7fd01c06e180 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd018009840 tx=0x7fd018009b50 comp rx=0 tx=0).stop 2023-08-25T03:35:38.028 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.020+0000 7fd023d87700 1 -- 172.21.15.2:0/1408002994 shutdown_connections 2023-08-25T03:35:38.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.020+0000 7fd023d87700 1 --2- 172.21.15.2:0/1408002994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd01c0716a0 0x7fd01c06e180 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.020+0000 7fd023d87700 1 -- 172.21.15.2:0/1408002994 >> 172.21.15.2:0/1408002994 conn(0x7fd01c0b1990 msgr2=0x7fd01c0b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:38.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.020+0000 7fd023d87700 1 -- 172.21.15.2:0/1408002994 shutdown_connections 2023-08-25T03:35:38.029 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.020+0000 7fd023d87700 1 -- 172.21.15.2:0/1408002994 wait complete. 2023-08-25T03:35:38.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.021+0000 7fd023d87700 1 Processor -- start 2023-08-25T03:35:38.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.021+0000 7fd023d87700 1 -- start start 2023-08-25T03:35:38.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.022+0000 7fd023d87700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd01c0716a0 0x7fd01c14d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:38.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.022+0000 7fd023d87700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd01c15a0b0 0x7fd01c0b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:38.030 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.022+0000 7fd023d87700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd01c15dbb0 0x7fd01c15ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:38.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.022+0000 7fd023d87700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c0b5820 con 0x7fd01c15dbb0 2023-08-25T03:35:38.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.022+0000 7fd023d87700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c0c7410 con 0x7fd01c0716a0 2023-08-25T03:35:38.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.022+0000 7fd023d87700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c14da70 con 0x7fd01c15a0b0 2023-08-25T03:35:38.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.022+0000 7fd022324700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd01c15dbb0 0x7fd01c15ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:38.031 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.022+0000 7fd022324700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd01c15dbb0 0x7fd01c15ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34000/0 (socket says 172.21.15.2:34000) 2023-08-25T03:35:38.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.022+0000 7fd022324700 1 -- 172.21.15.2:0/2544423775 learned_addr learned my addr 172.21.15.2:0/2544423775 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:38.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.022+0000 7fd021b23700 1 --2- 172.21.15.2:0/2544423775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd01c0716a0 0x7fd01c14d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:38.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.022+0000 7fd021322700 1 --2- 172.21.15.2:0/2544423775 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd01c15a0b0 0x7fd01c0b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:38.032 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.022+0000 7fd022324700 1 -- 172.21.15.2:0/2544423775 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd01c15a0b0 msgr2=0x7fd01c0b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.022+0000 7fd022324700 1 --2- 172.21.15.2:0/2544423775 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd01c15a0b0 0x7fd01c0b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.022+0000 7fd022324700 1 -- 172.21.15.2:0/2544423775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd01c0716a0 msgr2=0x7fd01c14d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.022+0000 7fd022324700 1 --2- 172.21.15.2:0/2544423775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd01c0716a0 0x7fd01c14d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.022+0000 7fd022324700 1 -- 172.21.15.2:0/2544423775 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0180094a0 con 0x7fd01c15dbb0 2023-08-25T03:35:38.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.022+0000 7fd021322700 1 --2- 172.21.15.2:0/2544423775 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd01c15a0b0 0x7fd01c0b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:38.033 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.022+0000 7fd022324700 1 --2- 172.21.15.2:0/2544423775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd01c15dbb0 0x7fd01c15ccd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd01400c270 tx=0x7fd01400c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:38.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.023+0000 7fd0127fc700 1 -- 172.21.15.2:0/2544423775 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0140070d0 con 0x7fd01c15dbb0 2023-08-25T03:35:38.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.023+0000 7fd0127fc700 1 -- 172.21.15.2:0/2544423775 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd014007230 con 0x7fd01c15dbb0 2023-08-25T03:35:38.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.023+0000 7fd023d87700 1 -- 172.21.15.2:0/2544423775 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd01c0c9260 con 0x7fd01c15dbb0 2023-08-25T03:35:38.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.023+0000 7fd0127fc700 1 -- 172.21.15.2:0/2544423775 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0140075e0 con 0x7fd01c15dbb0 2023-08-25T03:35:38.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.023+0000 7fd023d87700 1 -- 172.21.15.2:0/2544423775 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd01c0c9820 con 0x7fd01c15dbb0 2023-08-25T03:35:38.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.023+0000 7fd023d87700 1 -- 172.21.15.2:0/2544423775 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd01c15e9c0 con 0x7fd01c15dbb0 2023-08-25T03:35:38.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.024+0000 7fd0127fc700 1 -- 172.21.15.2:0/2544423775 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fd014007740 con 0x7fd01c15dbb0 2023-08-25T03:35:38.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.024+0000 7fd0127fc700 1 --2- 172.21.15.2:0/2544423775 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd00006a0e0 0x7fd00006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:38.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.025+0000 7fd0127fc700 1 -- 172.21.15.2:0/2544423775 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fd01408b8f0 con 0x7fd01c15dbb0 2023-08-25T03:35:38.035 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.025+0000 7fd021b23700 1 --2- 172.21.15.2:0/2544423775 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd00006a0e0 0x7fd00006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:38.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.025+0000 7fd0127fc700 1 --2- 172.21.15.2:0/2544423775 >> [v2:172.21.15.31:6800/1678889094,v1:172.21.15.31:6801/1678889094] conn(0x7fd000070560 0x7fd000072980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:38.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.025+0000 7fd0127fc700 1 -- 172.21.15.2:0/2544423775 --> [v2:172.21.15.31:6800/1678889094,v1:172.21.15.31:6801/1678889094] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd000073050 con 0x7fd000070560 2023-08-25T03:35:38.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.025+0000 7fd0127fc700 1 -- 172.21.15.2:0/2544423775 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd01408bd20 con 0x7fd01c15dbb0 2023-08-25T03:35:38.036 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.025+0000 7fd021b23700 1 --2- 172.21.15.2:0/2544423775 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd00006a0e0 0x7fd00006c5a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0011e0 tx=0x7fd00c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:38.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.025+0000 7fd021322700 1 --2- 172.21.15.2:0/2544423775 >> [v2:172.21.15.31:6800/1678889094,v1:172.21.15.31:6801/1678889094] conn(0x7fd000070560 0x7fd000072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:38.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.025+0000 7fd021322700 1 --2- 172.21.15.2:0/2544423775 >> [v2:172.21.15.31:6800/1678889094,v1:172.21.15.31:6801/1678889094] conn(0x7fd000070560 0x7fd000072980 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:38.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.026+0000 7fd0127fc700 1 -- 172.21.15.2:0/2544423775 <== osd.5 v2:172.21.15.31:6800/1678889094 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fd000073050 con 0x7fd000070560 2023-08-25T03:35:38.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.041+0000 7fd023d87700 1 -- 172.21.15.2:0/2544423775 --> [v2:172.21.15.31:6800/1678889094,v1:172.21.15.31:6801/1678889094] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd01c15e6e0 con 0x7fd000070560 2023-08-25T03:35:38.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.041+0000 7fd0127fc700 1 -- 172.21.15.2:0/2544423775 <== osd.5 v2:172.21.15.31:6800/1678889094 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fd018021040 con 0x7fd000070560 2023-08-25T03:35:38.043 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-08-25T03:35:38.043 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-08-25T03:35:38.043 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-08-25T03:35:38.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.042+0000 7fd023d87700 1 -- 172.21.15.2:0/2544423775 >> [v2:172.21.15.31:6800/1678889094,v1:172.21.15.31:6801/1678889094] conn(0x7fd000070560 msgr2=0x7fd000072980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.042+0000 7fd023d87700 1 --2- 172.21.15.2:0/2544423775 >> [v2:172.21.15.31:6800/1678889094,v1:172.21.15.31:6801/1678889094] conn(0x7fd000070560 0x7fd000072980 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.042+0000 7fd023d87700 1 -- 172.21.15.2:0/2544423775 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd00006a0e0 msgr2=0x7fd00006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.042+0000 7fd023d87700 1 --2- 172.21.15.2:0/2544423775 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fd00006a0e0 0x7fd00006c5a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0011e0 tx=0x7fd00c009040 comp rx=0 tx=0).stop 2023-08-25T03:35:38.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.042+0000 7fd023d87700 1 -- 172.21.15.2:0/2544423775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd01c15dbb0 msgr2=0x7fd01c15ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.042+0000 7fd023d87700 1 --2- 172.21.15.2:0/2544423775 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fd01c15dbb0 0x7fd01c15ccd0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fd01400c270 tx=0x7fd01400c580 comp rx=0 tx=0).stop 2023-08-25T03:35:38.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.042+0000 7fd022324700 1 -- 172.21.15.2:0/2544423775 reap_dead start 2023-08-25T03:35:38.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.042+0000 7fd023d87700 1 -- 172.21.15.2:0/2544423775 shutdown_connections 2023-08-25T03:35:38.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.042+0000 7fd023d87700 1 -- 172.21.15.2:0/2544423775 >> 172.21.15.2:0/2544423775 conn(0x7fd01c0b1990 msgr2=0x7fd01c0b1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:38.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.042+0000 7fd023d87700 1 -- 172.21.15.2:0/2544423775 shutdown_connections 2023-08-25T03:35:38.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.042+0000 7fd023d87700 1 -- 172.21.15.2:0/2544423775 wait complete. 2023-08-25T03:35:38.055 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-08-25T03:35:38.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.180+0000 7fe64b76e700 1 Processor -- start 2023-08-25T03:35:38.181 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.180+0000 7fe64b76e700 1 -- start start 2023-08-25T03:35:38.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.181+0000 7fe64b76e700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe644163030 0x7fe644165420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:38.186 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.181+0000 7fe64b76e700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe644071b40 con 0x7fe64415e890 2023-08-25T03:35:38.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.181+0000 7fe64b76e700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6440720b0 con 0x7fe6440b19a0 2023-08-25T03:35:38.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.181+0000 7fe64b76e700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe644071dd0 con 0x7fe644163030 2023-08-25T03:35:38.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.181+0000 7fe648d09700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7fe64415e890 0x7fe644071430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48906/0 (socket says 172.21.15.2:48906) 2023-08-25T03:35:38.187 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.181+0000 7fe648d09700 1 -- 172.21.15.2:0/3868580083 learned_addr learned my addr 172.21.15.2:0/3868580083 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:38.189 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.181+0000 7fe649d0b700 1 --2- 172.21.15.2:0/3868580083 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe644163030 0x7fe644165420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:38.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.181+0000 7fe63bfff700 1 -- 172.21.15.2:0/3868580083 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1292416130 0 0) 0x7fe644071b40 con 0x7fe64415e890 2023-08-25T03:35:38.190 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.181+0000 7fe63bfff700 1 -- 172.21.15.2:0/3868580083 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6300035b0 con 0x7fe64415e890 2023-08-25T03:35:38.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.181+0000 7fe63bfff700 1 -- 172.21.15.2:0/3868580083 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4291502432 0 0) 0x7fe6440720b0 con 0x7fe6440b19a0 2023-08-25T03:35:38.191 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.182+0000 7fe63bfff700 1 -- 172.21.15.2:0/3868580083 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe644071b40 con 0x7fe6440b19a0 2023-08-25T03:35:38.192 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.182+0000 7fe63bfff700 1 -- 172.21.15.2:0/3868580083 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 860652080 0 0) 0x7fe6300035b0 con 0x7fe64415e890 2023-08-25T03:35:38.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.182+0000 7fe649d0b700 1 -- 172.21.15.2:0/3868580083 >> v1:172.21.15.31:6789/0 conn(0x7fe6440b19a0 legacy=0x7fe6440b1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.193 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.182+0000 7fe649d0b700 1 -- 172.21.15.2:0/3868580083 >> v1:172.21.15.2:6789/0 conn(0x7fe64415e890 legacy=0x7fe644071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.182+0000 7fe649d0b700 1 -- 172.21.15.2:0/3868580083 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe64415ddb0 con 0x7fe644163030 2023-08-25T03:35:38.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.182+0000 7fe649d0b700 1 --2- 172.21.15.2:0/3868580083 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe644163030 0x7fe644165420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe63c009dc0 tx=0x7fe63c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29b3c13909d6b38d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:38.194 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.182+0000 7fe63bfff700 1 -- 172.21.15.2:0/3868580083 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63c00e070 con 0x7fe644163030 2023-08-25T03:35:38.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.182+0000 7fe63bfff700 1 -- 172.21.15.2:0/3868580083 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe63c003a20 con 0x7fe644163030 2023-08-25T03:35:38.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.182+0000 7fe63bfff700 1 -- 172.21.15.2:0/3868580083 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe640001670 con 0x7fe644163030 2023-08-25T03:35:38.195 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.182+0000 7fe64b76e700 1 -- 172.21.15.2:0/3868580083 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe644163030 msgr2=0x7fe644165420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.183+0000 7fe64b76e700 1 --2- 172.21.15.2:0/3868580083 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe644163030 0x7fe644165420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe63c009dc0 tx=0x7fe63c00b330 comp rx=0 tx=0).stop 2023-08-25T03:35:38.196 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.183+0000 7fe64b76e700 1 -- 172.21.15.2:0/3868580083 shutdown_connections 2023-08-25T03:35:38.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.183+0000 7fe64b76e700 1 --2- 172.21.15.2:0/3868580083 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe644163030 0x7fe644165420 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.183+0000 7fe64b76e700 1 -- 172.21.15.2:0/3868580083 >> 172.21.15.2:0/3868580083 conn(0x7fe6440cfea0 msgr2=0x7fe6440d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:38.197 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.183+0000 7fe64b76e700 1 -- 172.21.15.2:0/3868580083 shutdown_connections 2023-08-25T03:35:38.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.183+0000 7fe64b76e700 1 -- 172.21.15.2:0/3868580083 wait complete. 2023-08-25T03:35:38.198 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.184+0000 7fe64b76e700 1 Processor -- start 2023-08-25T03:35:38.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.184+0000 7fe64b76e700 1 -- start start 2023-08-25T03:35:38.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.185+0000 7fe64b76e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6440b19a0 0x7fe6441595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:38.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.185+0000 7fe64b76e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe64415e890 0x7fe6441530e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:38.199 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.185+0000 7fe64b76e700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe644163030 0x7fe644153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:38.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.185+0000 7fe64b76e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6440c6ee0 con 0x7fe6440b19a0 2023-08-25T03:35:38.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.185+0000 7fe64b76e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe64414f1c0 con 0x7fe64415e890 2023-08-25T03:35:38.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.185+0000 7fe64b76e700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe64415c6a0 con 0x7fe644163030 2023-08-25T03:35:38.200 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.185+0000 7fe64950a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6440b19a0 0x7fe6441595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:38.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.185+0000 7fe64950a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6440b19a0 0x7fe6441595c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34014/0 (socket says 172.21.15.2:34014) 2023-08-25T03:35:38.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.185+0000 7fe64950a700 1 -- 172.21.15.2:0/851651986 learned_addr learned my addr 172.21.15.2:0/851651986 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:38.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.185+0000 7fe648d09700 1 --2- 172.21.15.2:0/851651986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe64415e890 0x7fe6441530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:38.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.185+0000 7fe64950a700 1 -- 172.21.15.2:0/851651986 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe644163030 msgr2=0x7fe644153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.201 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.185+0000 7fe64950a700 1 --2- 172.21.15.2:0/851651986 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe644163030 0x7fe644153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.185+0000 7fe64950a700 1 -- 172.21.15.2:0/851651986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe64415e890 msgr2=0x7fe6441530e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.185+0000 7fe64950a700 1 --2- 172.21.15.2:0/851651986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe64415e890 0x7fe6441530e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.185+0000 7fe64950a700 1 -- 172.21.15.2:0/851651986 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe63c009a70 con 0x7fe6440b19a0 2023-08-25T03:35:38.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.185+0000 7fe64950a700 1 --2- 172.21.15.2:0/851651986 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6440b19a0 0x7fe6441595c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe63400c520 tx=0x7fe63400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:38.202 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.185+0000 7fe639ffb700 1 -- 172.21.15.2:0/851651986 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6340052d0 con 0x7fe6440b19a0 2023-08-25T03:35:38.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.185+0000 7fe639ffb700 1 -- 172.21.15.2:0/851651986 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe634005430 con 0x7fe6440b19a0 2023-08-25T03:35:38.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.185+0000 7fe64b76e700 1 -- 172.21.15.2:0/851651986 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe64415c980 con 0x7fe6440b19a0 2023-08-25T03:35:38.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.185+0000 7fe639ffb700 1 -- 172.21.15.2:0/851651986 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6340038c0 con 0x7fe6440b19a0 2023-08-25T03:35:38.203 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.186+0000 7fe64b76e700 1 -- 172.21.15.2:0/851651986 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe64415cf40 con 0x7fe6440b19a0 2023-08-25T03:35:38.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.186+0000 7fe64b76e700 1 -- 172.21.15.2:0/851651986 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe644066f70 con 0x7fe6440b19a0 2023-08-25T03:35:38.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.187+0000 7fe639ffb700 1 -- 172.21.15.2:0/851651986 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fe634003a20 con 0x7fe6440b19a0 2023-08-25T03:35:38.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.187+0000 7fe639ffb700 1 --2- 172.21.15.2:0/851651986 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe62806a110 0x7fe62806c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:38.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.188+0000 7fe639ffb700 1 -- 172.21.15.2:0/851651986 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fe634087870 con 0x7fe6440b19a0 2023-08-25T03:35:38.205 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.188+0000 7fe648d09700 1 --2- 172.21.15.2:0/851651986 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe62806a110 0x7fe62806c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:38.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.188+0000 7fe648d09700 1 --2- 172.21.15.2:0/851651986 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe62806a110 0x7fe62806c5d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe64415d810 tx=0x7fe640009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:38.206 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.192+0000 7fe639ffb700 1 -- 172.21.15.2:0/851651986 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fe6340531e0 con 0x7fe6440b19a0 2023-08-25T03:35:38.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.391+0000 7fe64b76e700 1 -- 172.21.15.2:0/851651986 --> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fe64415e370 con 0x7fe62806a110 2023-08-25T03:35:38.392 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.391+0000 7fe639ffb700 1 -- 172.21.15.2:0/851651986 <== mgr.4108 v2:172.21.15.2:6832/3528779779 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe6400070d0 con 0x7fe62806a110 2023-08-25T03:35:38.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.393+0000 7fe64b76e700 1 -- 172.21.15.2:0/851651986 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe62806a110 msgr2=0x7fe62806c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.394 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.393+0000 7fe64b76e700 1 --2- 172.21.15.2:0/851651986 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe62806a110 0x7fe62806c5d0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe64415d810 tx=0x7fe640009040 comp rx=0 tx=0).stop 2023-08-25T03:35:38.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.393+0000 7fe64b76e700 1 -- 172.21.15.2:0/851651986 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6440b19a0 msgr2=0x7fe6441595c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.393+0000 7fe64b76e700 1 --2- 172.21.15.2:0/851651986 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6440b19a0 0x7fe6441595c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe63400c520 tx=0x7fe63400c8e0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.395 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.393+0000 7fe64b76e700 1 -- 172.21.15.2:0/851651986 shutdown_connections 2023-08-25T03:35:38.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.393+0000 7fe64b76e700 1 --2- 172.21.15.2:0/851651986 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fe62806a110 0x7fe62806c5d0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.393+0000 7fe64b76e700 1 --2- 172.21.15.2:0/851651986 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fe6440b19a0 0x7fe6441595c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.396 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.393+0000 7fe64b76e700 1 --2- 172.21.15.2:0/851651986 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe644163030 0x7fe644153620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.393+0000 7fe64b76e700 1 --2- 172.21.15.2:0/851651986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe64415e890 0x7fe6441530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.393+0000 7fe64b76e700 1 -- 172.21.15.2:0/851651986 >> 172.21.15.2:0/851651986 conn(0x7fe6440cfea0 msgr2=0x7fe6440b11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:38.397 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.393+0000 7fe64b76e700 1 -- 172.21.15.2:0/851651986 shutdown_connections 2023-08-25T03:35:38.398 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.393+0000 7fe64b76e700 1 -- 172.21.15.2:0/851651986 wait complete. 2023-08-25T03:35:38.398 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 5 to deep-scrub 2023-08-25T03:35:38.406 INFO:tasks.ceph:Scrubbing osd.6 2023-08-25T03:35:38.407 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-08-25T03:35:38.536 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.536+0000 7f65d3dc9700 1 Processor -- start 2023-08-25T03:35:38.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.536+0000 7f65d3dc9700 1 -- start start 2023-08-25T03:35:38.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.537+0000 7f65d3dc9700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f65cc15a0b0 0x7f65cc06e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:38.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.537+0000 7f65d3dc9700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f65cc0716a0 0x7f65cc0ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:38.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.537+0000 7f65d3dc9700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65cc15a490 con 0x7f65cc15dbb0 2023-08-25T03:35:38.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.537+0000 7f65d3dc9700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65cc06e6c0 con 0x7f65cc0716a0 2023-08-25T03:35:38.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.537+0000 7f65d3dc9700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc15e120 con 0x7f65cc15a0b0 2023-08-25T03:35:38.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.537+0000 7f65d1b65700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f65cc15a0b0 0x7f65cc06e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:38.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.537+0000 7f65d1364700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f65cc15dbb0 0x7f65cc070f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.2:56944/0 (socket says 172.21.15.2:56944) 2023-08-25T03:35:38.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.537+0000 7f65d2366700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f65cc0716a0 0x7f65cc0ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:38.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.537+0000 7f65d1364700 1 -- 172.21.15.2:0/1900165372 learned_addr learned my addr 172.21.15.2:0/1900165372 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:38.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.538+0000 7f65d1b65700 1 -- 172.21.15.2:0/1900165372 >> v2:172.21.15.31:3300/0 conn(0x7f65cc0716a0 msgr2=0x7f65cc0ca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:38.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.538+0000 7f65d1b65700 1 --2- 172.21.15.2:0/1900165372 >> v2:172.21.15.31:3300/0 conn(0x7f65cc0716a0 0x7f65cc0ca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.538+0000 7f65d1b65700 1 -- 172.21.15.2:0/1900165372 >> v1:172.21.15.31:6789/0 conn(0x7f65cc15dbb0 legacy=0x7f65cc070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.538+0000 7f65d1b65700 1 -- 172.21.15.2:0/1900165372 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65cc15e400 con 0x7f65cc15a0b0 2023-08-25T03:35:38.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.538+0000 7f65d1b65700 1 --2- 172.21.15.2:0/1900165372 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f65cc15a0b0 0x7f65cc06e180 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009840 tx=0x7f65c8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75ec8b6721b7c1fa server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:38.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.538+0000 7f65d0b63700 1 -- 172.21.15.2:0/1900165372 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c8011830 con 0x7f65cc15a0b0 2023-08-25T03:35:38.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.538+0000 7f65d0b63700 1 -- 172.21.15.2:0/1900165372 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c8011990 con 0x7f65cc15a0b0 2023-08-25T03:35:38.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.538+0000 7f65d0b63700 1 -- 172.21.15.2:0/1900165372 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c8011d20 con 0x7f65cc15a0b0 2023-08-25T03:35:38.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.539+0000 7f65d3dc9700 1 -- 172.21.15.2:0/1900165372 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f65cc15a0b0 msgr2=0x7f65cc06e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.539+0000 7f65d3dc9700 1 --2- 172.21.15.2:0/1900165372 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f65cc15a0b0 0x7f65cc06e180 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009840 tx=0x7f65c8009b50 comp rx=0 tx=0).stop 2023-08-25T03:35:38.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.539+0000 7f65d3dc9700 1 -- 172.21.15.2:0/1900165372 shutdown_connections 2023-08-25T03:35:38.543 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.539+0000 7f65d3dc9700 1 --2- 172.21.15.2:0/1900165372 >> v2:172.21.15.31:3300/0 conn(0x7f65cc0716a0 0x7f65cc0ca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.539+0000 7f65d3dc9700 1 --2- 172.21.15.2:0/1900165372 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f65cc15a0b0 0x7f65cc06e180 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.539+0000 7f65d3dc9700 1 -- 172.21.15.2:0/1900165372 >> 172.21.15.2:0/1900165372 conn(0x7f65cc0b1990 msgr2=0x7f65cc0b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:38.544 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.539+0000 7f65d3dc9700 1 -- 172.21.15.2:0/1900165372 shutdown_connections 2023-08-25T03:35:38.545 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.539+0000 7f65d3dc9700 1 -- 172.21.15.2:0/1900165372 wait complete. 2023-08-25T03:35:38.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.540+0000 7f65d3dc9700 1 Processor -- start 2023-08-25T03:35:38.552 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.540+0000 7f65d3dc9700 1 -- start start 2023-08-25T03:35:38.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.541+0000 7f65d3dc9700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65cc0716a0 0x7f65cc1f2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:38.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.541+0000 7f65d3dc9700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65cc15a0b0 0x7f65cc15b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:38.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.541+0000 7f65d3dc9700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f65cc15dbb0 0x7f65cc1f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:38.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.541+0000 7f65d3dc9700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc0ac8e0 con 0x7f65cc0716a0 2023-08-25T03:35:38.553 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.541+0000 7f65d3dc9700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc1f6320 con 0x7f65cc15a0b0 2023-08-25T03:35:38.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.541+0000 7f65d3dc9700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc1f65c0 con 0x7f65cc15dbb0 2023-08-25T03:35:38.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.541+0000 7f65d1b65700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65cc0716a0 0x7f65cc1f2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:38.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.541+0000 7f65d1b65700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65cc0716a0 0x7f65cc1f2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34028/0 (socket says 172.21.15.2:34028) 2023-08-25T03:35:38.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.541+0000 7f65d1b65700 1 -- 172.21.15.2:0/4009543536 learned_addr learned my addr 172.21.15.2:0/4009543536 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:38.554 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.541+0000 7f65d1364700 1 --2- 172.21.15.2:0/4009543536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65cc15a0b0 0x7f65cc15b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:38.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.541+0000 7f65d2366700 1 --2- 172.21.15.2:0/4009543536 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f65cc15dbb0 0x7f65cc1f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:38.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.541+0000 7f65d1b65700 1 -- 172.21.15.2:0/4009543536 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f65cc15dbb0 msgr2=0x7f65cc1f5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.541+0000 7f65d1b65700 1 --2- 172.21.15.2:0/4009543536 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f65cc15dbb0 0x7f65cc1f5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.541+0000 7f65d1b65700 1 -- 172.21.15.2:0/4009543536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65cc15a0b0 msgr2=0x7f65cc15b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.555 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.541+0000 7f65d1b65700 1 --2- 172.21.15.2:0/4009543536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65cc15a0b0 0x7f65cc15b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.541+0000 7f65d1b65700 1 -- 172.21.15.2:0/4009543536 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c80094a0 con 0x7f65cc0716a0 2023-08-25T03:35:38.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.541+0000 7f65d1364700 1 --2- 172.21.15.2:0/4009543536 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65cc15a0b0 0x7f65cc15b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:38.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.541+0000 7f65d1b65700 1 --2- 172.21.15.2:0/4009543536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65cc0716a0 0x7f65cc1f2790 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009fd0 tx=0x7f65c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:38.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.542+0000 7f65be7fc700 1 -- 172.21.15.2:0/4009543536 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c80036a0 con 0x7f65cc0716a0 2023-08-25T03:35:38.556 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.542+0000 7f65be7fc700 1 -- 172.21.15.2:0/4009543536 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c801f070 con 0x7f65cc0716a0 2023-08-25T03:35:38.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.542+0000 7f65be7fc700 1 -- 172.21.15.2:0/4009543536 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c8003a20 con 0x7f65cc0716a0 2023-08-25T03:35:38.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.542+0000 7f65d3dc9700 1 -- 172.21.15.2:0/4009543536 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65cc1f68a0 con 0x7f65cc0716a0 2023-08-25T03:35:38.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.542+0000 7f65d3dc9700 1 -- 172.21.15.2:0/4009543536 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65cc0acb80 con 0x7f65cc0716a0 2023-08-25T03:35:38.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.543+0000 7f65d3dc9700 1 -- 172.21.15.2:0/4009543536 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f65cc15e9c0 con 0x7f65cc0716a0 2023-08-25T03:35:38.557 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.543+0000 7f65be7fc700 1 -- 172.21.15.2:0/4009543536 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f65c8003b80 con 0x7f65cc0716a0 2023-08-25T03:35:38.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.543+0000 7f65be7fc700 1 --2- 172.21.15.2:0/4009543536 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f65b006a0e0 0x7f65b006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:38.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.544+0000 7f65d1364700 1 --2- 172.21.15.2:0/4009543536 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f65b006a0e0 0x7f65b006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:38.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.544+0000 7f65be7fc700 1 -- 172.21.15.2:0/4009543536 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f65c8021070 con 0x7f65cc0716a0 2023-08-25T03:35:38.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.544+0000 7f65be7fc700 1 --2- 172.21.15.2:0/4009543536 >> [v2:172.21.15.31:6808/1680871561,v1:172.21.15.31:6809/1680871561] conn(0x7f65b0070560 0x7f65b0072980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:38.558 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.544+0000 7f65be7fc700 1 -- 172.21.15.2:0/4009543536 --> [v2:172.21.15.31:6808/1680871561,v1:172.21.15.31:6809/1680871561] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f65b0073050 con 0x7f65b0070560 2023-08-25T03:35:38.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.544+0000 7f65be7fc700 1 -- 172.21.15.2:0/4009543536 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65c808e2a0 con 0x7f65cc0716a0 2023-08-25T03:35:38.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.544+0000 7f65d1364700 1 --2- 172.21.15.2:0/4009543536 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f65b006a0e0 0x7f65b006c5a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f65c0007fd0 tx=0x7f65c0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:38.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.544+0000 7f65d2366700 1 --2- 172.21.15.2:0/4009543536 >> [v2:172.21.15.31:6808/1680871561,v1:172.21.15.31:6809/1680871561] conn(0x7f65b0070560 0x7f65b0072980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:38.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.544+0000 7f65d2366700 1 --2- 172.21.15.2:0/4009543536 >> [v2:172.21.15.31:6808/1680871561,v1:172.21.15.31:6809/1680871561] conn(0x7f65b0070560 0x7f65b0072980 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:38.559 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.545+0000 7f65be7fc700 1 -- 172.21.15.2:0/4009543536 <== osd.6 v2:172.21.15.31:6808/1680871561 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f65b0073050 con 0x7f65b0070560 2023-08-25T03:35:38.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.560+0000 7f65d3dc9700 1 -- 172.21.15.2:0/4009543536 --> [v2:172.21.15.31:6808/1680871561,v1:172.21.15.31:6809/1680871561] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f65cc15e6e0 con 0x7f65b0070560 2023-08-25T03:35:38.561 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.561+0000 7f65be7fc700 1 -- 172.21.15.2:0/4009543536 <== osd.6 v2:172.21.15.31:6808/1680871561 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f65c400f040 con 0x7f65b0070560 2023-08-25T03:35:38.562 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-08-25T03:35:38.562 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-08-25T03:35:38.562 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-08-25T03:35:38.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.561+0000 7f65d3dc9700 1 -- 172.21.15.2:0/4009543536 >> [v2:172.21.15.31:6808/1680871561,v1:172.21.15.31:6809/1680871561] conn(0x7f65b0070560 msgr2=0x7f65b0072980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.561+0000 7f65d3dc9700 1 --2- 172.21.15.2:0/4009543536 >> [v2:172.21.15.31:6808/1680871561,v1:172.21.15.31:6809/1680871561] conn(0x7f65b0070560 0x7f65b0072980 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.563 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.561+0000 7f65d3dc9700 1 -- 172.21.15.2:0/4009543536 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f65b006a0e0 msgr2=0x7f65b006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.561+0000 7f65d3dc9700 1 --2- 172.21.15.2:0/4009543536 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f65b006a0e0 0x7f65b006c5a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f65c0007fd0 tx=0x7f65c0009040 comp rx=0 tx=0).stop 2023-08-25T03:35:38.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.561+0000 7f65d3dc9700 1 -- 172.21.15.2:0/4009543536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65cc0716a0 msgr2=0x7f65cc1f2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.561+0000 7f65d3dc9700 1 --2- 172.21.15.2:0/4009543536 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f65cc0716a0 0x7f65cc1f2790 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f65c8009fd0 tx=0x7f65c8004750 comp rx=0 tx=0).stop 2023-08-25T03:35:38.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.561+0000 7f65d2366700 1 -- 172.21.15.2:0/4009543536 reap_dead start 2023-08-25T03:35:38.564 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.561+0000 7f65d3dc9700 1 -- 172.21.15.2:0/4009543536 shutdown_connections 2023-08-25T03:35:38.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.561+0000 7f65d3dc9700 1 -- 172.21.15.2:0/4009543536 >> 172.21.15.2:0/4009543536 conn(0x7f65cc0b1990 msgr2=0x7f65cc1592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:38.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.561+0000 7f65d3dc9700 1 -- 172.21.15.2:0/4009543536 shutdown_connections 2023-08-25T03:35:38.565 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.561+0000 7f65d3dc9700 1 -- 172.21.15.2:0/4009543536 wait complete. 2023-08-25T03:35:38.574 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-08-25T03:35:38.700 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.700+0000 7f8ff180f700 1 Processor -- start 2023-08-25T03:35:38.701 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.700+0000 7f8ff180f700 1 -- start start 2023-08-25T03:35:38.702 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.700+0000 7f8ff180f700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8fec1579b0 0x7f8fec070690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:38.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.700+0000 7f8ff180f700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8fec0c8fe0 0x7f8fec15dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:38.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.700+0000 7f8ff180f700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fec0c9eb0 con 0x7f8fec06b400 2023-08-25T03:35:38.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.700+0000 7f8ff180f700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fec161ed0 con 0x7f8fec1579b0 2023-08-25T03:35:38.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.700+0000 7f8ff180f700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec161040 con 0x7f8fec0c8fe0 2023-08-25T03:35:38.703 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.701+0000 7f8feb7fe700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8fec0c8fe0 0x7f8fec15dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:38.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.701+0000 7f8feaffd700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f8fec1579b0 0x7f8fec070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:38.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.701+0000 7f8feb7fe700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8fec0c8fe0 0x7f8fec15dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.2:34286/0 (socket says 172.21.15.2:34286) 2023-08-25T03:35:38.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.701+0000 7f8fea7fc700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f8fec06b400 0x7f8fec0c88d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.2:56962/0 (socket says 172.21.15.2:56962) 2023-08-25T03:35:38.704 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.701+0000 7f8feb7fe700 1 -- 172.21.15.2:0/1305196681 learned_addr learned my addr 172.21.15.2:0/1305196681 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:38.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.701+0000 7f8fe9ffb700 1 -- 172.21.15.2:0/1305196681 <== mon.1 v1:172.21.15.31:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 525126639 0 0) 0x7f8fec0c9eb0 con 0x7f8fec06b400 2023-08-25T03:35:38.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.701+0000 7f8fe9ffb700 1 -- 172.21.15.2:0/1305196681 --> v1:172.21.15.31:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fd80035b0 con 0x7f8fec06b400 2023-08-25T03:35:38.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.701+0000 7f8feb7fe700 1 -- 172.21.15.2:0/1305196681 >> v2:172.21.15.31:3300/0 conn(0x7f8fec1579b0 msgr2=0x7f8fec070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:38.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.701+0000 7f8feb7fe700 1 --2- 172.21.15.2:0/1305196681 >> v2:172.21.15.31:3300/0 conn(0x7f8fec1579b0 0x7f8fec070690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.705 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.701+0000 7f8feb7fe700 1 -- 172.21.15.2:0/1305196681 >> v1:172.21.15.31:6789/0 conn(0x7f8fec06b400 legacy=0x7f8fec0c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.701+0000 7f8feb7fe700 1 -- 172.21.15.2:0/1305196681 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec161250 con 0x7f8fec0c8fe0 2023-08-25T03:35:38.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.701+0000 7f8feaffd700 1 --2- 172.21.15.2:0/1305196681 >> v2:172.21.15.31:3300/0 conn(0x7f8fec1579b0 0x7f8fec070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:35:38.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.702+0000 7f8feb7fe700 1 --2- 172.21.15.2:0/1305196681 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8fec0c8fe0 0x7f8fec15dd00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009e50 tx=0x7f8fe400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7596ed76395adc4 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:38.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.702+0000 7f8fe9ffb700 1 -- 172.21.15.2:0/1305196681 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe400e070 con 0x7f8fec0c8fe0 2023-08-25T03:35:38.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.702+0000 7f8fe9ffb700 1 -- 172.21.15.2:0/1305196681 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe400b910 con 0x7f8fec0c8fe0 2023-08-25T03:35:38.706 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.702+0000 7f8fe9ffb700 1 -- 172.21.15.2:0/1305196681 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe400bc80 con 0x7f8fec0c8fe0 2023-08-25T03:35:38.710 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.702+0000 7f8ff180f700 1 -- 172.21.15.2:0/1305196681 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8fec0c8fe0 msgr2=0x7f8fec15dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.711 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.702+0000 7f8ff180f700 1 --2- 172.21.15.2:0/1305196681 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8fec0c8fe0 0x7f8fec15dd00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009e50 tx=0x7f8fe400b330 comp rx=0 tx=0).stop 2023-08-25T03:35:38.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.702+0000 7f8ff180f700 1 -- 172.21.15.2:0/1305196681 shutdown_connections 2023-08-25T03:35:38.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.702+0000 7f8ff180f700 1 --2- 172.21.15.2:0/1305196681 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8fec0c8fe0 0x7f8fec15dd00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.712 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.702+0000 7f8ff180f700 1 --2- 172.21.15.2:0/1305196681 >> v2:172.21.15.31:3300/0 conn(0x7f8fec1579b0 0x7f8fec070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.702+0000 7f8ff180f700 1 -- 172.21.15.2:0/1305196681 >> 172.21.15.2:0/1305196681 conn(0x7f8fec0cc020 msgr2=0x7f8fec0cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:38.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.702+0000 7f8ff180f700 1 -- 172.21.15.2:0/1305196681 shutdown_connections 2023-08-25T03:35:38.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.702+0000 7f8ff180f700 1 -- 172.21.15.2:0/1305196681 wait complete. 2023-08-25T03:35:38.713 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.703+0000 7f8ff180f700 1 Processor -- start 2023-08-25T03:35:38.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.704+0000 7f8ff180f700 1 -- start start 2023-08-25T03:35:38.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.704+0000 7f8ff180f700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fec06b400 0x7f8fec15a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:38.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.704+0000 7f8ff180f700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fec0c8fe0 0x7f8fec154270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:38.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.704+0000 7f8ff180f700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8fec1579b0 0x7f8fec1547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:38.714 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.704+0000 7f8ff180f700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec0b3ef0 con 0x7f8fec0c8fe0 2023-08-25T03:35:38.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.704+0000 7f8ff180f700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec0b2e50 con 0x7f8fec06b400 2023-08-25T03:35:38.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.704+0000 7f8ff180f700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec0b3130 con 0x7f8fec1579b0 2023-08-25T03:35:38.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.705+0000 7f8fea7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fec0c8fe0 0x7f8fec154270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:38.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.705+0000 7f8fea7fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fec0c8fe0 0x7f8fec154270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34046/0 (socket says 172.21.15.2:34046) 2023-08-25T03:35:38.715 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.705+0000 7f8fea7fc700 1 -- 172.21.15.2:0/2498988842 learned_addr learned my addr 172.21.15.2:0/2498988842 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:38.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.705+0000 7f8feaffd700 1 --2- 172.21.15.2:0/2498988842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fec06b400 0x7f8fec15a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:38.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.705+0000 7f8feb7fe700 1 --2- 172.21.15.2:0/2498988842 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8fec1579b0 0x7f8fec1547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:38.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.705+0000 7f8fea7fc700 1 -- 172.21.15.2:0/2498988842 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8fec1579b0 msgr2=0x7f8fec1547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.705+0000 7f8fea7fc700 1 --2- 172.21.15.2:0/2498988842 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8fec1579b0 0x7f8fec1547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.716 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.705+0000 7f8fea7fc700 1 -- 172.21.15.2:0/2498988842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fec06b400 msgr2=0x7f8fec15a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.705+0000 7f8fea7fc700 1 --2- 172.21.15.2:0/2498988842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fec06b400 0x7f8fec15a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.705+0000 7f8fea7fc700 1 -- 172.21.15.2:0/2498988842 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fdc0094a0 con 0x7f8fec0c8fe0 2023-08-25T03:35:38.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.705+0000 7f8fea7fc700 1 --2- 172.21.15.2:0/2498988842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fec0c8fe0 0x7f8fec154270 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8fd400c410 tx=0x7f8fd400c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:38.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.705+0000 7f8fe3fff700 1 -- 172.21.15.2:0/2498988842 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4005cb0 con 0x7f8fec0c8fe0 2023-08-25T03:35:38.717 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.705+0000 7f8fe3fff700 1 -- 172.21.15.2:0/2498988842 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd4005e10 con 0x7f8fec0c8fe0 2023-08-25T03:35:38.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.705+0000 7f8ff180f700 1 -- 172.21.15.2:0/2498988842 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe4009b30 con 0x7f8fec0c8fe0 2023-08-25T03:35:38.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.705+0000 7f8fe3fff700 1 -- 172.21.15.2:0/2498988842 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd4007310 con 0x7f8fec0c8fe0 2023-08-25T03:35:38.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.705+0000 7f8ff180f700 1 -- 172.21.15.2:0/2498988842 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fec0b36f0 con 0x7f8fec0c8fe0 2023-08-25T03:35:38.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.706+0000 7f8ff180f700 1 -- 172.21.15.2:0/2498988842 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fec0746d0 con 0x7f8fec0c8fe0 2023-08-25T03:35:38.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.707+0000 7f8fe3fff700 1 -- 172.21.15.2:0/2498988842 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f8fd4007470 con 0x7f8fec0c8fe0 2023-08-25T03:35:38.718 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.707+0000 7f8fe3fff700 1 --2- 172.21.15.2:0/2498988842 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8fcc06a0e0 0x7f8fcc06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:38.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.707+0000 7f8fe3fff700 1 -- 172.21.15.2:0/2498988842 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f8fd4087620 con 0x7f8fec0c8fe0 2023-08-25T03:35:38.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.707+0000 7f8feaffd700 1 --2- 172.21.15.2:0/2498988842 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8fcc06a0e0 0x7f8fcc06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:38.719 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.708+0000 7f8feaffd700 1 --2- 172.21.15.2:0/2498988842 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8fcc06a0e0 0x7f8fcc06c5a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc009d70 tx=0x7f8fdc009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:38.720 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.711+0000 7f8fe3fff700 1 -- 172.21.15.2:0/2498988842 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f8fd40530e0 con 0x7f8fec0c8fe0 2023-08-25T03:35:38.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.909+0000 7f8ff180f700 1 -- 172.21.15.2:0/2498988842 --> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fec161810 con 0x7f8fcc06a0e0 2023-08-25T03:35:38.910 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.910+0000 7f8fe3fff700 1 -- 172.21.15.2:0/2498988842 <== mgr.4108 v2:172.21.15.2:6832/3528779779 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8fdc0099c0 con 0x7f8fcc06a0e0 2023-08-25T03:35:38.912 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.911+0000 7f8ff180f700 1 -- 172.21.15.2:0/2498988842 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8fcc06a0e0 msgr2=0x7f8fcc06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.911+0000 7f8ff180f700 1 --2- 172.21.15.2:0/2498988842 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8fcc06a0e0 0x7f8fcc06c5a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc009d70 tx=0x7f8fdc009210 comp rx=0 tx=0).stop 2023-08-25T03:35:38.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.912+0000 7f8ff180f700 1 -- 172.21.15.2:0/2498988842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fec0c8fe0 msgr2=0x7f8fec154270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:38.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.912+0000 7f8ff180f700 1 --2- 172.21.15.2:0/2498988842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fec0c8fe0 0x7f8fec154270 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8fd400c410 tx=0x7f8fd400c720 comp rx=0 tx=0).stop 2023-08-25T03:35:38.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.912+0000 7f8ff180f700 1 -- 172.21.15.2:0/2498988842 shutdown_connections 2023-08-25T03:35:38.913 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.912+0000 7f8ff180f700 1 --2- 172.21.15.2:0/2498988842 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8fec1579b0 0x7f8fec1547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.912+0000 7f8ff180f700 1 --2- 172.21.15.2:0/2498988842 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8fec06b400 0x7f8fec15a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.912+0000 7f8ff180f700 1 --2- 172.21.15.2:0/2498988842 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f8fcc06a0e0 0x7f8fcc06c5a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.912+0000 7f8ff180f700 1 --2- 172.21.15.2:0/2498988842 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f8fec0c8fe0 0x7f8fec154270 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:38.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.912+0000 7f8ff180f700 1 -- 172.21.15.2:0/2498988842 >> 172.21.15.2:0/2498988842 conn(0x7f8fec0cc020 msgr2=0x7f8fec155ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:38.914 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.912+0000 7f8ff180f700 1 -- 172.21.15.2:0/2498988842 shutdown_connections 2023-08-25T03:35:38.915 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:38.912+0000 7f8ff180f700 1 -- 172.21.15.2:0/2498988842 wait complete. 2023-08-25T03:35:38.915 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 6 to deep-scrub 2023-08-25T03:35:38.924 INFO:tasks.ceph:Scrubbing osd.7 2023-08-25T03:35:38.925 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-08-25T03:35:39.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.052+0000 7ff018c84700 1 Processor -- start 2023-08-25T03:35:39.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.052+0000 7ff018c84700 1 -- start start 2023-08-25T03:35:39.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.053+0000 7ff018c84700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7ff01415a0b0 0x7ff01406e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:39.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.053+0000 7ff018c84700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff0140716a0 0x7ff0140ca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:39.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.053+0000 7ff018c84700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff01415a490 con 0x7ff01415dbb0 2023-08-25T03:35:39.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.053+0000 7ff018c84700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff01406e6c0 con 0x7ff0140716a0 2023-08-25T03:35:39.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.053+0000 7ff018c84700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff01415e120 con 0x7ff01415a0b0 2023-08-25T03:35:39.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.053+0000 7ff012d9d700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7ff0140716a0 0x7ff0140ca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:39.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.054+0000 7ff011d9b700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7ff01415dbb0 0x7ff014070f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48952/0 (socket says 172.21.15.2:48952) 2023-08-25T03:35:39.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.054+0000 7ff011d9b700 1 -- 172.21.15.2:0/1725073363 learned_addr learned my addr 172.21.15.2:0/1725073363 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:39.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.054+0000 7ff01259c700 1 --2- 172.21.15.2:0/1725073363 >> v2:172.21.15.31:3300/0 conn(0x7ff01415a0b0 0x7ff01406e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:39.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.054+0000 7ff012d9d700 1 -- 172.21.15.2:0/1725073363 >> v2:172.21.15.31:3300/0 conn(0x7ff01415a0b0 msgr2=0x7ff01406e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:39.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.054+0000 7ff012d9d700 1 --2- 172.21.15.2:0/1725073363 >> v2:172.21.15.31:3300/0 conn(0x7ff01415a0b0 0x7ff01406e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.054+0000 7ff012d9d700 1 -- 172.21.15.2:0/1725073363 >> v1:172.21.15.2:6789/0 conn(0x7ff01415dbb0 legacy=0x7ff014070f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:39.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.054+0000 7ff01159a700 1 -- 172.21.15.2:0/1725073363 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3338264136 0 0) 0x7ff008001110 con 0x7ff01415dbb0 2023-08-25T03:35:39.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.054+0000 7ff012d9d700 1 -- 172.21.15.2:0/1725073363 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff01415e400 con 0x7ff0140716a0 2023-08-25T03:35:39.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.054+0000 7ff012d9d700 1 --2- 172.21.15.2:0/1725073363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0140716a0 0x7ff0140ca9d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff01406fb20 tx=0x7ff00c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29df133b41536e4d server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:39.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.054+0000 7ff01159a700 1 -- 172.21.15.2:0/1725073363 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c00e070 con 0x7ff0140716a0 2023-08-25T03:35:39.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.054+0000 7ff01159a700 1 -- 172.21.15.2:0/1725073363 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00c00bb20 con 0x7ff0140716a0 2023-08-25T03:35:39.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.054+0000 7ff01159a700 1 -- 172.21.15.2:0/1725073363 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c012430 con 0x7ff0140716a0 2023-08-25T03:35:39.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.055+0000 7ff018c84700 1 -- 172.21.15.2:0/1725073363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0140716a0 msgr2=0x7ff0140ca9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:39.059 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.055+0000 7ff018c84700 1 --2- 172.21.15.2:0/1725073363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0140716a0 0x7ff0140ca9d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff01406fb20 tx=0x7ff00c00b330 comp rx=0 tx=0).stop 2023-08-25T03:35:39.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.055+0000 7ff018c84700 1 -- 172.21.15.2:0/1725073363 shutdown_connections 2023-08-25T03:35:39.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.055+0000 7ff018c84700 1 --2- 172.21.15.2:0/1725073363 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff0140716a0 0x7ff0140ca9d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.055+0000 7ff018c84700 1 --2- 172.21.15.2:0/1725073363 >> v2:172.21.15.31:3300/0 conn(0x7ff01415a0b0 0x7ff01406e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.055+0000 7ff018c84700 1 -- 172.21.15.2:0/1725073363 >> 172.21.15.2:0/1725073363 conn(0x7ff0140b1990 msgr2=0x7ff0140b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:39.060 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.055+0000 7ff018c84700 1 -- 172.21.15.2:0/1725073363 shutdown_connections 2023-08-25T03:35:39.061 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.055+0000 7ff018c84700 1 -- 172.21.15.2:0/1725073363 wait complete. 2023-08-25T03:35:39.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.056+0000 7ff018c84700 1 Processor -- start 2023-08-25T03:35:39.063 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.056+0000 7ff018c84700 1 -- start start 2023-08-25T03:35:39.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.057+0000 7ff018c84700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0140716a0 0x7ff0140ce000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:39.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.057+0000 7ff018c84700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff01415a0b0 0x7ff01414d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:39.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.057+0000 7ff018c84700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff01415dbb0 0x7ff01415c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:39.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.057+0000 7ff018c84700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0141f1370 con 0x7ff01415dbb0 2023-08-25T03:35:39.064 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.057+0000 7ff018c84700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01414d610 con 0x7ff0140716a0 2023-08-25T03:35:39.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.057+0000 7ff018c84700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0141f0760 con 0x7ff01415a0b0 2023-08-25T03:35:39.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.057+0000 7ff012d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff01415dbb0 0x7ff01415c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:39.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.057+0000 7ff012d9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff01415dbb0 0x7ff01415c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34062/0 (socket says 172.21.15.2:34062) 2023-08-25T03:35:39.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.057+0000 7ff012d9d700 1 -- 172.21.15.2:0/1511036689 learned_addr learned my addr 172.21.15.2:0/1511036689 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:39.065 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.057+0000 7ff01259c700 1 --2- 172.21.15.2:0/1511036689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0140716a0 0x7ff0140ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:39.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.057+0000 7ff011d9b700 1 --2- 172.21.15.2:0/1511036689 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff01415a0b0 0x7ff01414d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:39.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.058+0000 7ff012d9d700 1 -- 172.21.15.2:0/1511036689 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff01415a0b0 msgr2=0x7ff01414d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:39.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.058+0000 7ff012d9d700 1 --2- 172.21.15.2:0/1511036689 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff01415a0b0 0x7ff01414d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.058+0000 7ff012d9d700 1 -- 172.21.15.2:0/1511036689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0140716a0 msgr2=0x7ff0140ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:39.066 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.058+0000 7ff012d9d700 1 --2- 172.21.15.2:0/1511036689 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0140716a0 0x7ff0140ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.058+0000 7ff012d9d700 1 -- 172.21.15.2:0/1511036689 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff00c009a70 con 0x7ff01415dbb0 2023-08-25T03:35:39.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.058+0000 7ff011d9b700 1 --2- 172.21.15.2:0/1511036689 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff01415a0b0 0x7ff01414d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:39.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.058+0000 7ff012d9d700 1 --2- 172.21.15.2:0/1511036689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff01415dbb0 0x7ff01415c640 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff0140b2500 tx=0x7ff00c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:39.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.058+0000 7ff002ffd700 1 -- 172.21.15.2:0/1511036689 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c012430 con 0x7ff01415dbb0 2023-08-25T03:35:39.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.058+0000 7ff002ffd700 1 -- 172.21.15.2:0/1511036689 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00c020070 con 0x7ff01415dbb0 2023-08-25T03:35:39.067 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.058+0000 7ff018c84700 1 -- 172.21.15.2:0/1511036689 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0141f0a40 con 0x7ff01415dbb0 2023-08-25T03:35:39.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.058+0000 7ff002ffd700 1 -- 172.21.15.2:0/1511036689 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00c004280 con 0x7ff01415dbb0 2023-08-25T03:35:39.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.058+0000 7ff018c84700 1 -- 172.21.15.2:0/1511036689 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0141f1000 con 0x7ff01415dbb0 2023-08-25T03:35:39.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.059+0000 7ff018c84700 1 -- 172.21.15.2:0/1511036689 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff01415e9c0 con 0x7ff01415dbb0 2023-08-25T03:35:39.068 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.059+0000 7ff002ffd700 1 -- 172.21.15.2:0/1511036689 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7ff00c0043e0 con 0x7ff01415dbb0 2023-08-25T03:35:39.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.060+0000 7ff002ffd700 1 --2- 172.21.15.2:0/1511036689 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7feff406a110 0x7feff406c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:39.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.060+0000 7ff002ffd700 1 -- 172.21.15.2:0/1511036689 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7ff00c022070 con 0x7ff01415dbb0 2023-08-25T03:35:39.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.060+0000 7ff01259c700 1 --2- 172.21.15.2:0/1511036689 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7feff406a110 0x7feff406c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:39.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.060+0000 7ff002ffd700 1 --2- 172.21.15.2:0/1511036689 >> [v2:172.21.15.31:6816/3294787191,v1:172.21.15.31:6817/3294787191] conn(0x7feff4070590 0x7feff40729b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:39.069 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.060+0000 7ff002ffd700 1 -- 172.21.15.2:0/1511036689 --> [v2:172.21.15.31:6816/3294787191,v1:172.21.15.31:6817/3294787191] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feff4073080 con 0x7feff4070590 2023-08-25T03:35:39.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.060+0000 7ff002ffd700 1 -- 172.21.15.2:0/1511036689 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff00c08c750 con 0x7ff01415dbb0 2023-08-25T03:35:39.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.060+0000 7ff011d9b700 1 --2- 172.21.15.2:0/1511036689 >> [v2:172.21.15.31:6816/3294787191,v1:172.21.15.31:6817/3294787191] conn(0x7feff4070590 0x7feff40729b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:39.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.060+0000 7ff01259c700 1 --2- 172.21.15.2:0/1511036689 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7feff406a110 0x7feff406c5d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff004009be0 tx=0x7ff004009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:39.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.061+0000 7ff011d9b700 1 --2- 172.21.15.2:0/1511036689 >> [v2:172.21.15.31:6816/3294787191,v1:172.21.15.31:6817/3294787191] conn(0x7feff4070590 0x7feff40729b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:39.070 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.061+0000 7ff002ffd700 1 -- 172.21.15.2:0/1511036689 <== osd.7 v2:172.21.15.31:6816/3294787191 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7feff4073080 con 0x7feff4070590 2023-08-25T03:35:39.077 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.076+0000 7ff018c84700 1 -- 172.21.15.2:0/1511036689 --> [v2:172.21.15.31:6816/3294787191,v1:172.21.15.31:6817/3294787191] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff01415e6e0 con 0x7feff4070590 2023-08-25T03:35:39.078 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.077+0000 7ff002ffd700 1 -- 172.21.15.2:0/1511036689 <== osd.7 v2:172.21.15.31:6816/3294787191 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff008007170 con 0x7feff4070590 2023-08-25T03:35:39.078 INFO:teuthology.orchestra.run.smithi002.stdout:{ 2023-08-25T03:35:39.078 INFO:teuthology.orchestra.run.smithi002.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-08-25T03:35:39.079 INFO:teuthology.orchestra.run.smithi002.stdout:} 2023-08-25T03:35:39.079 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.077+0000 7ff018c84700 1 -- 172.21.15.2:0/1511036689 >> [v2:172.21.15.31:6816/3294787191,v1:172.21.15.31:6817/3294787191] conn(0x7feff4070590 msgr2=0x7feff40729b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:39.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.077+0000 7ff018c84700 1 --2- 172.21.15.2:0/1511036689 >> [v2:172.21.15.31:6816/3294787191,v1:172.21.15.31:6817/3294787191] conn(0x7feff4070590 0x7feff40729b0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.077+0000 7ff018c84700 1 -- 172.21.15.2:0/1511036689 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7feff406a110 msgr2=0x7feff406c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:39.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.077+0000 7ff018c84700 1 --2- 172.21.15.2:0/1511036689 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7feff406a110 0x7feff406c5d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff004009be0 tx=0x7ff004009210 comp rx=0 tx=0).stop 2023-08-25T03:35:39.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.077+0000 7ff018c84700 1 -- 172.21.15.2:0/1511036689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff01415dbb0 msgr2=0x7ff01415c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:39.080 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.077+0000 7ff018c84700 1 --2- 172.21.15.2:0/1511036689 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7ff01415dbb0 0x7ff01415c640 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff0140b2500 tx=0x7ff00c004ad0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.077+0000 7ff012d9d700 1 -- 172.21.15.2:0/1511036689 reap_dead start 2023-08-25T03:35:39.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.077+0000 7ff018c84700 1 -- 172.21.15.2:0/1511036689 shutdown_connections 2023-08-25T03:35:39.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.077+0000 7ff018c84700 1 -- 172.21.15.2:0/1511036689 >> 172.21.15.2:0/1511036689 conn(0x7ff0140b1990 msgr2=0x7ff0140b1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:39.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.077+0000 7ff018c84700 1 -- 172.21.15.2:0/1511036689 shutdown_connections 2023-08-25T03:35:39.081 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.077+0000 7ff018c84700 1 -- 172.21.15.2:0/1511036689 wait complete. 2023-08-25T03:35:39.090 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-08-25T03:35:39.214 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.214+0000 7f2050966700 1 Processor -- start 2023-08-25T03:35:39.215 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.214+0000 7f2050966700 1 -- start start 2023-08-25T03:35:39.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.215+0000 7f2050966700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f204c15ea20 0x7f204c0b2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:39.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.215+0000 7f2050966700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f204c0cca90 0x7f204c0d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:39.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.215+0000 7f2050966700 1 -- --> v1:172.21.15.31:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f204c0ccf10 con 0x7f204c0cf5b0 2023-08-25T03:35:39.216 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.215+0000 7f2050966700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f204c0cbd10 con 0x7f204c15ea20 2023-08-25T03:35:39.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.215+0000 7f2050966700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204c0cbfb0 con 0x7f204c0cca90 2023-08-25T03:35:39.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.215+0000 7f2049d9b700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f204c0cca90 0x7f204c0d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:39.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.215+0000 7f204ad9d700 1 --1- >> v1:172.21.15.31:6789/0 conn(0x7f204c0cf5b0 0x7f204c0b3e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.31:6789/0 says I am v1:172.21.15.2:56994/0 (socket says 172.21.15.2:56994) 2023-08-25T03:35:39.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.215+0000 7f204a59c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f204c15ea20 0x7f204c0b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:39.217 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.215+0000 7f204ad9d700 1 -- 172.21.15.2:0/418134734 learned_addr learned my addr 172.21.15.2:0/418134734 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:39.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.216+0000 7f204a59c700 1 -- 172.21.15.2:0/418134734 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f204c0cca90 msgr2=0x7f204c0d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:39.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.216+0000 7f204a59c700 1 --2- 172.21.15.2:0/418134734 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f204c0cca90 0x7f204c0d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.216+0000 7f204a59c700 1 -- 172.21.15.2:0/418134734 >> v1:172.21.15.31:6789/0 conn(0x7f204c0cf5b0 legacy=0x7f204c0b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:39.219 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.216+0000 7f204a59c700 1 -- 172.21.15.2:0/418134734 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f204c0cc290 con 0x7f204c15ea20 2023-08-25T03:35:39.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.216+0000 7f204a59c700 1 --2- 172.21.15.2:0/418134734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f204c15ea20 0x7f204c0b2020 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f203c009840 tx=0x7f203c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f26c7ac196304b3c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:39.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.216+0000 7f204959a700 1 -- 172.21.15.2:0/418134734 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c011830 con 0x7f204c15ea20 2023-08-25T03:35:39.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.216+0000 7f204959a700 1 -- 172.21.15.2:0/418134734 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203c011990 con 0x7f204c15ea20 2023-08-25T03:35:39.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.216+0000 7f204959a700 1 -- 172.21.15.2:0/418134734 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c011d20 con 0x7f204c15ea20 2023-08-25T03:35:39.220 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.216+0000 7f2050966700 1 -- 172.21.15.2:0/418134734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f204c15ea20 msgr2=0x7f204c0b2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:39.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.216+0000 7f2050966700 1 --2- 172.21.15.2:0/418134734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f204c15ea20 0x7f204c0b2020 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f203c009840 tx=0x7f203c009b50 comp rx=0 tx=0).stop 2023-08-25T03:35:39.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.217+0000 7f2050966700 1 -- 172.21.15.2:0/418134734 shutdown_connections 2023-08-25T03:35:39.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.217+0000 7f2050966700 1 --2- 172.21.15.2:0/418134734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f204c15ea20 0x7f204c0b2020 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.217+0000 7f2050966700 1 --2- 172.21.15.2:0/418134734 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f204c0cca90 0x7f204c0d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.221 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.217+0000 7f2050966700 1 -- 172.21.15.2:0/418134734 >> 172.21.15.2:0/418134734 conn(0x7f204c09a2b0 msgr2=0x7f204c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:39.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.217+0000 7f2050966700 1 -- 172.21.15.2:0/418134734 shutdown_connections 2023-08-25T03:35:39.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.217+0000 7f2050966700 1 -- 172.21.15.2:0/418134734 wait complete. 2023-08-25T03:35:39.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.218+0000 7f2050966700 1 Processor -- start 2023-08-25T03:35:39.222 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.218+0000 7f2050966700 1 -- start start 2023-08-25T03:35:39.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.219+0000 7f2050966700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f204c0cca90 0x7f204c15c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:39.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.219+0000 7f2050966700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f204c0cf5b0 0x7f204c0c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:39.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.219+0000 7f2050966700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f204c15ea20 0x7f204c0ca170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:39.224 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.219+0000 7f2050966700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c15bf30 con 0x7f204c0cca90 2023-08-25T03:35:39.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.219+0000 7f2050966700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f204c0cac70 con 0x7f204c0cf5b0 2023-08-25T03:35:39.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.219+0000 7f2050966700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f204c0c3f40 con 0x7f204c15ea20 2023-08-25T03:35:39.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.219+0000 7f204a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f204c0cca90 0x7f204c15c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:39.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.219+0000 7f204a59c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f204c0cca90 0x7f204c15c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34074/0 (socket says 172.21.15.2:34074) 2023-08-25T03:35:39.225 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.219+0000 7f204a59c700 1 -- 172.21.15.2:0/1823109019 learned_addr learned my addr 172.21.15.2:0/1823109019 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:39.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.219+0000 7f2049d9b700 1 --2- 172.21.15.2:0/1823109019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f204c0cf5b0 0x7f204c0c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:39.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.219+0000 7f204ad9d700 1 --2- 172.21.15.2:0/1823109019 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f204c15ea20 0x7f204c0ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:39.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.219+0000 7f204a59c700 1 -- 172.21.15.2:0/1823109019 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f204c15ea20 msgr2=0x7f204c0ca170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:39.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.219+0000 7f204a59c700 1 --2- 172.21.15.2:0/1823109019 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f204c15ea20 0x7f204c0ca170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.226 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.219+0000 7f204a59c700 1 -- 172.21.15.2:0/1823109019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f204c0cf5b0 msgr2=0x7f204c0c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:39.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.219+0000 7f204a59c700 1 --2- 172.21.15.2:0/1823109019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f204c0cf5b0 0x7f204c0c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.219+0000 7f204a59c700 1 -- 172.21.15.2:0/1823109019 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f203c0094a0 con 0x7f204c0cca90 2023-08-25T03:35:39.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.219+0000 7f204a59c700 1 --2- 172.21.15.2:0/1823109019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f204c0cca90 0x7f204c15c350 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f203c009fd0 tx=0x7f203c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:39.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.220+0000 7f203affd700 1 -- 172.21.15.2:0/1823109019 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c0036a0 con 0x7f204c0cca90 2023-08-25T03:35:39.227 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.220+0000 7f203affd700 1 -- 172.21.15.2:0/1823109019 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203c01e070 con 0x7f204c0cca90 2023-08-25T03:35:39.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.220+0000 7f203affd700 1 -- 172.21.15.2:0/1823109019 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c003a60 con 0x7f204c0cca90 2023-08-25T03:35:39.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.220+0000 7f2050966700 1 -- 172.21.15.2:0/1823109019 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f204c0caf30 con 0x7f204c0cca90 2023-08-25T03:35:39.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.220+0000 7f2050966700 1 -- 172.21.15.2:0/1823109019 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f204c0cb210 con 0x7f204c0cca90 2023-08-25T03:35:39.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.221+0000 7f2050966700 1 -- 172.21.15.2:0/1823109019 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f204c0746d0 con 0x7f204c0cca90 2023-08-25T03:35:39.228 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.221+0000 7f203affd700 1 -- 172.21.15.2:0/1823109019 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f203c003bc0 con 0x7f204c0cca90 2023-08-25T03:35:39.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.222+0000 7f203affd700 1 --2- 172.21.15.2:0/1823109019 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f202c06a0e0 0x7f202c06c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:39.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.222+0000 7f2049d9b700 1 --2- 172.21.15.2:0/1823109019 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f202c06a0e0 0x7f202c06c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:39.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.222+0000 7f203affd700 1 -- 172.21.15.2:0/1823109019 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f203c0891f0 con 0x7f204c0cca90 2023-08-25T03:35:39.229 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.222+0000 7f2049d9b700 1 --2- 172.21.15.2:0/1823109019 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f202c06a0e0 0x7f202c06c5a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f204c157940 tx=0x7f2040008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:39.230 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.226+0000 7f203affd700 1 -- 172.21.15.2:0/1823109019 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f203c054c70 con 0x7f204c0cca90 2023-08-25T03:35:39.426 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.426+0000 7f2050966700 1 -- 172.21.15.2:0/1823109019 --> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f204c15ee00 con 0x7f202c06a0e0 2023-08-25T03:35:39.427 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.426+0000 7f203affd700 1 -- 172.21.15.2:0/1823109019 <== mgr.4108 v2:172.21.15.2:6832/3528779779 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f2040011780 con 0x7f202c06a0e0 2023-08-25T03:35:39.429 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.428+0000 7f2050966700 1 -- 172.21.15.2:0/1823109019 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f202c06a0e0 msgr2=0x7f202c06c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:39.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.428+0000 7f2050966700 1 --2- 172.21.15.2:0/1823109019 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f202c06a0e0 0x7f202c06c5a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f204c157940 tx=0x7f2040008040 comp rx=0 tx=0).stop 2023-08-25T03:35:39.430 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.428+0000 7f2050966700 1 -- 172.21.15.2:0/1823109019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f204c0cca90 msgr2=0x7f204c15c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:39.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.428+0000 7f2050966700 1 --2- 172.21.15.2:0/1823109019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f204c0cca90 0x7f204c15c350 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f203c009fd0 tx=0x7f203c004750 comp rx=0 tx=0).stop 2023-08-25T03:35:39.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.428+0000 7f2050966700 1 -- 172.21.15.2:0/1823109019 shutdown_connections 2023-08-25T03:35:39.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.428+0000 7f2050966700 1 --2- 172.21.15.2:0/1823109019 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f202c06a0e0 0x7f202c06c5a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.431 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.428+0000 7f2050966700 1 --2- 172.21.15.2:0/1823109019 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f204c0cca90 0x7f204c15c350 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.429+0000 7f2050966700 1 --2- 172.21.15.2:0/1823109019 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f204c15ea20 0x7f204c0ca170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.429+0000 7f2050966700 1 --2- 172.21.15.2:0/1823109019 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f204c0cf5b0 0x7f204c0c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.432 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.429+0000 7f2050966700 1 -- 172.21.15.2:0/1823109019 >> 172.21.15.2:0/1823109019 conn(0x7f204c09a2b0 msgr2=0x7f204c15a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:39.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.429+0000 7f2050966700 1 -- 172.21.15.2:0/1823109019 shutdown_connections 2023-08-25T03:35:39.433 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.429+0000 7f2050966700 1 -- 172.21.15.2:0/1823109019 wait complete. 2023-08-25T03:35:39.434 INFO:teuthology.orchestra.run.smithi002.stderr:instructed osd(s) 7 to deep-scrub 2023-08-25T03:35:39.442 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T03:35:39.566 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.566+0000 7fee8772c700 1 Processor -- start 2023-08-25T03:35:39.567 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.566+0000 7fee8772c700 1 -- start start 2023-08-25T03:35:39.569 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.567+0000 7fee8772c700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7fee800b19a0 0x7fee8015e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.567+0000 7fee8772c700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fee800b1d80 0x7fee80071430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:39.570 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.567+0000 7fee8772c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fee8006b2f0 0x7fee8006b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.567+0000 7fee8772c700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee800720b0 con 0x7fee8006b2f0 2023-08-25T03:35:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.567+0000 7fee8772c700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee80071dd0 con 0x7fee800b19a0 2023-08-25T03:35:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.567+0000 7fee8772c700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee8015ddb0 con 0x7fee800b1d80 2023-08-25T03:35:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.567+0000 7fee85cc9700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fee8006b2f0 0x7fee8006b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:39.571 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.567+0000 7fee85cc9700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7fee8006b2f0 0x7fee8006b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34086/0 (socket says 172.21.15.2:34086) 2023-08-25T03:35:39.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.567+0000 7fee85cc9700 1 -- 172.21.15.2:0/4169887344 learned_addr learned my addr 172.21.15.2:0/4169887344 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:39.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.567+0000 7fee854c8700 1 --2- 172.21.15.2:0/4169887344 >> v2:172.21.15.31:3300/0 conn(0x7fee800b19a0 0x7fee8015e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:39.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.567+0000 7fee84cc7700 1 --2- 172.21.15.2:0/4169887344 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fee800b1d80 0x7fee80071430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:39.572 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.567+0000 7fee85cc9700 1 -- 172.21.15.2:0/4169887344 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fee800b1d80 msgr2=0x7fee80071430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:39.573 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.567+0000 7fee85cc9700 1 --2- 172.21.15.2:0/4169887344 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fee800b1d80 0x7fee80071430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.567+0000 7fee85cc9700 1 -- 172.21.15.2:0/4169887344 >> v2:172.21.15.31:3300/0 conn(0x7fee800b19a0 msgr2=0x7fee8015e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:39.574 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.567+0000 7fee85cc9700 1 --2- 172.21.15.2:0/4169887344 >> v2:172.21.15.31:3300/0 conn(0x7fee800b19a0 0x7fee8015e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.567+0000 7fee85cc9700 1 -- 172.21.15.2:0/4169887344 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee8015e090 con 0x7fee8006b2f0 2023-08-25T03:35:39.575 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.567+0000 7fee85cc9700 1 --2- 172.21.15.2:0/4169887344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee8006b2f0 0x7fee8006b780 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fee8006c390 tx=0x7fee7800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8de57f71780a21 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:39.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.567+0000 7fee77fff700 1 -- 172.21.15.2:0/4169887344 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7800e070 con 0x7fee8006b2f0 2023-08-25T03:35:39.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.567+0000 7fee77fff700 1 -- 172.21.15.2:0/4169887344 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7800b9c0 con 0x7fee8006b2f0 2023-08-25T03:35:39.576 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.567+0000 7fee77fff700 1 -- 172.21.15.2:0/4169887344 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7800bd70 con 0x7fee8006b2f0 2023-08-25T03:35:39.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.568+0000 7fee8772c700 1 -- 172.21.15.2:0/4169887344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee8006b2f0 msgr2=0x7fee8006b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:39.577 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.568+0000 7fee8772c700 1 --2- 172.21.15.2:0/4169887344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee8006b2f0 0x7fee8006b780 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fee8006c390 tx=0x7fee7800b330 comp rx=0 tx=0).stop 2023-08-25T03:35:39.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.568+0000 7fee8772c700 1 -- 172.21.15.2:0/4169887344 shutdown_connections 2023-08-25T03:35:39.578 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.568+0000 7fee8772c700 1 --2- 172.21.15.2:0/4169887344 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee8006b2f0 0x7fee8006b780 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.568+0000 7fee8772c700 1 --2- 172.21.15.2:0/4169887344 >> v2:172.21.15.31:3300/0 conn(0x7fee800b19a0 0x7fee8015e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.568+0000 7fee8772c700 1 --2- 172.21.15.2:0/4169887344 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fee800b1d80 0x7fee80071430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.579 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.568+0000 7fee8772c700 1 -- 172.21.15.2:0/4169887344 >> 172.21.15.2:0/4169887344 conn(0x7fee800cfea0 msgr2=0x7fee800d0280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:39.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.568+0000 7fee8772c700 1 -- 172.21.15.2:0/4169887344 shutdown_connections 2023-08-25T03:35:39.580 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.568+0000 7fee8772c700 1 -- 172.21.15.2:0/4169887344 wait complete. 2023-08-25T03:35:39.582 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.569+0000 7fee8772c700 1 Processor -- start 2023-08-25T03:35:39.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.569+0000 7fee8772c700 1 -- start start 2023-08-25T03:35:39.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.570+0000 7fee8772c700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fee8006b2f0 0x7fee80143f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:39.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.570+0000 7fee8772c700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee800b19a0 0x7fee800c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:39.583 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.570+0000 7fee8772c700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee800b1d80 0x7fee800c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:39.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.570+0000 7fee8772c700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8015d360 con 0x7fee800b19a0 2023-08-25T03:35:39.584 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.570+0000 7fee8772c700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee8015d640 con 0x7fee800b1d80 2023-08-25T03:35:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.570+0000 7fee8772c700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee800b5650 con 0x7fee8006b2f0 2023-08-25T03:35:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.570+0000 7fee84cc7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee800b19a0 0x7fee800c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.570+0000 7fee84cc7700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee800b19a0 0x7fee800c7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34090/0 (socket says 172.21.15.2:34090) 2023-08-25T03:35:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.570+0000 7fee84cc7700 1 -- 172.21.15.2:0/3845872851 learned_addr learned my addr 172.21.15.2:0/3845872851 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:39.585 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.570+0000 7fee854c8700 1 --2- 172.21.15.2:0/3845872851 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fee8006b2f0 0x7fee80143f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:39.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.570+0000 7fee85cc9700 1 --2- 172.21.15.2:0/3845872851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee800b1d80 0x7fee800c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:39.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.571+0000 7fee84cc7700 1 -- 172.21.15.2:0/3845872851 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fee8006b2f0 msgr2=0x7fee80143f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:39.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.571+0000 7fee84cc7700 1 --2- 172.21.15.2:0/3845872851 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fee8006b2f0 0x7fee80143f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.571+0000 7fee84cc7700 1 -- 172.21.15.2:0/3845872851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee800b1d80 msgr2=0x7fee800c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:39.586 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.571+0000 7fee84cc7700 1 --2- 172.21.15.2:0/3845872851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee800b1d80 0x7fee800c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.571+0000 7fee84cc7700 1 -- 172.21.15.2:0/3845872851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee78009b30 con 0x7fee800b19a0 2023-08-25T03:35:39.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.571+0000 7fee84cc7700 1 --2- 172.21.15.2:0/3845872851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee800b19a0 0x7fee800c7b30 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fee7000d800 tx=0x7fee7000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:39.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.571+0000 7fee75ffb700 1 -- 172.21.15.2:0/3845872851 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee700125f0 con 0x7fee800b19a0 2023-08-25T03:35:39.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.571+0000 7fee75ffb700 1 -- 172.21.15.2:0/3845872851 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee70012750 con 0x7fee800b19a0 2023-08-25T03:35:39.587 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.571+0000 7fee8772c700 1 -- 172.21.15.2:0/3845872851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee800b5930 con 0x7fee800b19a0 2023-08-25T03:35:39.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.571+0000 7fee75ffb700 1 -- 172.21.15.2:0/3845872851 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee70012b00 con 0x7fee800b19a0 2023-08-25T03:35:39.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.571+0000 7fee8772c700 1 -- 172.21.15.2:0/3845872851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee80157070 con 0x7fee800b19a0 2023-08-25T03:35:39.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.572+0000 7fee8772c700 1 -- 172.21.15.2:0/3845872851 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee800b54c0 con 0x7fee800b19a0 2023-08-25T03:35:39.588 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.573+0000 7fee75ffb700 1 -- 172.21.15.2:0/3845872851 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7fee70012c60 con 0x7fee800b19a0 2023-08-25T03:35:39.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.573+0000 7fee75ffb700 1 --2- 172.21.15.2:0/3845872851 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fee6406a0e0 0x7fee6406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:39.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.573+0000 7fee75ffb700 1 -- 172.21.15.2:0/3845872851 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fee70088240 con 0x7fee800b19a0 2023-08-25T03:35:39.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.573+0000 7fee854c8700 1 --2- 172.21.15.2:0/3845872851 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fee6406a0e0 0x7fee6406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:39.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.573+0000 7fee854c8700 1 --2- 172.21.15.2:0/3845872851 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fee6406a0e0 0x7fee6406c5a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009950 tx=0x7fee7c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:39.589 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.577+0000 7fee75ffb700 1 -- 172.21.15.2:0/3845872851 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7fee70053ce0 con 0x7fee800b19a0 2023-08-25T03:35:39.776 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.775+0000 7fee8772c700 1 -- 172.21.15.2:0/3845872851 --> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fee8015d7b0 con 0x7fee6406a0e0 2023-08-25T03:35:39.779 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.778+0000 7fee75ffb700 1 -- 172.21.15.2:0/3845872851 <== mgr.4108 v2:172.21.15.2:6832/3528779779 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72690 (secure 0 0 0) 0x7fee8015d7b0 con 0x7fee6406a0e0 2023-08-25T03:35:39.780 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:35:39.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.780+0000 7fee8772c700 1 -- 172.21.15.2:0/3845872851 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fee6406a0e0 msgr2=0x7fee6406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:39.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.780+0000 7fee8772c700 1 --2- 172.21.15.2:0/3845872851 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fee6406a0e0 0x7fee6406c5a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009950 tx=0x7fee7c008040 comp rx=0 tx=0).stop 2023-08-25T03:35:39.781 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.780+0000 7fee8772c700 1 -- 172.21.15.2:0/3845872851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee800b19a0 msgr2=0x7fee800c7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:39.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.780+0000 7fee8772c700 1 --2- 172.21.15.2:0/3845872851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee800b19a0 0x7fee800c7b30 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fee7000d800 tx=0x7fee7000db10 comp rx=0 tx=0).stop 2023-08-25T03:35:39.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.780+0000 7fee8772c700 1 -- 172.21.15.2:0/3845872851 shutdown_connections 2023-08-25T03:35:39.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.780+0000 7fee8772c700 1 --2- 172.21.15.2:0/3845872851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee800b1d80 0x7fee800c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.782 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.780+0000 7fee8772c700 1 --2- 172.21.15.2:0/3845872851 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fee8006b2f0 0x7fee80143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.780+0000 7fee8772c700 1 --2- 172.21.15.2:0/3845872851 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7fee6406a0e0 0x7fee6406c5a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.780+0000 7fee8772c700 1 --2- 172.21.15.2:0/3845872851 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7fee800b19a0 0x7fee800c7b30 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:39.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.780+0000 7fee8772c700 1 -- 172.21.15.2:0/3845872851 >> 172.21.15.2:0/3845872851 conn(0x7fee800cfea0 msgr2=0x7fee8006b6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:39.783 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.780+0000 7fee8772c700 1 -- 172.21.15.2:0/3845872851 shutdown_connections 2023-08-25T03:35:39.784 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:39.780+0000 7fee8772c700 1 -- 172.21.15.2:0/3845872851 wait complete. 2023-08-25T03:35:39.784 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-08-25T03:35:39.795 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2023-08-25T03:35:39.523959+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":220944,"kb_used_data":4776,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754753776,"statfs":{"total":773094113280,"available":772867866624,"internally_reserved":0,"allocated":4890624,"data_stored":2716568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.603098"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:28.924025+0000","last_change":"2023-08-25T03:33:57.570267+0000","last_active":"2023-08-25T03:35:28.924025+0000","last_peered":"2023-08-25T03:35:28.924025+0000","last_clean":"2023-08-25T03:35:28.924025+0000","last_became_active":"2023-08-25T03:33:57.569882+0000","last_became_peered":"2023-08-25T03:33:57.569882+0000","last_unstale":"2023-08-25T03:35:28.924025+0000","last_undegraded":"2023-08-25T03:35:28.924025+0000","last_fullsized":"2023-08-25T03:35:28.924025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:29.169807+0000","last_change":"2023-08-25T03:33:57.569595+0000","last_active":"2023-08-25T03:35:29.169807+0000","last_peered":"2023-08-25T03:35:29.169807+0000","last_clean":"2023-08-25T03:35:29.169807+0000","last_became_active":"2023-08-25T03:33:57.569307+0000","last_became_peered":"2023-08-25T03:33:57.569307+0000","last_unstale":"2023-08-25T03:35:29.169807+0000","last_undegraded":"2023-08-25T03:35:29.169807+0000","last_fullsized":"2023-08-25T03:35:29.169807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T10:28:54.310632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":52,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:28.927478+0000","last_change":"2023-08-25T03:33:57.886704+0000","last_active":"2023-08-25T03:35:28.927478+0000","last_peered":"2023-08-25T03:35:28.927478+0000","last_clean":"2023-08-25T03:35:28.927478+0000","last_became_active":"2023-08-25T03:33:57.886294+0000","last_became_peered":"2023-08-25T03:33:57.886294+0000","last_unstale":"2023-08-25T03:35:28.927478+0000","last_undegraded":"2023-08-25T03:35:28.927478+0000","last_fullsized":"2023-08-25T03:35:28.927478+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:29.173591+0000","last_change":"2023-08-25T03:33:57.570657+0000","last_active":"2023-08-25T03:35:29.173591+0000","last_peered":"2023-08-25T03:35:29.173591+0000","last_clean":"2023-08-25T03:35:29.173591+0000","last_became_active":"2023-08-25T03:33:57.570425+0000","last_became_peered":"2023-08-25T03:33:57.570425+0000","last_unstale":"2023-08-25T03:35:29.173591+0000","last_undegraded":"2023-08-25T03:35:29.173591+0000","last_fullsized":"2023-08-25T03:35:29.173591+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T15:21:31.978492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":201,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:29.165285+0000","last_change":"2023-08-25T03:33:51.259898+0000","last_active":"2023-08-25T03:35:29.165285+0000","last_peered":"2023-08-25T03:35:29.165285+0000","last_clean":"2023-08-25T03:35:29.165285+0000","last_became_active":"2023-08-25T03:33:51.258780+0000","last_became_peered":"2023-08-25T03:33:51.258780+0000","last_unstale":"2023-08-25T03:35:29.165285+0000","last_undegraded":"2023-08-25T03:35:29.165285+0000","last_fullsized":"2023-08-25T03:35:29.165285+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:49.666858+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:49.666858+0000","last_clean_scrub_stamp":"2023-08-25T03:33:49.666858+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:58:03.603885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:29.165223+0000","last_change":"2023-08-25T03:33:57.571821+0000","last_active":"2023-08-25T03:35:29.165223+0000","last_peered":"2023-08-25T03:35:29.165223+0000","last_clean":"2023-08-25T03:35:29.165223+0000","last_became_active":"2023-08-25T03:33:57.571528+0000","last_became_peered":"2023-08-25T03:33:57.571528+0000","last_unstale":"2023-08-25T03:35:29.165223+0000","last_undegraded":"2023-08-25T03:35:29.165223+0000","last_fullsized":"2023-08-25T03:35:29.165223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:11:29.277327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":45,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:29.166729+0000","last_change":"2023-08-25T03:33:57.571730+0000","last_active":"2023-08-25T03:35:29.166729+0000","last_peered":"2023-08-25T03:35:29.166729+0000","last_clean":"2023-08-25T03:35:29.166729+0000","last_became_active":"2023-08-25T03:33:57.571442+0000","last_became_peered":"2023-08-25T03:33:57.571442+0000","last_unstale":"2023-08-25T03:35:29.166729+0000","last_undegraded":"2023-08-25T03:35:29.166729+0000","last_fullsized":"2023-08-25T03:35:29.166729+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:20:03.803176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":46,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:28.923923+0000","last_change":"2023-08-25T03:33:57.570506+0000","last_active":"2023-08-25T03:35:28.923923+0000","last_peered":"2023-08-25T03:35:28.923923+0000","last_clean":"2023-08-25T03:35:28.923923+0000","last_became_active":"2023-08-25T03:33:57.570171+0000","last_became_peered":"2023-08-25T03:33:57.570171+0000","last_unstale":"2023-08-25T03:35:28.923923+0000","last_undegraded":"2023-08-25T03:35:28.923923+0000","last_fullsized":"2023-08-25T03:35:28.923923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:28.924694+0000","last_change":"2023-08-25T03:33:57.570598+0000","last_active":"2023-08-25T03:35:28.924694+0000","last_peered":"2023-08-25T03:35:28.924694+0000","last_clean":"2023-08-25T03:35:28.924694+0000","last_became_active":"2023-08-25T03:33:57.570215+0000","last_became_peered":"2023-08-25T03:33:57.570215+0000","last_unstale":"2023-08-25T03:35:28.924694+0000","last_undegraded":"2023-08-25T03:35:28.924694+0000","last_fullsized":"2023-08-25T03:35:28.924694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T14:37:39.062456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542170,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1056768,"data_stored":782347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.73299999999999998}]},{"osd":1,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.68300000000000005}]},{"osd":2,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.754}]},{"osd":3,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.69999999999999996}]},{"osd":4,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.66400000000000003}]},{"osd":5,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.624}]},{"osd":6,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.60799999999999998}]}]},{"osd":6,"up_from":14,"seq":60129542170,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":191979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":2.0489999999999999,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.42599999999999999}]},{"osd":1,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":4.5419999999999998,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":0.53200000000000003}]},{"osd":2,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":5.5179999999999998,"5min":5.5179999999999998,"15min":5.5179999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.49199999999999999}]},{"osd":3,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.58599999999999997}]},{"osd":4,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":6.4050000000000002,"5min":6.4050000000000002,"15min":6.4050000000000002},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.64800000000000002}]},{"osd":5,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":7.4029999999999996,"5min":7.4029999999999996,"15min":7.4029999999999996},"last":0.71399999999999997}]},{"osd":7,"last update":"Fri Aug 25 03:34:52 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.63600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":191979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":6.6120000000000001,"5min":6.6120000000000001,"15min":6.6120000000000001},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":3.3530000000000002,"5min":3.3530000000000002,"15min":3.3530000000000002},"last":0.39100000000000001}]},{"osd":2,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":9.0950000000000006,"5min":9.0950000000000006,"15min":9.0950000000000006},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.42999999999999999}]},{"osd":3,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":3.4820000000000002,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":0.313},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":6.1970000000000001,"5min":6.1970000000000001,"15min":6.1970000000000001},"last":0.46600000000000003}]},{"osd":4,"last update":"Fri Aug 25 03:34:49 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":6.4400000000000004,"5min":6.4400000000000004,"15min":6.4400000000000004},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":6.5860000000000003,"5min":6.5860000000000003,"15min":6.5860000000000003},"last":0.57099999999999995}]},{"osd":5,"last update":"Fri Aug 25 03:34:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":3.3900000000000001,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":6.399,"5min":6.399,"15min":6.399},"last":0.51900000000000002}]},{"osd":6,"last update":"Fri Aug 25 03:34:49 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.872,"15min":0.872},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":6.5229999999999997,"5min":6.5229999999999997,"15min":6.5229999999999997},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":3.4529999999999998,"5min":3.4529999999999998,"15min":3.4529999999999998},"last":0.53600000000000003}]},{"osd":7,"last update":"Fri Aug 25 03:34:49 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":6.5590000000000002,"5min":6.5590000000000002,"15min":6.5590000000000002},"last":0.629},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":3.4220000000000002,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.499}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343784,"statfs":{"total":96636764160,"available":96608034816,"internally_reserved":0,"allocated":1056768,"data_stored":782347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.20000000000000001,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"last":0.29799999999999999}]},{"osd":2,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"last":0.434},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.36899999999999999}]},{"osd":3,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.40300000000000002}]},{"osd":4,"last update":"Fri Aug 25 03:34:53 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.498}]},{"osd":5,"last update":"Fri Aug 25 03:34:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.51400000000000001}]},{"osd":6,"last update":"Fri Aug 25 03:34:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.60299999999999998}]},{"osd":7,"last update":"Fri Aug 25 03:34:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.62}]}]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":191979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.42099999999999999}]},{"osd":1,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.33500000000000002}]},{"osd":3,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.378}]},{"osd":4,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.66000000000000003}]},{"osd":5,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.624}]},{"osd":6,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.752}]},{"osd":7,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.64400000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":191979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.34100000000000003},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":12.923999999999999,"5min":12.923999999999999,"15min":12.923999999999999},"last":0.309}]},{"osd":1,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":2.6680000000000001,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.27700000000000002}]},{"osd":2,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.38200000000000001}]},{"osd":4,"last update":"Fri Aug 25 03:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.59999999999999998}]},{"osd":5,"last update":"Fri Aug 25 03:34:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":6.3520000000000003,"5min":6.3520000000000003,"15min":6.3520000000000003},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.69199999999999995}]},{"osd":6,"last update":"Fri Aug 25 03:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.9849999999999999,"5min":2.9849999999999999,"15min":2.9849999999999999},"last":0.61899999999999999}]},{"osd":7,"last update":"Fri Aug 25 03:34:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.607,"5min":1.607,"15min":1.607},"last":0.57799999999999996}]}]},{"osd":4,"up_from":14,"seq":60129542170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":191979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.58699999999999997}]},{"osd":1,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":8.327,"5min":8.327,"15min":8.327},"last":0.747},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.56299999999999994}]},{"osd":2,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.70499999999999996}]},{"osd":3,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.67200000000000004}]},{"osd":5,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.435}]},{"osd":6,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.496},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.54500000000000004}]},{"osd":7,"last update":"Fri Aug 25 03:34:50 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.60299999999999998}]}]},{"osd":5,"up_from":14,"seq":60129542169,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":191979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.44},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.36299999999999999}]},{"osd":1,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.56399999999999995}]},{"osd":2,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.52800000000000002}]},{"osd":3,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.50600000000000001}]},{"osd":4,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.54000000000000004}]},{"osd":6,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.57299999999999995}]},{"osd":7,"last update":"Fri Aug 25 03:34:48 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.55100000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-25T03:35:39.801 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-08-25T03:33:56.554337+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=3, tm_min=33, tm_sec=56, tm_wday=4, tm_yday=237, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=3, tm_min=35, tm_sec=34, tm_wday=4, tm_yday=237, tm_isdst=0) 2023-08-25T03:35:39.801 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-08-25T03:33:56.554337+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=3, tm_min=33, tm_sec=56, tm_wday=4, tm_yday=237, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=3, tm_min=35, tm_sec=34, tm_wday=4, tm_yday=237, tm_isdst=0) 2023-08-25T03:35:39.802 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-08-25T03:33:56.554337+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=3, tm_min=33, tm_sec=56, tm_wday=4, tm_yday=237, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=3, tm_min=35, tm_sec=34, tm_wday=4, tm_yday=237, tm_isdst=0) 2023-08-25T03:35:39.802 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-08-25T03:33:56.554337+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=3, tm_min=33, tm_sec=56, tm_wday=4, tm_yday=237, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=3, tm_min=35, tm_sec=34, tm_wday=4, tm_yday=237, tm_isdst=0) 2023-08-25T03:35:39.802 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-08-25T03:33:49.666858+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=3, tm_min=33, tm_sec=49, tm_wday=4, tm_yday=237, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=3, tm_min=35, tm_sec=34, tm_wday=4, tm_yday=237, tm_isdst=0) 2023-08-25T03:35:39.803 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-08-25T03:33:56.554337+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=3, tm_min=33, tm_sec=56, tm_wday=4, tm_yday=237, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=3, tm_min=35, tm_sec=34, tm_wday=4, tm_yday=237, tm_isdst=0) 2023-08-25T03:35:39.803 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-08-25T03:33:56.554337+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=3, tm_min=33, tm_sec=56, tm_wday=4, tm_yday=237, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=3, tm_min=35, tm_sec=34, tm_wday=4, tm_yday=237, tm_isdst=0) 2023-08-25T03:35:39.803 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-08-25T03:33:56.554337+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=3, tm_min=33, tm_sec=56, tm_wday=4, tm_yday=237, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=3, tm_min=35, tm_sec=34, tm_wday=4, tm_yday=237, tm_isdst=0) 2023-08-25T03:35:39.804 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-08-25T03:33:56.554337+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=3, tm_min=33, tm_sec=56, tm_wday=4, tm_yday=237, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=3, tm_min=35, tm_sec=34, tm_wday=4, tm_yday=237, tm_isdst=0) 2023-08-25T03:35:39.804 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-08-25T03:35:59.810 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T03:35:59.952 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.950+0000 7f3ddb57e700 1 Processor -- start 2023-08-25T03:35:59.953 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.950+0000 7f3ddb57e700 1 -- start start 2023-08-25T03:35:59.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.951+0000 7f3ddb57e700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3dd40cfcc0 0x7f3dd40b2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:59.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.951+0000 7f3ddb57e700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f3dd40cca90 0x7f3dd40d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:59.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.951+0000 7f3ddb57e700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dd40ccf00 con 0x7f3dd415ea20 2023-08-25T03:35:59.955 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.951+0000 7f3ddb57e700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dd40cbd10 con 0x7f3dd40cca90 2023-08-25T03:35:59.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.951+0000 7f3ddb57e700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd40cbfb0 con 0x7f3dd40cfcc0 2023-08-25T03:35:59.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.951+0000 7f3dd9b1b700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f3dd40cca90 0x7f3dd40d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:59.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.951+0000 7f3dd931a700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f3dd415ea20 0x7f3dd40cf5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:48994/0 (socket says 172.21.15.2:48994) 2023-08-25T03:35:59.956 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.951+0000 7f3dd9b1b700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f3dd40cca90 0x7f3dd40d1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34100/0 (socket says 172.21.15.2:34100) 2023-08-25T03:35:59.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.951+0000 7f3dd931a700 1 -- 172.21.15.2:0/4272176293 learned_addr learned my addr 172.21.15.2:0/4272176293 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:35:59.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.951+0000 7f3dd8b19700 1 --2- 172.21.15.2:0/4272176293 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3dd40cfcc0 0x7f3dd40b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:59.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.951+0000 7f3dd9b1b700 1 -- 172.21.15.2:0/4272176293 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3dd40cfcc0 msgr2=0x7f3dd40b2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:35:59.957 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.952+0000 7f3dd9b1b700 1 --2- 172.21.15.2:0/4272176293 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3dd40cfcc0 0x7f3dd40b2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:59.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.952+0000 7f3dd9b1b700 1 -- 172.21.15.2:0/4272176293 >> v1:172.21.15.2:6789/0 conn(0x7f3dd415ea20 legacy=0x7f3dd40cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:59.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.952+0000 7f3dcbfff700 1 -- 172.21.15.2:0/4272176293 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3010398547 0 0) 0x7f3dd40ccf00 con 0x7f3dd415ea20 2023-08-25T03:35:59.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.952+0000 7f3dd9b1b700 1 -- 172.21.15.2:0/4272176293 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd40cc290 con 0x7f3dd40cca90 2023-08-25T03:35:59.958 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.952+0000 7f3dd9b1b700 1 --2- 172.21.15.2:0/4272176293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3dd40cca90 0x7f3dd40d1a10 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc009d90 tx=0x7f3dcc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=589029d35f15fb6 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:59.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.952+0000 7f3dcbfff700 1 -- 172.21.15.2:0/4272176293 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc00e070 con 0x7f3dd40cca90 2023-08-25T03:35:59.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.952+0000 7f3dcbfff700 1 -- 172.21.15.2:0/4272176293 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dcc00bbd0 con 0x7f3dd40cca90 2023-08-25T03:35:59.959 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.952+0000 7f3dcbfff700 1 -- 172.21.15.2:0/4272176293 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc012430 con 0x7f3dd40cca90 2023-08-25T03:35:59.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.952+0000 7f3ddb57e700 1 -- 172.21.15.2:0/4272176293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3dd40cca90 msgr2=0x7f3dd40d1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:59.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.952+0000 7f3ddb57e700 1 --2- 172.21.15.2:0/4272176293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3dd40cca90 0x7f3dd40d1a10 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc009d90 tx=0x7f3dcc00b330 comp rx=0 tx=0).stop 2023-08-25T03:35:59.960 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.952+0000 7f3ddb57e700 1 -- 172.21.15.2:0/4272176293 shutdown_connections 2023-08-25T03:35:59.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.952+0000 7f3ddb57e700 1 --2- 172.21.15.2:0/4272176293 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3dd40cca90 0x7f3dd40d1a10 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:59.961 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.952+0000 7f3ddb57e700 1 --2- 172.21.15.2:0/4272176293 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3dd40cfcc0 0x7f3dd40b2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:59.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.952+0000 7f3ddb57e700 1 -- 172.21.15.2:0/4272176293 >> 172.21.15.2:0/4272176293 conn(0x7f3dd409a2b0 msgr2=0x7f3dd40d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:35:59.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.953+0000 7f3ddb57e700 1 -- 172.21.15.2:0/4272176293 shutdown_connections 2023-08-25T03:35:59.962 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.953+0000 7f3ddb57e700 1 -- 172.21.15.2:0/4272176293 wait complete. 2023-08-25T03:35:59.966 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.954+0000 7f3ddb57e700 1 Processor -- start 2023-08-25T03:35:59.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.954+0000 7f3ddb57e700 1 -- start start 2023-08-25T03:35:59.967 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.955+0000 7f3ddb57e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3dd40cca90 0x7f3dd406ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:59.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.955+0000 7f3ddb57e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dd40cfcc0 0x7f3dd4157880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:59.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.955+0000 7f3ddb57e700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3dd415ea20 0x7f3dd4158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:59.968 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.955+0000 7f3ddb57e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd406cfd0 con 0x7f3dd40cca90 2023-08-25T03:35:59.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.955+0000 7f3ddb57e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd40c52a0 con 0x7f3dd40cfcc0 2023-08-25T03:35:59.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.955+0000 7f3ddb57e700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd406b780 con 0x7f3dd415ea20 2023-08-25T03:35:59.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.955+0000 7f3dd931a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3dd40cca90 0x7f3dd406ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:59.969 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.955+0000 7f3dd931a700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3dd40cca90 0x7f3dd406ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34102/0 (socket says 172.21.15.2:34102) 2023-08-25T03:35:59.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.955+0000 7f3dd931a700 1 -- 172.21.15.2:0/2842895528 learned_addr learned my addr 172.21.15.2:0/2842895528 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:35:59.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.955+0000 7f3dd9b1b700 1 --2- 172.21.15.2:0/2842895528 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3dd415ea20 0x7f3dd4158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:59.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.955+0000 7f3dd8b19700 1 --2- 172.21.15.2:0/2842895528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dd40cfcc0 0x7f3dd4157880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:59.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.955+0000 7f3dd931a700 1 -- 172.21.15.2:0/2842895528 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3dd415ea20 msgr2=0x7f3dd4158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:59.970 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.955+0000 7f3dd931a700 1 --2- 172.21.15.2:0/2842895528 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3dd415ea20 0x7f3dd4158910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:59.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.955+0000 7f3dd931a700 1 -- 172.21.15.2:0/2842895528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dd40cfcc0 msgr2=0x7f3dd4157880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:35:59.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.955+0000 7f3dd931a700 1 --2- 172.21.15.2:0/2842895528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dd40cfcc0 0x7f3dd4157880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:35:59.971 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.955+0000 7f3dd931a700 1 -- 172.21.15.2:0/2842895528 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dcc009a70 con 0x7f3dd40cca90 2023-08-25T03:35:59.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.955+0000 7f3dd8b19700 1 --2- 172.21.15.2:0/2842895528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dd40cfcc0 0x7f3dd4157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-08-25T03:35:59.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.955+0000 7f3dd931a700 1 --2- 172.21.15.2:0/2842895528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3dd40cca90 0x7f3dd406ca90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400c550 tx=0x7f3dc400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:59.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.955+0000 7f3dc9ffb700 1 -- 172.21.15.2:0/2842895528 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4004c80 con 0x7f3dd40cca90 2023-08-25T03:35:59.972 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.955+0000 7f3dc9ffb700 1 -- 172.21.15.2:0/2842895528 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc4004e00 con 0x7f3dd40cca90 2023-08-25T03:35:59.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.955+0000 7f3ddb57e700 1 -- 172.21.15.2:0/2842895528 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd406ba60 con 0x7f3dd40cca90 2023-08-25T03:35:59.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.955+0000 7f3dc9ffb700 1 -- 172.21.15.2:0/2842895528 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4007310 con 0x7f3dd40cca90 2023-08-25T03:35:59.973 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.956+0000 7f3ddb57e700 1 -- 172.21.15.2:0/2842895528 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd406c020 con 0x7f3dd40cca90 2023-08-25T03:35:59.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.957+0000 7f3ddb57e700 1 -- 172.21.15.2:0/2842895528 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dd40746d0 con 0x7f3dd40cca90 2023-08-25T03:35:59.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.957+0000 7f3dc9ffb700 1 -- 172.21.15.2:0/2842895528 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f3dc4007470 con 0x7f3dd40cca90 2023-08-25T03:35:59.974 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.957+0000 7f3dc9ffb700 1 --2- 172.21.15.2:0/2842895528 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3db806a0e0 0x7f3db806c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:35:59.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.957+0000 7f3dc9ffb700 1 -- 172.21.15.2:0/2842895528 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f3dc4087e40 con 0x7f3dd40cca90 2023-08-25T03:35:59.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.957+0000 7f3dd8b19700 1 --2- 172.21.15.2:0/2842895528 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3db806a0e0 0x7f3db806c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:35:59.975 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.958+0000 7f3dd8b19700 1 --2- 172.21.15.2:0/2842895528 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3db806a0e0 0x7f3db806c5a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3dd406d380 tx=0x7f3dd0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:35:59.976 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:35:59.962+0000 7f3dc9ffb700 1 -- 172.21.15.2:0/2842895528 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f3dc40538c0 con 0x7f3dd40cca90 2023-08-25T03:36:00.159 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:00.157+0000 7f3ddb57e700 1 -- 172.21.15.2:0/2842895528 --> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3dd415ee00 con 0x7f3db806a0e0 2023-08-25T03:36:00.162 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:00.160+0000 7f3dc9ffb700 1 -- 172.21.15.2:0/2842895528 <== mgr.4108 v2:172.21.15.2:6832/3528779779 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72915 (secure 0 0 0) 0x7f3dd0009680 con 0x7f3db806a0e0 2023-08-25T03:36:00.164 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:36:00.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:00.162+0000 7f3ddb57e700 1 -- 172.21.15.2:0/2842895528 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3db806a0e0 msgr2=0x7f3db806c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:36:00.166 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:00.162+0000 7f3ddb57e700 1 --2- 172.21.15.2:0/2842895528 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3db806a0e0 0x7f3db806c5a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3dd406d380 tx=0x7f3dd0008040 comp rx=0 tx=0).stop 2023-08-25T03:36:00.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:00.162+0000 7f3ddb57e700 1 -- 172.21.15.2:0/2842895528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3dd40cca90 msgr2=0x7f3dd406ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:36:00.167 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:00.162+0000 7f3ddb57e700 1 --2- 172.21.15.2:0/2842895528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3dd40cca90 0x7f3dd406ca90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400c550 tx=0x7f3dc400c860 comp rx=0 tx=0).stop 2023-08-25T03:36:00.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:00.163+0000 7f3ddb57e700 1 -- 172.21.15.2:0/2842895528 shutdown_connections 2023-08-25T03:36:00.168 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:00.163+0000 7f3ddb57e700 1 --2- 172.21.15.2:0/2842895528 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f3db806a0e0 0x7f3db806c5a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:00.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:00.163+0000 7f3ddb57e700 1 --2- 172.21.15.2:0/2842895528 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f3dd40cca90 0x7f3dd406ca90 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:00.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:00.163+0000 7f3ddb57e700 1 --2- 172.21.15.2:0/2842895528 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3dd415ea20 0x7f3dd4158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:00.169 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:00.163+0000 7f3ddb57e700 1 --2- 172.21.15.2:0/2842895528 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3dd40cfcc0 0x7f3dd4157880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:00.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:00.163+0000 7f3ddb57e700 1 -- 172.21.15.2:0/2842895528 >> 172.21.15.2:0/2842895528 conn(0x7f3dd409a2b0 msgr2=0x7f3dd40d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:36:00.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:00.163+0000 7f3ddb57e700 1 -- 172.21.15.2:0/2842895528 shutdown_connections 2023-08-25T03:36:00.170 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:00.163+0000 7f3ddb57e700 1 -- 172.21.15.2:0/2842895528 wait complete. 2023-08-25T03:36:00.170 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-08-25T03:36:00.176 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2023-08-25T03:35:59.528799+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":220912,"kb_used_data":4776,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754753808,"statfs":{"total":773094113280,"available":772867899392,"internally_reserved":0,"allocated":4890624,"data_stored":2716568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002980"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":54,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:39.998542+0000","last_change":"2023-08-25T03:35:39.998587+0000","last_active":"2023-08-25T03:35:39.998542+0000","last_peered":"2023-08-25T03:35:39.998542+0000","last_clean":"2023-08-25T03:35:39.998542+0000","last_became_active":"2023-08-25T03:33:57.569882+0000","last_became_peered":"2023-08-25T03:33:57.569882+0000","last_unstale":"2023-08-25T03:35:39.998542+0000","last_undegraded":"2023-08-25T03:35:39.998542+0000","last_fullsized":"2023-08-25T03:35:39.998542+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:39.998432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:39.998432+0000","last_clean_scrub_stamp":"2023-08-25T03:35:39.998432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:13:38.564993+0000","scrub_duration":0.001391677,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":60,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:46.350995+0000","last_change":"2023-08-25T03:35:46.351065+0000","last_active":"2023-08-25T03:35:46.350995+0000","last_peered":"2023-08-25T03:35:46.350995+0000","last_clean":"2023-08-25T03:35:46.350995+0000","last_became_active":"2023-08-25T03:33:57.569307+0000","last_became_peered":"2023-08-25T03:33:57.569307+0000","last_unstale":"2023-08-25T03:35:46.350995+0000","last_undegraded":"2023-08-25T03:35:46.350995+0000","last_fullsized":"2023-08-25T03:35:46.350995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:46.350793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:46.350793+0000","last_clean_scrub_stamp":"2023-08-25T03:35:46.350793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:29:54.269403+0000","scrub_duration":0.0023654909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":60,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:38.906180+0000","last_change":"2023-08-25T03:35:38.906220+0000","last_active":"2023-08-25T03:35:38.906180+0000","last_peered":"2023-08-25T03:35:38.906180+0000","last_clean":"2023-08-25T03:35:38.906180+0000","last_became_active":"2023-08-25T03:33:57.886294+0000","last_became_peered":"2023-08-25T03:33:57.886294+0000","last_unstale":"2023-08-25T03:35:38.906180+0000","last_undegraded":"2023-08-25T03:35:38.906180+0000","last_fullsized":"2023-08-25T03:35:38.906180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:38.906096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:38.906096+0000","last_clean_scrub_stamp":"2023-08-25T03:35:38.906096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:43:27.444744+0000","scrub_duration":0.0014635690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":60,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:50.451744+0000","last_change":"2023-08-25T03:35:50.451798+0000","last_active":"2023-08-25T03:35:50.451744+0000","last_peered":"2023-08-25T03:35:50.451744+0000","last_clean":"2023-08-25T03:35:50.451744+0000","last_became_active":"2023-08-25T03:33:57.570425+0000","last_became_peered":"2023-08-25T03:33:57.570425+0000","last_unstale":"2023-08-25T03:35:50.451744+0000","last_undegraded":"2023-08-25T03:35:50.451744+0000","last_fullsized":"2023-08-25T03:35:50.451744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:50.451625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:50.451625+0000","last_clean_scrub_stamp":"2023-08-25T03:35:50.451625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:36:40.699603+0000","scrub_duration":0.0022454900000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"40'119","reported_seq":213,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:51.195659+0000","last_change":"2023-08-25T03:35:51.195659+0000","last_active":"2023-08-25T03:35:51.195659+0000","last_peered":"2023-08-25T03:35:51.195659+0000","last_clean":"2023-08-25T03:35:51.195659+0000","last_became_active":"2023-08-25T03:33:51.258780+0000","last_became_peered":"2023-08-25T03:33:51.258780+0000","last_unstale":"2023-08-25T03:35:51.195659+0000","last_undegraded":"2023-08-25T03:35:51.195659+0000","last_fullsized":"2023-08-25T03:35:51.195659+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'119","last_scrub_stamp":"2023-08-25T03:35:51.195426+0000","last_deep_scrub":"40'119","last_deep_scrub_stamp":"2023-08-25T03:35:51.195426+0000","last_clean_scrub_stamp":"2023-08-25T03:35:51.195426+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:29:59.114036+0000","scrub_duration":0.007008617,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":46,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:29.165283+0000","last_change":"2023-08-25T03:33:57.571821+0000","last_active":"2023-08-25T03:35:29.165283+0000","last_peered":"2023-08-25T03:35:29.165283+0000","last_clean":"2023-08-25T03:35:29.165283+0000","last_became_active":"2023-08-25T03:33:57.571528+0000","last_became_peered":"2023-08-25T03:33:57.571528+0000","last_unstale":"2023-08-25T03:35:29.165283+0000","last_undegraded":"2023-08-25T03:35:29.165283+0000","last_fullsized":"2023-08-25T03:35:29.165283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":46,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:29.166785+0000","last_change":"2023-08-25T03:33:57.571730+0000","last_active":"2023-08-25T03:35:29.166785+0000","last_peered":"2023-08-25T03:35:29.166785+0000","last_clean":"2023-08-25T03:35:29.166785+0000","last_became_active":"2023-08-25T03:33:57.571442+0000","last_became_peered":"2023-08-25T03:33:57.571442+0000","last_unstale":"2023-08-25T03:35:29.166785+0000","last_undegraded":"2023-08-25T03:35:29.166785+0000","last_fullsized":"2023-08-25T03:35:29.166785+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:33:56.554337+0000","last_clean_scrub_stamp":"2023-08-25T03:33:56.554337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":54,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:38.968796+0000","last_change":"2023-08-25T03:35:38.968834+0000","last_active":"2023-08-25T03:35:38.968796+0000","last_peered":"2023-08-25T03:35:38.968796+0000","last_clean":"2023-08-25T03:35:38.968796+0000","last_became_active":"2023-08-25T03:33:57.570171+0000","last_became_peered":"2023-08-25T03:33:57.570171+0000","last_unstale":"2023-08-25T03:35:38.968796+0000","last_undegraded":"2023-08-25T03:35:38.968796+0000","last_fullsized":"2023-08-25T03:35:38.968796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:38.968709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:38.968709+0000","last_clean_scrub_stamp":"2023-08-25T03:35:38.968709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:13:08.804331+0000","scrub_duration":0.0017777489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":60,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:39.707565+0000","last_change":"2023-08-25T03:35:39.707651+0000","last_active":"2023-08-25T03:35:39.707565+0000","last_peered":"2023-08-25T03:35:39.707565+0000","last_clean":"2023-08-25T03:35:39.707565+0000","last_became_active":"2023-08-25T03:33:57.570215+0000","last_became_peered":"2023-08-25T03:33:57.570215+0000","last_unstale":"2023-08-25T03:35:39.707565+0000","last_undegraded":"2023-08-25T03:35:39.707565+0000","last_fullsized":"2023-08-25T03:35:39.707565+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:39.707398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:39.707398+0000","last_clean_scrub_stamp":"2023-08-25T03:35:39.707398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:50:00.023015+0000","scrub_duration":0.0016770590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542174,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1056768,"data_stored":782347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.69199999999999995}]},{"osd":1,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.748},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.65200000000000002}]},{"osd":2,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.59899999999999998}]},{"osd":3,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.629},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.71099999999999997}]},{"osd":4,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.51700000000000002}]},{"osd":5,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.499},{"interface":"front","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.54000000000000004}]},{"osd":6,"last update":"Fri Aug 25 03:34:51 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.56200000000000006}]}]},{"osd":6,"up_from":14,"seq":60129542174,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":462848,"data_stored":191979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56699999999999995,"15min":0.56799999999999995},"min":{"1min":0.36699999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90400000000000003,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.67900000000000005}]},{"osd":1,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.96299999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.67200000000000004,"15min":0.68799999999999994},"min":{"1min":0.38300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.78500000000000003,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":0.71799999999999997}]},{"osd":2,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.75,"15min":0.77000000000000002},"min":{"1min":0.46800000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.98799999999999999,"5min":5.5179999999999998,"15min":5.5179999999999998},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.374,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.70599999999999996}]},{"osd":3,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.40899999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93799999999999994,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60999999999999999,"15min":0.61299999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86599999999999999,"5min":1.198,"15min":1.198},"last":0.621}]},{"osd":4,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.76400000000000001,"15min":0.79900000000000004},"min":{"1min":0.35799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79800000000000004,"5min":6.4050000000000002,"15min":6.4050000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.63900000000000001,"15min":0.64300000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89600000000000002,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.64000000000000001}]},{"osd":5,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61799999999999999,"15min":0.627},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86099999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.502},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.90300000000000002,"15min":0.94599999999999995},"min":{"1min":0.46300000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.84299999999999997,"5min":7.4029999999999996,"15min":7.4029999999999996},"last":0.66100000000000003}]},{"osd":7,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.67700000000000005,"15min":0.68600000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94599999999999995,"5min":1.111,"15min":1.111},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69299999999999995,"15min":0.70099999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97899999999999998,"5min":1.137,"15min":1.137},"last":0.69199999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":462848,"data_stored":191979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.58499999999999996,"15min":0.627},"min":{"1min":0.24199999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.52300000000000002,"5min":6.6120000000000001,"15min":6.6120000000000001},"last":0.308},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.57399999999999995,"15min":0.60799999999999998},"min":{"1min":0.27600000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.45100000000000001,"5min":3.3530000000000002,"15min":3.3530000000000002},"last":0.45000000000000001}]},{"osd":2,"last update":"Fri Aug 25 03:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.73299999999999998,"15min":0.79500000000000004},"min":{"1min":0.26800000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.48599999999999999,"5min":9.0950000000000006,"15min":9.0950000000000006},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.498,"15min":0.51700000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.56699999999999995,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Aug 25 03:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.55200000000000005,"15min":0.57999999999999996},"min":{"1min":0.29299999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.53000000000000003,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.68700000000000006,"15min":0.73499999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.55000000000000004,"5min":6.1970000000000001,"15min":6.1970000000000001},"last":0.501}]},{"osd":4,"last update":"Fri Aug 25 03:35:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.85199999999999998,"15min":0.88800000000000001},"min":{"1min":0.435,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79800000000000004,"5min":6.4400000000000004,"15min":6.4400000000000004},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.84999999999999998,"15min":0.88400000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.86699999999999999,"5min":6.5860000000000003,"15min":6.5860000000000003},"last":0.58899999999999997}]},{"osd":5,"last update":"Fri Aug 25 03:35:52 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.749,"15min":0.76900000000000002},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81299999999999994,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.83599999999999997,"15min":0.873},"min":{"1min":0.46000000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78200000000000003,"5min":6.399,"15min":6.399},"last":0.60799999999999998}]},{"osd":6,"last update":"Fri Aug 25 03:35:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.82099999999999995,"15min":0.85499999999999998},"min":{"1min":0.40799999999999997,"5min":0.371,"15min":0.371},"max":{"1min":0.80600000000000005,"5min":6.5229999999999997,"15min":6.5229999999999997},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.78600000000000003,"15min":0.81000000000000005},"min":{"1min":0.498,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78200000000000003,"5min":3.4529999999999998,"15min":3.4529999999999998},"last":0.56699999999999995}]},{"osd":7,"last update":"Fri Aug 25 03:35:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.83099999999999996,"15min":0.86399999999999999},"min":{"1min":0.46300000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83999999999999997,"5min":6.5590000000000002,"15min":6.5590000000000002},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.79900000000000004,"15min":0.82299999999999995},"min":{"1min":0.497,"5min":0.434,"15min":0.434},"max":{"1min":0.82499999999999996,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.66200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1056768,"data_stored":782347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 03:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33600000000000002,"15min":0.33300000000000002},"min":{"1min":0.25700000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.34300000000000003,"15min":0.33900000000000002},"min":{"1min":0.246,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.47699999999999998}]},{"osd":2,"last update":"Fri Aug 25 03:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.371,"15min":0.36499999999999999},"min":{"1min":0.28799999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.38,"5min":0.36599999999999999,"15min":0.36399999999999999},"min":{"1min":0.253,"5min":0.222,"15min":0.222},"max":{"1min":0.63700000000000001,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.504}]},{"osd":3,"last update":"Fri Aug 25 03:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.39500000000000002,"15min":0.39100000000000001},"min":{"1min":0.309,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.58699999999999997,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.32000000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.51000000000000001,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.54000000000000004}]},{"osd":4,"last update":"Fri Aug 25 03:35:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.46400000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.43099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.67200000000000004}]},{"osd":5,"last update":"Fri Aug 25 03:35:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.59899999999999998,"15min":0.59199999999999997},"min":{"1min":0.51000000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.85599999999999998}]},{"osd":6,"last update":"Fri Aug 25 03:35:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.48099999999999998,"5min":0.432,"15min":0.432},"max":{"1min":0.85399999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63700000000000001,"15min":0.63200000000000001},"min":{"1min":0.55000000000000004,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.89800000000000002}]},{"osd":7,"last update":"Fri Aug 25 03:35:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.60999999999999999,"15min":0.60199999999999998},"min":{"1min":0.49199999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.623,"15min":0.61499999999999999},"min":{"1min":0.54400000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.79800000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":462848,"data_stored":191979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.37,"5min":0.38,"15min":0.38200000000000001},"min":{"1min":0.23499999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.58799999999999997}]},{"osd":1,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.36299999999999999,"15min":0.36699999999999999},"min":{"1min":0.219,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.50700000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.40400000000000003,"15min":0.40600000000000003},"min":{"1min":0.29399999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59099999999999997,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.53100000000000003}]},{"osd":3,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.39500000000000002,"15min":0.39800000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.54600000000000004,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.42499999999999999,"15min":0.42899999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.52500000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.56699999999999995}]},{"osd":4,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.625,"15min":0.627},"min":{"1min":0.48299999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.82499999999999996,"5min":1.032,"15min":1.032},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.45900000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84899999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.90500000000000003}]},{"osd":5,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46800000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.48099999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90000000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.96799999999999997}]},{"osd":6,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.46899999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97499999999999998,"5min":1.125,"15min":1.125},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.66600000000000004,"15min":0.67200000000000004},"min":{"1min":0.45200000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.93000000000000005,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.98999999999999999}]},{"osd":7,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65800000000000003,"15min":0.66000000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.091,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.67200000000000004,"15min":0.67800000000000005},"min":{"1min":0.49199999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0509999999999999,"5min":1.091,"15min":1.091},"last":0.92600000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":191979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.40300000000000002,"15min":0.41599999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.52300000000000002,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.96299999999999997,"15min":1.0609999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.59699999999999998,"5min":12.923999999999999,"15min":12.923999999999999},"last":0.32100000000000001}]},{"osd":1,"last update":"Fri Aug 25 03:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.53100000000000003,"15min":0.55300000000000005},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60199999999999998,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.42199999999999999,"15min":0.433},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.56499999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.36299999999999999}]},{"osd":2,"last update":"Fri Aug 25 03:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.46000000000000002,"15min":0.46700000000000003},"min":{"1min":0.32700000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71399999999999997,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.47299999999999998,"15min":0.48499999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.56699999999999995,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.378}]},{"osd":4,"last update":"Fri Aug 25 03:35:54 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61499999999999999,"15min":0.61299999999999999},"min":{"1min":0.42799999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81799999999999995,"5min":1.016,"15min":1.016},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.623,"15min":0.63},"min":{"1min":0.39400000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.79600000000000004,"5min":1.234,"15min":1.234},"last":0.69199999999999995}]},{"osd":5,"last update":"Fri Aug 25 03:35:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":1.004,"15min":1.0700000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.78800000000000003,"5min":6.3520000000000003,"15min":6.3520000000000003},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.622,"15min":0.626},"min":{"1min":0.45300000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82299999999999995,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.71199999999999997}]},{"osd":6,"last update":"Fri Aug 25 03:35:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64400000000000002,"15min":0.65000000000000002},"min":{"1min":0.436,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92700000000000005,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.78600000000000003,"15min":0.81399999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.77600000000000002,"5min":2.9849999999999999,"15min":2.9849999999999999},"last":0.66300000000000003}]},{"osd":7,"last update":"Fri Aug 25 03:35:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.67500000000000004,"15min":0.68500000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.879,"5min":1.169,"15min":1.169},"last":0.621},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.69899999999999995,"15min":0.70899999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.95899999999999996,"5min":1.607,"15min":1.607},"last":0.63400000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542174,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":191979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.502,"5min":0.58599999999999997,"15min":0.59999999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.65700000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.53900000000000003}]},{"osd":1,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.85499999999999998,"15min":0.90900000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77600000000000002,"5min":8.327,"15min":8.327},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70499999999999996,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.438}]},{"osd":2,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.60599999999999998,"15min":0.61599999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.79600000000000004,"5min":1.123,"15min":1.123},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.58499999999999996,"15min":0.58099999999999996},"min":{"1min":0.434,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.66700000000000004}]},{"osd":3,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65200000000000002,"15min":0.65400000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.86799999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.67600000000000005,"15min":0.68500000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92700000000000005,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.56200000000000006}]},{"osd":5,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.376,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81999999999999995,"5min":1.214,"15min":1.214},"last":0.79600000000000004}]},{"osd":6,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.374,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65400000000000003,"15min":0.64800000000000002},"min":{"1min":0.46700000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.76800000000000002}]},{"osd":7,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63500000000000001,"15min":0.63400000000000001},"min":{"1min":0.40000000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84899999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.623},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.67500000000000004,"15min":0.67000000000000004},"min":{"1min":0.55500000000000005,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.95499999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.74399999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542173,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":462848,"data_stored":191979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:35:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.51400000000000001,"15min":0.50800000000000001},"min":{"1min":0.34499999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.497},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52700000000000002,"15min":0.53000000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71999999999999997,"5min":0.75,"15min":0.75},"last":0.42999999999999999}]},{"osd":1,"last update":"Fri Aug 25 03:35:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.42199999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.74399999999999999}]},{"osd":2,"last update":"Fri Aug 25 03:35:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.40699999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55200000000000005,"15min":0.54100000000000004},"min":{"1min":0.505,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.59399999999999997}]},{"osd":3,"last update":"Fri Aug 25 03:35:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.505,"5min":0.442,"15min":0.442},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.626}]},{"osd":4,"last update":"Fri Aug 25 03:35:50 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.38700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.44800000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.77400000000000002}]},{"osd":6,"last update":"Fri Aug 25 03:35:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.61799999999999999,"15min":0.61199999999999999},"min":{"1min":0.46200000000000002,"5min":0.318,"15min":0.318},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.624,"15min":0.61899999999999999},"min":{"1min":0.48799999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.78800000000000003}]},{"osd":7,"last update":"Fri Aug 25 03:35:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.66900000000000004,"15min":0.67200000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69099999999999995,"15min":0.68899999999999995},"min":{"1min":0.52000000000000002,"5min":0.374,"15min":0.374},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.67200000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-25T03:36:00.180 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-08-25T03:33:56.554337+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=3, tm_min=33, tm_sec=56, tm_wday=4, tm_yday=237, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=3, tm_min=35, tm_sec=34, tm_wday=4, tm_yday=237, tm_isdst=0) 2023-08-25T03:36:00.181 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-08-25T03:33:56.554337+0000 time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=3, tm_min=33, tm_sec=56, tm_wday=4, tm_yday=237, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=8, tm_mday=25, tm_hour=3, tm_min=35, tm_sec=34, tm_wday=4, tm_yday=237, tm_isdst=0) 2023-08-25T03:36:00.182 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-08-25T03:36:03.162 DEBUG:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T03:36:03.165 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T03:36:20.183 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-08-25T03:36:20.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.322+0000 7f9776425700 1 Processor -- start 2023-08-25T03:36:20.323 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.322+0000 7f9776425700 1 -- start start 2023-08-25T03:36:20.326 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.323+0000 7f9776425700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f977015ea20 0x7f97700b2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:36:20.327 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.323+0000 7f9776425700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f97700cfcc0 0x7f97700d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:36:20.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.323+0000 7f9776425700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97700ccf10 con 0x7f97700cca90 2023-08-25T03:36:20.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.323+0000 7f9776425700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97700cbd10 con 0x7f97700cfcc0 2023-08-25T03:36:20.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.323+0000 7f9776425700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97700cbfb0 con 0x7f977015ea20 2023-08-25T03:36:20.328 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.323+0000 7f976f7fe700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f97700cca90 0x7f97700cf5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:49008/0 (socket says 172.21.15.2:49008) 2023-08-25T03:36:20.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.323+0000 7f976f7fe700 1 -- 172.21.15.2:0/3720190194 learned_addr learned my addr 172.21.15.2:0/3720190194 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:36:20.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.323+0000 7f97749c2700 1 --2- 172.21.15.2:0/3720190194 >> v2:172.21.15.31:3300/0 conn(0x7f97700cfcc0 0x7f97700d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:36:20.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.323+0000 7f976ffff700 1 --2- 172.21.15.2:0/3720190194 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f977015ea20 0x7f97700b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:36:20.329 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.323+0000 7f976effd700 1 -- 172.21.15.2:0/3720190194 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 392847472 0 0) 0x7f9758001110 con 0x7f97700cca90 2023-08-25T03:36:20.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.323+0000 7f976effd700 1 -- 172.21.15.2:0/3720190194 --> v1:172.21.15.2:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f975c0035b0 con 0x7f97700cca90 2023-08-25T03:36:20.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.323+0000 7f976effd700 1 -- 172.21.15.2:0/3720190194 <== mon.0 v1:172.21.15.2:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3721676441 0 0) 0x7f975c0035b0 con 0x7f97700cca90 2023-08-25T03:36:20.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.324+0000 7f97749c2700 1 -- 172.21.15.2:0/3720190194 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f977015ea20 msgr2=0x7f97700b2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:36:20.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.324+0000 7f97749c2700 1 --2- 172.21.15.2:0/3720190194 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f977015ea20 0x7f97700b2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:20.330 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.324+0000 7f97749c2700 1 -- 172.21.15.2:0/3720190194 >> v1:172.21.15.2:6789/0 conn(0x7f97700cca90 legacy=0x7f97700cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:36:20.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.324+0000 7f97749c2700 1 -- 172.21.15.2:0/3720190194 --> v2:172.21.15.31:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97700cc290 con 0x7f97700cfcc0 2023-08-25T03:36:20.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.324+0000 7f976ffff700 1 --2- 172.21.15.2:0/3720190194 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f977015ea20 0x7f97700b2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-08-25T03:36:20.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.324+0000 7f97749c2700 1 --2- 172.21.15.2:0/3720190194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97700cfcc0 0x7f97700d1a10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9768009d90 tx=0x7f976800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8a46c6aa3053495 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:36:20.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.324+0000 7f976effd700 1 -- 172.21.15.2:0/3720190194 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976800e070 con 0x7f97700cfcc0 2023-08-25T03:36:20.331 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.324+0000 7f976effd700 1 -- 172.21.15.2:0/3720190194 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9768003a20 con 0x7f97700cfcc0 2023-08-25T03:36:20.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.324+0000 7f976effd700 1 -- 172.21.15.2:0/3720190194 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9758001800 con 0x7f97700cfcc0 2023-08-25T03:36:20.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.324+0000 7f9776425700 1 -- 172.21.15.2:0/3720190194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97700cfcc0 msgr2=0x7f97700d1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:36:20.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.325+0000 7f9776425700 1 --2- 172.21.15.2:0/3720190194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97700cfcc0 0x7f97700d1a10 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9768009d90 tx=0x7f976800b330 comp rx=0 tx=0).stop 2023-08-25T03:36:20.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.325+0000 7f9776425700 1 -- 172.21.15.2:0/3720190194 shutdown_connections 2023-08-25T03:36:20.332 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.325+0000 7f9776425700 1 --2- 172.21.15.2:0/3720190194 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97700cfcc0 0x7f97700d1a10 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:20.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.325+0000 7f9776425700 1 --2- 172.21.15.2:0/3720190194 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f977015ea20 0x7f97700b2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:20.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.325+0000 7f9776425700 1 -- 172.21.15.2:0/3720190194 >> 172.21.15.2:0/3720190194 conn(0x7f977009a2b0 msgr2=0x7f97700d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:36:20.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.325+0000 7f9776425700 1 -- 172.21.15.2:0/3720190194 shutdown_connections 2023-08-25T03:36:20.333 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.325+0000 7f9776425700 1 -- 172.21.15.2:0/3720190194 wait complete. 2023-08-25T03:36:20.334 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.326+0000 7f9776425700 1 Processor -- start 2023-08-25T03:36:20.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.326+0000 7f9776425700 1 -- start start 2023-08-25T03:36:20.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.327+0000 7f9776425700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97700cca90 0x7f9770158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:36:20.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.327+0000 7f9776425700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97700cfcc0 0x7f97700c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:36:20.336 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.327+0000 7f9776425700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f977015ea20 0x7f97700c6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:36:20.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.327+0000 7f9776425700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97700c70c0 con 0x7f97700cfcc0 2023-08-25T03:36:20.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.327+0000 7f9776425700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9770159430 con 0x7f97700cca90 2023-08-25T03:36:20.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.327+0000 7f9776425700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97700b42c0 con 0x7f977015ea20 2023-08-25T03:36:20.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.327+0000 7f976f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97700cfcc0 0x7f97700c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:36:20.337 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.327+0000 7f976f7fe700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97700cfcc0 0x7f97700c6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34116/0 (socket says 172.21.15.2:34116) 2023-08-25T03:36:20.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.327+0000 7f976f7fe700 1 -- 172.21.15.2:0/1830761226 learned_addr learned my addr 172.21.15.2:0/1830761226 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:36:20.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.327+0000 7f976ffff700 1 --2- 172.21.15.2:0/1830761226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97700cca90 0x7f9770158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:36:20.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.327+0000 7f97749c2700 1 --2- 172.21.15.2:0/1830761226 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f977015ea20 0x7f97700c6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:36:20.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.327+0000 7f976f7fe700 1 -- 172.21.15.2:0/1830761226 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f977015ea20 msgr2=0x7f97700c6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:36:20.338 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.327+0000 7f976f7fe700 1 --2- 172.21.15.2:0/1830761226 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f977015ea20 0x7f97700c6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:20.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.327+0000 7f976f7fe700 1 -- 172.21.15.2:0/1830761226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97700cca90 msgr2=0x7f9770158890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:36:20.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.327+0000 7f976f7fe700 1 --2- 172.21.15.2:0/1830761226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97700cca90 0x7f9770158890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:20.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.327+0000 7f976f7fe700 1 -- 172.21.15.2:0/1830761226 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97600094a0 con 0x7f97700cfcc0 2023-08-25T03:36:20.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.328+0000 7f976f7fe700 1 --2- 172.21.15.2:0/1830761226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97700cfcc0 0x7f97700c6500 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f975800c500 tx=0x7f975800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:36:20.339 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.328+0000 7f976cff9700 1 -- 172.21.15.2:0/1830761226 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9758004c80 con 0x7f97700cfcc0 2023-08-25T03:36:20.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.328+0000 7f976cff9700 1 -- 172.21.15.2:0/1830761226 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9758004de0 con 0x7f97700cfcc0 2023-08-25T03:36:20.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.328+0000 7f9776425700 1 -- 172.21.15.2:0/1830761226 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9768009a70 con 0x7f97700cfcc0 2023-08-25T03:36:20.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.328+0000 7f976cff9700 1 -- 172.21.15.2:0/1830761226 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97580072d0 con 0x7f97700cfcc0 2023-08-25T03:36:20.340 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.328+0000 7f9776425700 1 -- 172.21.15.2:0/1830761226 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97700b3c70 con 0x7f97700cfcc0 2023-08-25T03:36:20.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.329+0000 7f9776425700 1 -- 172.21.15.2:0/1830761226 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97701f4a20 con 0x7f97700cfcc0 2023-08-25T03:36:20.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.329+0000 7f976cff9700 1 -- 172.21.15.2:0/1830761226 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9758007430 con 0x7f97700cfcc0 2023-08-25T03:36:20.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.330+0000 7f976cff9700 1 --2- 172.21.15.2:0/1830761226 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f975006a0e0 0x7f975006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:36:20.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.330+0000 7f976cff9700 1 -- 172.21.15.2:0/1830761226 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9758087ec0 con 0x7f97700cfcc0 2023-08-25T03:36:20.341 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.330+0000 7f976ffff700 1 --2- 172.21.15.2:0/1830761226 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f975006a0e0 0x7f975006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:36:20.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.330+0000 7f976ffff700 1 --2- 172.21.15.2:0/1830761226 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f975006a0e0 0x7f975006c5a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9760009c60 tx=0x7f9760008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:36:20.342 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.334+0000 7f976cff9700 1 -- 172.21.15.2:0/1830761226 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f97580538b0 con 0x7f97700cfcc0 2023-08-25T03:36:20.531 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.530+0000 7f9776425700 1 -- 172.21.15.2:0/1830761226 --> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f977015ee00 con 0x7f975006a0e0 2023-08-25T03:36:20.533 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.533+0000 7f976cff9700 1 -- 172.21.15.2:0/1830761226 <== mgr.4108 v2:172.21.15.2:6832/3528779779 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73227 (secure 0 0 0) 0x7f9760009da0 con 0x7f975006a0e0 2023-08-25T03:36:20.534 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:36:20.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.535+0000 7f9776425700 1 -- 172.21.15.2:0/1830761226 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f975006a0e0 msgr2=0x7f975006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:36:20.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.535+0000 7f9776425700 1 --2- 172.21.15.2:0/1830761226 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f975006a0e0 0x7f975006c5a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f9760009c60 tx=0x7f9760008040 comp rx=0 tx=0).stop 2023-08-25T03:36:20.537 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.535+0000 7f9776425700 1 -- 172.21.15.2:0/1830761226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97700cfcc0 msgr2=0x7f97700c6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:36:20.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.535+0000 7f9776425700 1 --2- 172.21.15.2:0/1830761226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97700cfcc0 0x7f97700c6500 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f975800c500 tx=0x7f975800c8c0 comp rx=0 tx=0).stop 2023-08-25T03:36:20.538 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.535+0000 7f9776425700 1 -- 172.21.15.2:0/1830761226 shutdown_connections 2023-08-25T03:36:20.539 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.535+0000 7f9776425700 1 --2- 172.21.15.2:0/1830761226 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f977015ea20 0x7f97700c6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:20.539 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.535+0000 7f9776425700 1 --2- 172.21.15.2:0/1830761226 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f97700cca90 0x7f9770158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:20.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.535+0000 7f9776425700 1 --2- 172.21.15.2:0/1830761226 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f975006a0e0 0x7f975006c5a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:20.540 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.535+0000 7f9776425700 1 --2- 172.21.15.2:0/1830761226 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f97700cfcc0 0x7f97700c6500 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:20.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.535+0000 7f9776425700 1 -- 172.21.15.2:0/1830761226 >> 172.21.15.2:0/1830761226 conn(0x7f977009a2b0 msgr2=0x7f97700ca0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:36:20.541 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.535+0000 7f9776425700 1 -- 172.21.15.2:0/1830761226 shutdown_connections 2023-08-25T03:36:20.542 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.535+0000 7f9776425700 1 -- 172.21.15.2:0/1830761226 wait complete. 2023-08-25T03:36:20.542 INFO:teuthology.orchestra.run.smithi002.stderr:dumped all 2023-08-25T03:36:20.548 INFO:teuthology.orchestra.run.smithi002.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2023-08-25T03:36:19.533621+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":220912,"kb_used_data":4776,"kb_used_omap":13,"kb_used_meta":216050,"kb_avail":754753808,"statfs":{"total":773094113280,"available":772867899392,"internally_reserved":0,"allocated":4890624,"data_stored":2716568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":14248,"internal_metadata":221235288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002880"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":55,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:39.998587+0000","last_change":"2023-08-25T03:35:39.998587+0000","last_active":"2023-08-25T03:35:39.998587+0000","last_peered":"2023-08-25T03:35:39.998587+0000","last_clean":"2023-08-25T03:35:39.998587+0000","last_became_active":"2023-08-25T03:33:57.569882+0000","last_became_peered":"2023-08-25T03:33:57.569882+0000","last_unstale":"2023-08-25T03:35:39.998587+0000","last_undegraded":"2023-08-25T03:35:39.998587+0000","last_fullsized":"2023-08-25T03:35:39.998587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:39.998432+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:39.998432+0000","last_clean_scrub_stamp":"2023-08-25T03:35:39.998432+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:13:38.564993+0000","scrub_duration":0.001391677,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":61,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:46.351065+0000","last_change":"2023-08-25T03:35:46.351065+0000","last_active":"2023-08-25T03:35:46.351065+0000","last_peered":"2023-08-25T03:35:46.351065+0000","last_clean":"2023-08-25T03:35:46.351065+0000","last_became_active":"2023-08-25T03:33:57.569307+0000","last_became_peered":"2023-08-25T03:33:57.569307+0000","last_unstale":"2023-08-25T03:35:46.351065+0000","last_undegraded":"2023-08-25T03:35:46.351065+0000","last_fullsized":"2023-08-25T03:35:46.351065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:46.350793+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:46.350793+0000","last_clean_scrub_stamp":"2023-08-25T03:35:46.350793+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:29:54.269403+0000","scrub_duration":0.0023654909999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":61,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:38.906220+0000","last_change":"2023-08-25T03:35:38.906220+0000","last_active":"2023-08-25T03:35:38.906220+0000","last_peered":"2023-08-25T03:35:38.906220+0000","last_clean":"2023-08-25T03:35:38.906220+0000","last_became_active":"2023-08-25T03:33:57.886294+0000","last_became_peered":"2023-08-25T03:33:57.886294+0000","last_unstale":"2023-08-25T03:35:38.906220+0000","last_undegraded":"2023-08-25T03:35:38.906220+0000","last_fullsized":"2023-08-25T03:35:38.906220+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:38.906096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:38.906096+0000","last_clean_scrub_stamp":"2023-08-25T03:35:38.906096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:43:27.444744+0000","scrub_duration":0.0014635690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":61,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:50.451798+0000","last_change":"2023-08-25T03:35:50.451798+0000","last_active":"2023-08-25T03:35:50.451798+0000","last_peered":"2023-08-25T03:35:50.451798+0000","last_clean":"2023-08-25T03:35:50.451798+0000","last_became_active":"2023-08-25T03:33:57.570425+0000","last_became_peered":"2023-08-25T03:33:57.570425+0000","last_unstale":"2023-08-25T03:35:50.451798+0000","last_undegraded":"2023-08-25T03:35:50.451798+0000","last_fullsized":"2023-08-25T03:35:50.451798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:50.451625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:50.451625+0000","last_clean_scrub_stamp":"2023-08-25T03:35:50.451625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:36:40.699603+0000","scrub_duration":0.0022454900000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"40'119","reported_seq":213,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:51.195659+0000","last_change":"2023-08-25T03:35:51.195659+0000","last_active":"2023-08-25T03:35:51.195659+0000","last_peered":"2023-08-25T03:35:51.195659+0000","last_clean":"2023-08-25T03:35:51.195659+0000","last_became_active":"2023-08-25T03:33:51.258780+0000","last_became_peered":"2023-08-25T03:33:51.258780+0000","last_unstale":"2023-08-25T03:35:51.195659+0000","last_undegraded":"2023-08-25T03:35:51.195659+0000","last_fullsized":"2023-08-25T03:35:51.195659+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"40'119","last_scrub_stamp":"2023-08-25T03:35:51.195426+0000","last_deep_scrub":"40'119","last_deep_scrub_stamp":"2023-08-25T03:35:51.195426+0000","last_clean_scrub_stamp":"2023-08-25T03:35:51.195426+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T13:29:59.114036+0000","scrub_duration":0.007008617,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":55,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:55.190207+0000","last_change":"2023-08-25T03:35:55.190207+0000","last_active":"2023-08-25T03:35:55.190207+0000","last_peered":"2023-08-25T03:35:55.190207+0000","last_clean":"2023-08-25T03:35:55.190207+0000","last_became_active":"2023-08-25T03:33:57.571528+0000","last_became_peered":"2023-08-25T03:33:57.571528+0000","last_unstale":"2023-08-25T03:35:55.190207+0000","last_undegraded":"2023-08-25T03:35:55.190207+0000","last_fullsized":"2023-08-25T03:35:55.190207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:55.190029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:55.190029+0000","last_clean_scrub_stamp":"2023-08-25T03:35:55.190029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T09:36:45.438006+0000","scrub_duration":0.0019855720000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":55,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:56.214342+0000","last_change":"2023-08-25T03:35:56.214342+0000","last_active":"2023-08-25T03:35:56.214342+0000","last_peered":"2023-08-25T03:35:56.214342+0000","last_clean":"2023-08-25T03:35:56.214342+0000","last_became_active":"2023-08-25T03:33:57.571442+0000","last_became_peered":"2023-08-25T03:33:57.571442+0000","last_unstale":"2023-08-25T03:35:56.214342+0000","last_undegraded":"2023-08-25T03:35:56.214342+0000","last_fullsized":"2023-08-25T03:35:56.214342+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:56.214164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:56.214164+0000","last_clean_scrub_stamp":"2023-08-25T03:35:56.214164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T07:31:28.738118+0000","scrub_duration":0.0021802089999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":55,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:38.968834+0000","last_change":"2023-08-25T03:35:38.968834+0000","last_active":"2023-08-25T03:35:38.968834+0000","last_peered":"2023-08-25T03:35:38.968834+0000","last_clean":"2023-08-25T03:35:38.968834+0000","last_became_active":"2023-08-25T03:33:57.570171+0000","last_became_peered":"2023-08-25T03:33:57.570171+0000","last_unstale":"2023-08-25T03:35:38.968834+0000","last_undegraded":"2023-08-25T03:35:38.968834+0000","last_fullsized":"2023-08-25T03:35:38.968834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:38.968709+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:38.968709+0000","last_clean_scrub_stamp":"2023-08-25T03:35:38.968709+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T12:13:08.804331+0000","scrub_duration":0.0017777489999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":61,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-08-25T03:35:39.707651+0000","last_change":"2023-08-25T03:35:39.707651+0000","last_active":"2023-08-25T03:35:39.707651+0000","last_peered":"2023-08-25T03:35:39.707651+0000","last_clean":"2023-08-25T03:35:39.707651+0000","last_became_active":"2023-08-25T03:33:57.570215+0000","last_became_peered":"2023-08-25T03:33:57.570215+0000","last_unstale":"2023-08-25T03:35:39.707651+0000","last_undegraded":"2023-08-25T03:35:39.707651+0000","last_fullsized":"2023-08-25T03:35:39.707651+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-08-25T03:35:39.707398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-08-25T03:35:39.707398+0000","last_clean_scrub_stamp":"2023-08-25T03:35:39.707398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-08-26T06:50:00.023015+0000","scrub_duration":0.0016770590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542178,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1056768,"data_stored":782347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:35:54 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"min":{"1min":0.374,"5min":0.318,"15min":0.318},"max":{"1min":0.71999999999999997,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.55200000000000005,"15min":0.54800000000000004},"min":{"1min":0.32000000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.68999999999999995}]},{"osd":1,"last update":"Fri Aug 25 03:35:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.54700000000000004,"15min":0.53900000000000003},"min":{"1min":0.39200000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53400000000000003,"15min":0.52800000000000002},"min":{"1min":0.32800000000000001,"5min":0.311,"15min":0.311},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.77400000000000002}]},{"osd":2,"last update":"Fri Aug 25 03:35:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.55100000000000005,"15min":0.54900000000000004},"min":{"1min":0.40200000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61599999999999999,"15min":0.61899999999999999},"min":{"1min":0.40200000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.78900000000000003}]},{"osd":3,"last update":"Fri Aug 25 03:35:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.55400000000000005,"15min":0.55100000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.63700000000000001}]},{"osd":4,"last update":"Fri Aug 25 03:35:54 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78700000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.35099999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.52900000000000003}]},{"osd":5,"last update":"Fri Aug 25 03:35:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61499999999999999,"15min":0.61799999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.63100000000000001,"15min":0.629},"min":{"1min":0.47599999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.55800000000000005}]},{"osd":6,"last update":"Fri Aug 25 03:35:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.66000000000000003,"15min":0.65600000000000003},"min":{"1min":0.54500000000000004,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.65100000000000002,"15min":0.64800000000000002},"min":{"1min":0.433,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.45300000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542178,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":462848,"data_stored":191979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56699999999999995,"15min":0.56799999999999995},"min":{"1min":0.36699999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.90400000000000003,"5min":2.0489999999999999,"15min":2.0489999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57199999999999995,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.34799999999999998}]},{"osd":1,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.57799999999999996,"15min":0.58199999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.96299999999999997,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.67200000000000004,"15min":0.68799999999999994},"min":{"1min":0.38300000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.78500000000000003,"5min":4.5419999999999998,"15min":4.5419999999999998},"last":0.61899999999999999}]},{"osd":2,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.75,"15min":0.77000000000000002},"min":{"1min":0.46800000000000003,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.98799999999999999,"5min":5.5179999999999998,"15min":5.5179999999999998},"last":0.432},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.54400000000000004,"15min":0.54000000000000004},"min":{"1min":0.374,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.52400000000000002}]},{"osd":3,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.40899999999999997,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93799999999999994,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.60999999999999999,"15min":0.61299999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86599999999999999,"5min":1.198,"15min":1.198},"last":0.53900000000000003}]},{"osd":4,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.76400000000000001,"15min":0.79900000000000004},"min":{"1min":0.35799999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79800000000000004,"5min":6.4050000000000002,"15min":6.4050000000000002},"last":0.378},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.63900000000000001,"15min":0.64300000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89600000000000002,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.46999999999999997}]},{"osd":5,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61799999999999999,"15min":0.627},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.86099999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.90300000000000002,"15min":0.94599999999999995},"min":{"1min":0.46300000000000002,"5min":0.438,"15min":0.438},"max":{"1min":0.84299999999999997,"5min":7.4029999999999996,"15min":7.4029999999999996},"last":0.499}]},{"osd":7,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.67700000000000005,"15min":0.68600000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.94599999999999995,"5min":1.111,"15min":1.111},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.69299999999999995,"15min":0.70099999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.97899999999999998,"5min":1.137,"15min":1.137},"last":0.64700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":462848,"data_stored":191979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.58499999999999996,"15min":0.627},"min":{"1min":0.24199999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.52300000000000002,"5min":6.6120000000000001,"15min":6.6120000000000001},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.57399999999999995,"15min":0.60799999999999998},"min":{"1min":0.27600000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.45100000000000001,"5min":3.3530000000000002,"15min":3.3530000000000002},"last":0.44700000000000001}]},{"osd":2,"last update":"Fri Aug 25 03:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.73299999999999998,"15min":0.79500000000000004},"min":{"1min":0.26800000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.48599999999999999,"5min":9.0950000000000006,"15min":9.0950000000000006},"last":0.374},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.498,"15min":0.51700000000000002},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.56699999999999995,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.47899999999999998}]},{"osd":3,"last update":"Fri Aug 25 03:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.55200000000000005,"15min":0.57999999999999996},"min":{"1min":0.29299999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.53000000000000003,"5min":3.4820000000000002,"15min":3.4820000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.68700000000000006,"15min":0.73499999999999999},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.55000000000000004,"5min":6.1970000000000001,"15min":6.1970000000000001},"last":0.42399999999999999}]},{"osd":4,"last update":"Fri Aug 25 03:35:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.85199999999999998,"15min":0.88800000000000001},"min":{"1min":0.435,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.79800000000000004,"5min":6.4400000000000004,"15min":6.4400000000000004},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.84999999999999998,"15min":0.88400000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.86699999999999999,"5min":6.5860000000000003,"15min":6.5860000000000003},"last":0.79600000000000004}]},{"osd":5,"last update":"Fri Aug 25 03:35:52 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.749,"15min":0.76900000000000002},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81299999999999994,"5min":3.3900000000000001,"15min":3.3900000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.83599999999999997,"15min":0.873},"min":{"1min":0.46000000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.78200000000000003,"5min":6.399,"15min":6.399},"last":0.63800000000000001}]},{"osd":6,"last update":"Fri Aug 25 03:35:52 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.82099999999999995,"15min":0.85499999999999998},"min":{"1min":0.40799999999999997,"5min":0.371,"15min":0.371},"max":{"1min":0.80600000000000005,"5min":6.5229999999999997,"15min":6.5229999999999997},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.78600000000000003,"15min":0.81000000000000005},"min":{"1min":0.498,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.78200000000000003,"5min":3.4529999999999998,"15min":3.4529999999999998},"last":0.65300000000000002}]},{"osd":7,"last update":"Fri Aug 25 03:35:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.83099999999999996,"15min":0.86399999999999999},"min":{"1min":0.46300000000000002,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.83999999999999997,"5min":6.5590000000000002,"15min":6.5590000000000002},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.79900000000000004,"15min":0.82299999999999995},"min":{"1min":0.497,"5min":0.434,"15min":0.434},"max":{"1min":0.82499999999999996,"5min":3.4220000000000002,"15min":3.4220000000000002},"last":0.61099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343792,"statfs":{"total":96636764160,"available":96608043008,"internally_reserved":0,"allocated":1056768,"data_stored":782347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Aug 25 03:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33600000000000002,"15min":0.33300000000000002},"min":{"1min":0.25700000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.371,"5min":0.34300000000000003,"15min":0.33900000000000002},"min":{"1min":0.246,"5min":0.20000000000000001,"15min":0.20000000000000001},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.53800000000000003}]},{"osd":2,"last update":"Fri Aug 25 03:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.371,"15min":0.36499999999999999},"min":{"1min":0.28799999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.627},{"interface":"front","average":{"1min":0.38,"5min":0.36599999999999999,"15min":0.36399999999999999},"min":{"1min":0.253,"5min":0.222,"15min":0.222},"max":{"1min":0.63700000000000001,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.60699999999999998}]},{"osd":3,"last update":"Fri Aug 25 03:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.39500000000000002,"15min":0.39100000000000001},"min":{"1min":0.309,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.58699999999999997,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.32000000000000001,"5min":0.254,"15min":0.254},"max":{"1min":0.51000000000000001,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.67600000000000005}]},{"osd":4,"last update":"Fri Aug 25 03:35:56 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.46400000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.58699999999999997,"15min":0.57899999999999996},"min":{"1min":0.43099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.74399999999999999}]},{"osd":5,"last update":"Fri Aug 25 03:35:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.59599999999999997,"15min":0.58999999999999997},"min":{"1min":0.42399999999999999,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.59899999999999998,"15min":0.59199999999999997},"min":{"1min":0.51000000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.88100000000000001}]},{"osd":6,"last update":"Fri Aug 25 03:35:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.61899999999999999,"15min":0.61499999999999999},"min":{"1min":0.48099999999999998,"5min":0.432,"15min":0.432},"max":{"1min":0.85399999999999998,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63700000000000001,"15min":0.63200000000000001},"min":{"1min":0.55000000000000004,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.82999999999999996}]},{"osd":7,"last update":"Fri Aug 25 03:35:56 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.60999999999999999,"15min":0.60199999999999998},"min":{"1min":0.49199999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.623,"15min":0.61499999999999999},"min":{"1min":0.54400000000000004,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.79900000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":462848,"data_stored":191979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.37,"5min":0.38,"15min":0.38200000000000001},"min":{"1min":0.23499999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.39200000000000002}]},{"osd":1,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.36299999999999999,"15min":0.36699999999999999},"min":{"1min":0.219,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.50700000000000001,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.39600000000000002,"5min":0.40400000000000003,"15min":0.40600000000000003},"min":{"1min":0.29399999999999998,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59099999999999997,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.41999999999999998}]},{"osd":3,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.39500000000000002,"15min":0.39800000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.54600000000000004,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.42499999999999999,"15min":0.42899999999999999},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.52500000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.60199999999999998}]},{"osd":4,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.625,"15min":0.627},"min":{"1min":0.48299999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.82499999999999996,"5min":1.032,"15min":1.032},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.63400000000000001,"15min":0.63800000000000001},"min":{"1min":0.45900000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84899999999999998,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":1.0229999999999999}]},{"osd":5,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.46800000000000003,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.151,"5min":1.151,"15min":1.151},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.48099999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.90000000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.90600000000000003}]},{"osd":6,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.65100000000000002,"15min":0.65300000000000002},"min":{"1min":0.46899999999999997,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.97499999999999998,"5min":1.125,"15min":1.125},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.66600000000000004,"15min":0.67200000000000004},"min":{"1min":0.45200000000000001,"5min":0.443,"15min":0.443},"max":{"1min":0.93000000000000005,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.97199999999999998}]},{"osd":7,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65800000000000003,"15min":0.66000000000000003},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.091,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.67200000000000004,"15min":0.67800000000000005},"min":{"1min":0.49199999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0509999999999999,"5min":1.091,"15min":1.091},"last":0.92300000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":191979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.32400000000000001,"5min":0.40300000000000002,"15min":0.41599999999999998},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.52300000000000002,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.372,"5min":0.96299999999999997,"15min":1.0609999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.59699999999999998,"5min":12.923999999999999,"15min":12.923999999999999},"last":0.50900000000000001}]},{"osd":1,"last update":"Fri Aug 25 03:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.53100000000000003,"15min":0.55300000000000005},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60199999999999998,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.42199999999999999,"15min":0.433},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.56499999999999995,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.46800000000000003}]},{"osd":2,"last update":"Fri Aug 25 03:35:49 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.46000000000000002,"15min":0.46700000000000003},"min":{"1min":0.32700000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.71399999999999997,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.47299999999999998,"15min":0.48499999999999999},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.56699999999999995,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.57999999999999996}]},{"osd":4,"last update":"Fri Aug 25 03:35:54 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61499999999999999,"15min":0.61299999999999999},"min":{"1min":0.42799999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.81799999999999995,"5min":1.016,"15min":1.016},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.623,"15min":0.63},"min":{"1min":0.39400000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.79600000000000004,"5min":1.234,"15min":1.234},"last":1.0229999999999999}]},{"osd":5,"last update":"Fri Aug 25 03:35:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":1.004,"15min":1.0700000000000001},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.78800000000000003,"5min":6.3520000000000003,"15min":6.3520000000000003},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.622,"15min":0.626},"min":{"1min":0.45300000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.82299999999999995,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":1.0029999999999999}]},{"osd":6,"last update":"Fri Aug 25 03:35:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64400000000000002,"15min":0.65000000000000002},"min":{"1min":0.436,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.92700000000000005,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.78600000000000003,"15min":0.81399999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":0.77600000000000002,"5min":2.9849999999999999,"15min":2.9849999999999999},"last":0.90600000000000003}]},{"osd":7,"last update":"Fri Aug 25 03:35:54 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.67500000000000004,"15min":0.68500000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.879,"5min":1.169,"15min":1.169},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.69899999999999995,"15min":0.70899999999999996},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.95899999999999996,"5min":1.607,"15min":1.607},"last":0.93799999999999994}]}]},{"osd":4,"up_from":14,"seq":60129542178,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27472,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344368,"statfs":{"total":96636764160,"available":96608632832,"internally_reserved":0,"allocated":462848,"data_stored":191979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.502,"5min":0.58599999999999997,"15min":0.59999999999999998},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.65700000000000003,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.876}]},{"osd":1,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.85499999999999998,"15min":0.90900000000000003},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.77600000000000002,"5min":8.327,"15min":8.327},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56399999999999995,"15min":0.56999999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.70499999999999996,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.84299999999999997}]},{"osd":2,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.60599999999999998,"15min":0.61599999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.79600000000000004,"5min":1.123,"15min":1.123},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.58499999999999996,"15min":0.58099999999999996},"min":{"1min":0.434,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.69199999999999995}]},{"osd":3,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65200000000000002,"15min":0.65400000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.86799999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.67600000000000005,"15min":0.68500000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.92700000000000005,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.77700000000000002}]},{"osd":5,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57399999999999995,"15min":0.56999999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.59099999999999997,"15min":0.58899999999999997},"min":{"1min":0.376,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81999999999999995,"5min":1.214,"15min":1.214},"last":0.5}]},{"osd":6,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.374,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65400000000000003,"15min":0.64800000000000002},"min":{"1min":0.46700000000000003,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.54800000000000004}]},{"osd":7,"last update":"Fri Aug 25 03:35:53 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63500000000000001,"15min":0.63400000000000001},"min":{"1min":0.40000000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84899999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.67500000000000004,"15min":0.67000000000000004},"min":{"1min":0.55500000000000005,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.95499999999999996,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.52600000000000002}]}]},{"osd":5,"up_from":14,"seq":60129542177,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":452,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":462848,"data_stored":191979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1781,"internal_metadata":27654411},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Aug 25 03:35:50 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.51400000000000001,"15min":0.50800000000000001},"min":{"1min":0.34499999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52700000000000002,"15min":0.53000000000000003},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71999999999999997,"5min":0.75,"15min":0.75},"last":0.70299999999999996}]},{"osd":1,"last update":"Fri Aug 25 03:35:50 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52800000000000002,"15min":0.52400000000000002},"min":{"1min":0.42199999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.748}]},{"osd":2,"last update":"Fri Aug 25 03:35:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.55400000000000005,"15min":0.54600000000000004},"min":{"1min":0.40699999999999997,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.55200000000000005,"15min":0.54100000000000004},"min":{"1min":0.505,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.71799999999999997}]},{"osd":3,"last update":"Fri Aug 25 03:35:50 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57999999999999996,"15min":0.57599999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61799999999999999,"15min":0.61099999999999999},"min":{"1min":0.505,"5min":0.442,"15min":0.442},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.76600000000000001}]},{"osd":4,"last update":"Fri Aug 25 03:35:50 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.38700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":2.0150000000000001,"5min":2.0150000000000001,"15min":2.0150000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.44800000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.67600000000000005}]},{"osd":6,"last update":"Fri Aug 25 03:35:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.61799999999999999,"15min":0.61199999999999999},"min":{"1min":0.46200000000000002,"5min":0.318,"15min":0.318},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.624,"15min":0.61899999999999999},"min":{"1min":0.48799999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.81000000000000005}]},{"osd":7,"last update":"Fri Aug 25 03:35:50 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.66900000000000004,"15min":0.67200000000000004},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":1.4359999999999999,"5min":1.4359999999999999,"15min":1.4359999999999999},"last":0.63},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.69099999999999995,"15min":0.68899999999999995},"min":{"1min":0.52000000000000002,"5min":0.374,"15min":0.374},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.65000000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-08-25T03:36:20.551 DEBUG:teuthology.orchestra.run.smithi002:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-08-25T03:36:20.674 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.674+0000 7f78d4a2e700 1 Processor -- start 2023-08-25T03:36:20.675 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.674+0000 7f78d4a2e700 1 -- start start 2023-08-25T03:36:20.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.675+0000 7f78d4a2e700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f78d01449a0 0x7f78d006fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:36:20.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.675+0000 7f78d4a2e700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78d00ce7e0 0x7f78d00cb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:36:20.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.675+0000 7f78d4a2e700 1 -- --> v1:172.21.15.2:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d00cc8e0 con 0x7f78d006be40 2023-08-25T03:36:20.677 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.675+0000 7f78d4a2e700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78d00c9360 con 0x7f78d01449a0 2023-08-25T03:36:20.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.675+0000 7f78d4a2e700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d00c9640 con 0x7f78d00ce7e0 2023-08-25T03:36:20.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.675+0000 7f78ce59c700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f78d01449a0 0x7f78d006fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:36:20.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.675+0000 7f78ced9d700 1 --1- >> v1:172.21.15.2:6789/0 conn(0x7f78d006be40 0x7f78d00cd0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.2:6789/0 says I am v1:172.21.15.2:49020/0 (socket says 172.21.15.2:49020) 2023-08-25T03:36:20.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.675+0000 7f78ced9d700 1 -- 172.21.15.2:0/4058071135 learned_addr learned my addr 172.21.15.2:0/4058071135 (peer_addr_for_me v1:172.21.15.2:0/0) 2023-08-25T03:36:20.678 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.675+0000 7f78ce59c700 1 -- 172.21.15.2:0/4058071135 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78d00ce7e0 msgr2=0x7f78d00cb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:36:20.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.675+0000 7f78cdd9b700 1 --2- 172.21.15.2:0/4058071135 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78d00ce7e0 0x7f78d00cb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:36:20.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.675+0000 7f78ce59c700 1 --2- 172.21.15.2:0/4058071135 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78d00ce7e0 0x7f78d00cb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:20.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.675+0000 7f78cd59a700 1 -- 172.21.15.2:0/4058071135 <== mon.0 v1:172.21.15.2:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3573762799 0 0) 0x7f78d00cc8e0 con 0x7f78d006be40 2023-08-25T03:36:20.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.675+0000 7f78ce59c700 1 -- 172.21.15.2:0/4058071135 >> v1:172.21.15.2:6789/0 conn(0x7f78d006be40 legacy=0x7f78d00cd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:36:20.679 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.675+0000 7f78ce59c700 1 -- 172.21.15.2:0/4058071135 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d00ccb80 con 0x7f78d01449a0 2023-08-25T03:36:20.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.675+0000 7f78cdd9b700 1 --2- 172.21.15.2:0/4058071135 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78d00ce7e0 0x7f78d00cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:36:20.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.675+0000 7f78ce59c700 1 --2- 172.21.15.2:0/4058071135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78d01449a0 0x7f78d006fda0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009840 tx=0x7f78c4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e1388bf10d6b6c server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:36:20.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.676+0000 7f78cd59a700 1 -- 172.21.15.2:0/4058071135 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c4011830 con 0x7f78d01449a0 2023-08-25T03:36:20.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.676+0000 7f78cd59a700 1 -- 172.21.15.2:0/4058071135 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c4011990 con 0x7f78d01449a0 2023-08-25T03:36:20.680 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.676+0000 7f78cd59a700 1 -- 172.21.15.2:0/4058071135 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c4011d20 con 0x7f78d01449a0 2023-08-25T03:36:20.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.676+0000 7f78d4a2e700 1 -- 172.21.15.2:0/4058071135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78d01449a0 msgr2=0x7f78d006fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:36:20.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.676+0000 7f78d4a2e700 1 --2- 172.21.15.2:0/4058071135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78d01449a0 0x7f78d006fda0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009840 tx=0x7f78c4009b50 comp rx=0 tx=0).stop 2023-08-25T03:36:20.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.676+0000 7f78d4a2e700 1 -- 172.21.15.2:0/4058071135 shutdown_connections 2023-08-25T03:36:20.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.676+0000 7f78d4a2e700 1 --2- 172.21.15.2:0/4058071135 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78d01449a0 0x7f78d006fda0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:20.681 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.676+0000 7f78d4a2e700 1 --2- 172.21.15.2:0/4058071135 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78d00ce7e0 0x7f78d00cb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.676+0000 7f78d4a2e700 1 -- 172.21.15.2:0/4058071135 >> 172.21.15.2:0/4058071135 conn(0x7f78d00b1240 msgr2=0x7f78d006a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:36:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.676+0000 7f78d4a2e700 1 -- 172.21.15.2:0/4058071135 shutdown_connections 2023-08-25T03:36:20.682 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.676+0000 7f78d4a2e700 1 -- 172.21.15.2:0/4058071135 wait complete. 2023-08-25T03:36:20.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.678+0000 7f78d4a2e700 1 Processor -- start 2023-08-25T03:36:20.684 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.678+0000 7f78d4a2e700 1 -- start start 2023-08-25T03:36:20.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.678+0000 7f78d4a2e700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78d006be40 0x7f78d015ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:36:20.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.678+0000 7f78d4a2e700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d00ce7e0 0x7f78d01405c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:36:20.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.678+0000 7f78d4a2e700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78d01449a0 0x7f78d0143460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:36:20.685 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.678+0000 7f78d4a2e700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d0143040 con 0x7f78d01449a0 2023-08-25T03:36:20.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.678+0000 7f78d4a2e700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d015b060 con 0x7f78d00ce7e0 2023-08-25T03:36:20.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.678+0000 7f78d4a2e700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d01423c0 con 0x7f78d006be40 2023-08-25T03:36:20.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.678+0000 7f78ced9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78d01449a0 0x7f78d0143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:36:20.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.679+0000 7f78ced9d700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78d01449a0 0x7f78d0143460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34132/0 (socket says 172.21.15.2:34132) 2023-08-25T03:36:20.686 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.679+0000 7f78ced9d700 1 -- 172.21.15.2:0/3021931694 learned_addr learned my addr 172.21.15.2:0/3021931694 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:36:20.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.679+0000 7f78ce59c700 1 --2- 172.21.15.2:0/3021931694 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78d006be40 0x7f78d015ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:36:20.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.679+0000 7f78cdd9b700 1 --2- 172.21.15.2:0/3021931694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d00ce7e0 0x7f78d01405c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:36:20.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.679+0000 7f78ced9d700 1 -- 172.21.15.2:0/3021931694 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78d006be40 msgr2=0x7f78d015ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:36:20.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.679+0000 7f78ced9d700 1 --2- 172.21.15.2:0/3021931694 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78d006be40 0x7f78d015ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:20.687 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.679+0000 7f78ced9d700 1 -- 172.21.15.2:0/3021931694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d00ce7e0 msgr2=0x7f78d01405c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:36:20.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.679+0000 7f78ced9d700 1 --2- 172.21.15.2:0/3021931694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d00ce7e0 0x7f78d01405c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:20.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.679+0000 7f78ced9d700 1 -- 172.21.15.2:0/3021931694 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c40094a0 con 0x7f78d01449a0 2023-08-25T03:36:20.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.679+0000 7f78cdd9b700 1 --2- 172.21.15.2:0/3021931694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d00ce7e0 0x7f78d01405c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-08-25T03:36:20.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.679+0000 7f78ced9d700 1 --2- 172.21.15.2:0/3021931694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78d01449a0 0x7f78d0143460 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f78c8002230 tx=0x7f78c800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:36:20.688 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.679+0000 7f78beffd700 1 -- 172.21.15.2:0/3021931694 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c800ac20 con 0x7f78d01449a0 2023-08-25T03:36:20.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.679+0000 7f78beffd700 1 -- 172.21.15.2:0/3021931694 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c800ada0 con 0x7f78d01449a0 2023-08-25T03:36:20.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.679+0000 7f78d4a2e700 1 -- 172.21.15.2:0/3021931694 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d01426a0 con 0x7f78d01449a0 2023-08-25T03:36:20.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.679+0000 7f78beffd700 1 -- 172.21.15.2:0/3021931694 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c8007310 con 0x7f78d01449a0 2023-08-25T03:36:20.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.679+0000 7f78d4a2e700 1 -- 172.21.15.2:0/3021931694 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d0142c60 con 0x7f78d01449a0 2023-08-25T03:36:20.689 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.680+0000 7f78d4a2e700 1 -- 172.21.15.2:0/3021931694 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d00746d0 con 0x7f78d01449a0 2023-08-25T03:36:20.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.681+0000 7f78beffd700 1 -- 172.21.15.2:0/3021931694 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f78c8007470 con 0x7f78d01449a0 2023-08-25T03:36:20.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.681+0000 7f78beffd700 1 --2- 172.21.15.2:0/3021931694 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f78b006a0e0 0x7f78b006c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:36:20.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.681+0000 7f78beffd700 1 -- 172.21.15.2:0/3021931694 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f78c8087d20 con 0x7f78d01449a0 2023-08-25T03:36:20.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.681+0000 7f78ce59c700 1 --2- 172.21.15.2:0/3021931694 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f78b006a0e0 0x7f78b006c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:36:20.690 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.682+0000 7f78ce59c700 1 --2- 172.21.15.2:0/3021931694 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f78b006a0e0 0x7f78b006c5a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f78c40090b0 tx=0x7f78c401c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:36:20.691 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.686+0000 7f78beffd700 1 -- 172.21.15.2:0/3021931694 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f78c80537a0 con 0x7f78d01449a0 2023-08-25T03:36:20.885 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.883+0000 7f78d4a2e700 1 -- 172.21.15.2:0/3021931694 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f78d006d160 con 0x7f78d01449a0 2023-08-25T03:36:20.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.887+0000 7f78beffd700 1 -- 172.21.15.2:0/3021931694 <== mon.0 v2:172.21.15.2:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f78c808b030 con 0x7f78d01449a0 2023-08-25T03:36:20.888 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.888+0000 7f78beffd700 1 -- 172.21.15.2:0/3021931694 <== mon.0 v2:172.21.15.2:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v12)=0 v12) v1 ==== 123+0+0 (secure 0 0 0) 0x7f78c80180c0 con 0x7f78d01449a0 2023-08-25T03:36:20.891 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.890+0000 7f78d4a2e700 1 -- 172.21.15.2:0/3021931694 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f78b006a0e0 msgr2=0x7f78b006c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:36:20.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.890+0000 7f78d4a2e700 1 --2- 172.21.15.2:0/3021931694 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f78b006a0e0 0x7f78b006c5a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f78c40090b0 tx=0x7f78c401c040 comp rx=0 tx=0).stop 2023-08-25T03:36:20.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.890+0000 7f78d4a2e700 1 -- 172.21.15.2:0/3021931694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78d01449a0 msgr2=0x7f78d0143460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:36:20.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.890+0000 7f78d4a2e700 1 --2- 172.21.15.2:0/3021931694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78d01449a0 0x7f78d0143460 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f78c8002230 tx=0x7f78c800a060 comp rx=0 tx=0).stop 2023-08-25T03:36:20.892 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.890+0000 7f78d4a2e700 1 -- 172.21.15.2:0/3021931694 shutdown_connections 2023-08-25T03:36:20.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.890+0000 7f78d4a2e700 1 --2- 172.21.15.2:0/3021931694 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f78b006a0e0 0x7f78b006c5a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:20.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.890+0000 7f78d4a2e700 1 --2- 172.21.15.2:0/3021931694 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f78d01449a0 0x7f78d0143460 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:20.893 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.890+0000 7f78d4a2e700 1 --2- 172.21.15.2:0/3021931694 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78d006be40 0x7f78d015ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:20.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.890+0000 7f78d4a2e700 1 --2- 172.21.15.2:0/3021931694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78d00ce7e0 0x7f78d01405c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:20.894 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.890+0000 7f78d4a2e700 1 -- 172.21.15.2:0/3021931694 >> 172.21.15.2:0/3021931694 conn(0x7f78d00b1240 msgr2=0x7f78d0070370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:36:20.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.890+0000 7f78d4a2e700 1 -- 172.21.15.2:0/3021931694 shutdown_connections 2023-08-25T03:36:20.895 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:20.890+0000 7f78d4a2e700 1 -- 172.21.15.2:0/3021931694 wait complete. 2023-08-25T03:36:20.904 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-08-25T03:36:20.904 DEBUG:teuthology.orchestra.run.smithi002:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-08-25T03:36:21.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.033+0000 7f9a4d744700 1 Processor -- start 2023-08-25T03:36:21.034 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.033+0000 7f9a4d744700 1 -- start start 2023-08-25T03:36:21.037 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.034+0000 7f9a4d744700 1 --2- >> v2:172.21.15.31:3300/0 conn(0x7f9a480b23d0 0x7f9a480ce250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:36:21.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.034+0000 7f9a4d744700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f9a480ce790 0x7f9a480cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:36:21.038 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.034+0000 7f9a4d744700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9a480d00a0 0x7f9a480b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:36:21.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.034+0000 7f9a4d744700 1 -- --> v2:172.21.15.2:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a480cec20 con 0x7f9a480ce790 2023-08-25T03:36:21.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.034+0000 7f9a4d744700 1 -- --> v2:172.21.15.31:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a480d2360 con 0x7f9a480b23d0 2023-08-25T03:36:21.039 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.034+0000 7f9a4d744700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a480d2640 con 0x7f9a480d00a0 2023-08-25T03:36:21.040 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.034+0000 7f9a467fc700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f9a480ce790 0x7f9a480cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:36:21.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.034+0000 7f9a467fc700 1 --2- >> v2:172.21.15.2:3300/0 conn(0x7f9a480ce790 0x7f9a480cfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34136/0 (socket says 172.21.15.2:34136) 2023-08-25T03:36:21.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.034+0000 7f9a467fc700 1 -- 172.21.15.2:0/3858773603 learned_addr learned my addr 172.21.15.2:0/3858773603 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:36:21.041 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.034+0000 7f9a477fe700 1 --2- 172.21.15.2:0/3858773603 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9a480d00a0 0x7f9a480b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:36:21.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.034+0000 7f9a467fc700 1 -- 172.21.15.2:0/3858773603 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9a480d00a0 msgr2=0x7f9a480b4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:36:21.042 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.034+0000 7f9a46ffd700 1 --2- 172.21.15.2:0/3858773603 >> v2:172.21.15.31:3300/0 conn(0x7f9a480b23d0 0x7f9a480ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:36:21.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.034+0000 7f9a467fc700 1 --2- 172.21.15.2:0/3858773603 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9a480d00a0 0x7f9a480b4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:21.043 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.034+0000 7f9a467fc700 1 -- 172.21.15.2:0/3858773603 >> v2:172.21.15.31:3300/0 conn(0x7f9a480b23d0 msgr2=0x7f9a480ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-08-25T03:36:21.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.034+0000 7f9a467fc700 1 --2- 172.21.15.2:0/3858773603 >> v2:172.21.15.31:3300/0 conn(0x7f9a480b23d0 0x7f9a480ce250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:21.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.034+0000 7f9a467fc700 1 -- 172.21.15.2:0/3858773603 --> v2:172.21.15.2:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a480d2920 con 0x7f9a480ce790 2023-08-25T03:36:21.044 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.034+0000 7f9a467fc700 1 --2- 172.21.15.2:0/3858773603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a480ce790 0x7f9a480cfb60 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9a480c6c80 tx=0x7f9a38009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e9027a3da01f75 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:36:21.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.034+0000 7f9a45ffb700 1 -- 172.21.15.2:0/3858773603 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3800b870 con 0x7f9a480ce790 2023-08-25T03:36:21.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.035+0000 7f9a45ffb700 1 -- 172.21.15.2:0/3858773603 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9a3800b9d0 con 0x7f9a480ce790 2023-08-25T03:36:21.045 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.035+0000 7f9a45ffb700 1 -- 172.21.15.2:0/3858773603 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3800bd80 con 0x7f9a480ce790 2023-08-25T03:36:21.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.035+0000 7f9a4d744700 1 -- 172.21.15.2:0/3858773603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a480ce790 msgr2=0x7f9a480cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:36:21.046 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.035+0000 7f9a4d744700 1 --2- 172.21.15.2:0/3858773603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a480ce790 0x7f9a480cfb60 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9a480c6c80 tx=0x7f9a38009bd0 comp rx=0 tx=0).stop 2023-08-25T03:36:21.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.035+0000 7f9a4d744700 1 -- 172.21.15.2:0/3858773603 shutdown_connections 2023-08-25T03:36:21.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.035+0000 7f9a4d744700 1 --2- 172.21.15.2:0/3858773603 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9a480d00a0 0x7f9a480b4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:21.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.035+0000 7f9a4d744700 1 --2- 172.21.15.2:0/3858773603 >> v2:172.21.15.31:3300/0 conn(0x7f9a480b23d0 0x7f9a480ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:21.047 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.035+0000 7f9a4d744700 1 --2- 172.21.15.2:0/3858773603 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a480ce790 0x7f9a480cfb60 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:21.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.035+0000 7f9a4d744700 1 -- 172.21.15.2:0/3858773603 >> 172.21.15.2:0/3858773603 conn(0x7f9a4806ab90 msgr2=0x7f9a4806af70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:36:21.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.035+0000 7f9a4d744700 1 -- 172.21.15.2:0/3858773603 shutdown_connections 2023-08-25T03:36:21.048 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.035+0000 7f9a4d744700 1 -- 172.21.15.2:0/3858773603 wait complete. 2023-08-25T03:36:21.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.036+0000 7f9a4d744700 1 Processor -- start 2023-08-25T03:36:21.049 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.037+0000 7f9a4d744700 1 -- start start 2023-08-25T03:36:21.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.037+0000 7f9a4d744700 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9a480b23d0 0x7f9a481e6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:36:21.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.037+0000 7f9a4d744700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a480ce790 0x7f9a481e7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:36:21.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.037+0000 7f9a4d744700 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a480d00a0 0x7f9a480cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:36:21.052 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.037+0000 7f9a4d744700 1 -- --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a480c90b0 con 0x7f9a480ce790 2023-08-25T03:36:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.037+0000 7f9a4d744700 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a480c9390 con 0x7f9a480d00a0 2023-08-25T03:36:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.037+0000 7f9a4d744700 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a480c9670 con 0x7f9a480b23d0 2023-08-25T03:36:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.037+0000 7f9a467fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a480ce790 0x7f9a481e7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:36:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.037+0000 7f9a467fc700 1 --2- >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a480ce790 0x7f9a481e7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.2:3300/0 says I am v2:172.21.15.2:34144/0 (socket says 172.21.15.2:34144) 2023-08-25T03:36:21.053 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.037+0000 7f9a467fc700 1 -- 172.21.15.2:0/656460566 learned_addr learned my addr 172.21.15.2:0/656460566 (peer_addr_for_me v2:172.21.15.2:0/0) 2023-08-25T03:36:21.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.037+0000 7f9a46ffd700 1 --2- 172.21.15.2:0/656460566 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9a480b23d0 0x7f9a481e6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:36:21.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.038+0000 7f9a477fe700 1 --2- 172.21.15.2:0/656460566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a480d00a0 0x7f9a480cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:36:21.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.038+0000 7f9a467fc700 1 -- 172.21.15.2:0/656460566 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9a480b23d0 msgr2=0x7f9a481e6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:36:21.054 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.038+0000 7f9a467fc700 1 --2- 172.21.15.2:0/656460566 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9a480b23d0 0x7f9a481e6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.038+0000 7f9a467fc700 1 -- 172.21.15.2:0/656460566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a480d00a0 msgr2=0x7f9a480cc260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:36:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.038+0000 7f9a467fc700 1 --2- 172.21.15.2:0/656460566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a480d00a0 0x7f9a480cc260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.038+0000 7f9a467fc700 1 -- 172.21.15.2:0/656460566 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a38009550 con 0x7f9a480ce790 2023-08-25T03:36:21.055 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.038+0000 7f9a467fc700 1 --2- 172.21.15.2:0/656460566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a480ce790 0x7f9a481e7b80 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9a4806b900 tx=0x7f9a38004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:36:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.038+0000 7f9a2f7fe700 1 -- 172.21.15.2:0/656460566 <== mon.0 v2:172.21.15.2:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a38004310 con 0x7f9a480ce790 2023-08-25T03:36:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.038+0000 7f9a2f7fe700 1 -- 172.21.15.2:0/656460566 <== mon.0 v2:172.21.15.2:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9a3801f070 con 0x7f9a480ce790 2023-08-25T03:36:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.038+0000 7f9a2f7fe700 1 -- 172.21.15.2:0/656460566 <== mon.0 v2:172.21.15.2:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a380036a0 con 0x7f9a480ce790 2023-08-25T03:36:21.056 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.038+0000 7f9a4d744700 1 -- 172.21.15.2:0/656460566 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a480c9950 con 0x7f9a480ce790 2023-08-25T03:36:21.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.038+0000 7f9a4d744700 1 -- 172.21.15.2:0/656460566 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a481e7480 con 0x7f9a480ce790 2023-08-25T03:36:21.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.039+0000 7f9a4d744700 1 -- 172.21.15.2:0/656460566 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a480d40f0 con 0x7f9a480ce790 2023-08-25T03:36:21.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.040+0000 7f9a2f7fe700 1 -- 172.21.15.2:0/656460566 <== mon.0 v2:172.21.15.2:3300/0 4 ==== mgrmap(e 5) v1 ==== 88341+0+0 (secure 0 0 0) 0x7f9a38021030 con 0x7f9a480ce790 2023-08-25T03:36:21.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.040+0000 7f9a2f7fe700 1 --2- 172.21.15.2:0/656460566 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9a3406a0e0 0x7f9a3406c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-08-25T03:36:21.057 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.040+0000 7f9a2f7fe700 1 -- 172.21.15.2:0/656460566 <== mon.0 v2:172.21.15.2:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f9a380896d0 con 0x7f9a480ce790 2023-08-25T03:36:21.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.040+0000 7f9a46ffd700 1 --2- 172.21.15.2:0/656460566 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9a3406a0e0 0x7f9a3406c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-08-25T03:36:21.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.040+0000 7f9a46ffd700 1 --2- 172.21.15.2:0/656460566 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9a3406a0e0 0x7f9a3406c5a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00ab60 tx=0x7f9a3c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-08-25T03:36:21.058 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.044+0000 7f9a2f7fe700 1 -- 172.21.15.2:0/656460566 <== mon.0 v2:172.21.15.2:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178558 (secure 0 0 0) 0x7f9a38054fa0 con 0x7f9a480ce790 2023-08-25T03:36:21.291 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.291+0000 7f9a4d744700 1 -- 172.21.15.2:0/656460566 --> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a480746d0 con 0x7f9a480ce790 2023-08-25T03:36:21.292 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.291+0000 7f9a2f7fe700 1 -- 172.21.15.2:0/656460566 <== mon.0 v2:172.21.15.2:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3693 (secure 0 0 0) 0x7f9a38056ed0 con 0x7f9a480ce790 2023-08-25T03:36:21.293 INFO:teuthology.orchestra.run.smithi002.stdout: 2023-08-25T03:36:21.293 INFO:teuthology.orchestra.run.smithi002.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4751,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6834/3802947908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3802947908},{"type":"v1","addr":"172.21.15.31:6834","nonce":3802947908}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5214,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6837/1470640897","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6835","nonce":1470640897},{"type":"v1","addr":"172.21.15.2:6837","nonce":1470640897}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":5217,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.2:6836/3706466654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.2:6834","nonce":3706466654},{"type":"v1","addr":"172.21.15.2:6836","nonce":3706466654}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":5269,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1756382923","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6833","nonce":1756382923},{"type":"v1","addr":"172.21.15.31:6835","nonce":1756382923}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[]} 2023-08-25T03:36:21.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.293+0000 7f9a4d744700 1 -- 172.21.15.2:0/656460566 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9a3406a0e0 msgr2=0x7f9a3406c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:36:21.294 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.293+0000 7f9a4d744700 1 --2- 172.21.15.2:0/656460566 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9a3406a0e0 0x7f9a3406c5a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00ab60 tx=0x7f9a3c008040 comp rx=0 tx=0).stop 2023-08-25T03:36:21.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.293+0000 7f9a4d744700 1 -- 172.21.15.2:0/656460566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a480ce790 msgr2=0x7f9a481e7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-08-25T03:36:21.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.293+0000 7f9a4d744700 1 --2- 172.21.15.2:0/656460566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a480ce790 0x7f9a481e7b80 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9a4806b900 tx=0x7f9a38004750 comp rx=0 tx=0).stop 2023-08-25T03:36:21.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.293+0000 7f9a4d744700 1 -- 172.21.15.2:0/656460566 shutdown_connections 2023-08-25T03:36:21.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.293+0000 7f9a4d744700 1 --2- 172.21.15.2:0/656460566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9a480d00a0 0x7f9a480cc260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:21.295 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.293+0000 7f9a4d744700 1 --2- 172.21.15.2:0/656460566 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9a480b23d0 0x7f9a481e6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:21.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.293+0000 7f9a4d744700 1 --2- 172.21.15.2:0/656460566 >> [v2:172.21.15.2:6832/3528779779,v1:172.21.15.2:6833/3528779779] conn(0x7f9a3406a0e0 0x7f9a3406c5a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:21.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.293+0000 7f9a4d744700 1 --2- 172.21.15.2:0/656460566 >> [v2:172.21.15.2:3300/0,v1:172.21.15.2:6789/0] conn(0x7f9a480ce790 0x7f9a481e7b80 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-08-25T03:36:21.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.293+0000 7f9a4d744700 1 -- 172.21.15.2:0/656460566 >> 172.21.15.2:0/656460566 conn(0x7f9a4806ab90 msgr2=0x7f9a480b4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-08-25T03:36:21.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.293+0000 7f9a4d744700 1 -- 172.21.15.2:0/656460566 shutdown_connections 2023-08-25T03:36:21.296 INFO:teuthology.orchestra.run.smithi002.stderr:2023-08-25T03:36:21.293+0000 7f9a4d744700 1 -- 172.21.15.2:0/656460566 wait complete. 2023-08-25T03:36:21.297 INFO:teuthology.orchestra.run.smithi002.stderr:dumped fsmap epoch 19 2023-08-25T03:36:21.306 DEBUG:tasks.cephfs.filesystem:already dead... 2023-08-25T03:36:21.307 INFO:teuthology.misc:Shutting down mds daemons... 2023-08-25T03:36:21.307 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-08-25T03:36:21.308 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T03:36:21.316 INFO:tasks.ceph.mds.a:Stopped 2023-08-25T03:36:21.316 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-08-25T03:36:21.317 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T03:36:21.366 INFO:tasks.ceph.mds.c:Stopped 2023-08-25T03:36:21.366 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-08-25T03:36:21.367 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T03:36:21.378 INFO:tasks.ceph.mds.b:Stopped 2023-08-25T03:36:21.379 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-08-25T03:36:21.379 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T03:36:21.429 INFO:tasks.ceph.mds.d:Stopped 2023-08-25T03:36:21.430 INFO:teuthology.misc:Shutting down osd daemons... 2023-08-25T03:36:21.430 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-08-25T03:36:21.431 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T03:36:21.461 INFO:tasks.ceph.osd.0:Stopped 2023-08-25T03:36:21.461 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-08-25T03:36:21.462 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T03:36:21.531 INFO:tasks.ceph.osd.1:Stopped 2023-08-25T03:36:21.531 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-08-25T03:36:21.532 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T03:36:21.606 INFO:tasks.ceph.osd.2:Stopped 2023-08-25T03:36:21.607 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-08-25T03:36:21.607 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T03:36:21.675 INFO:tasks.ceph.osd.3:Stopped 2023-08-25T03:36:21.675 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-08-25T03:36:21.676 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T03:36:21.719 INFO:tasks.ceph.osd.4:Stopped 2023-08-25T03:36:21.720 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-08-25T03:36:21.720 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T03:36:21.759 INFO:tasks.ceph.osd.5:Stopped 2023-08-25T03:36:21.759 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-08-25T03:36:21.760 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T03:36:21.841 INFO:tasks.ceph.osd.6:Stopped 2023-08-25T03:36:21.842 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-08-25T03:36:21.842 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T03:36:21.872 INFO:tasks.ceph.osd.7:Stopped 2023-08-25T03:36:21.873 INFO:teuthology.misc:Shutting down mgr daemons... 2023-08-25T03:36:21.873 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-08-25T03:36:21.874 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T03:36:21.898 INFO:tasks.ceph.mgr.y:Stopped 2023-08-25T03:36:21.899 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-08-25T03:36:21.899 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T03:36:21.939 INFO:tasks.ceph.mgr.x:Stopped 2023-08-25T03:36:21.940 INFO:teuthology.misc:Shutting down mon daemons... 2023-08-25T03:36:21.940 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-08-25T03:36:21.941 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T03:36:21.951 INFO:tasks.ceph.mon.a:Stopped 2023-08-25T03:36:21.951 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-08-25T03:36:21.952 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T03:36:21.993 INFO:tasks.ceph.mon.b:Stopped 2023-08-25T03:36:21.993 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-08-25T03:36:21.994 INFO:teuthology.orchestra.run:waiting for 300 2023-08-25T03:36:22.045 INFO:tasks.ceph.mon.c:Stopped 2023-08-25T03:36:22.045 INFO:tasks.ceph:Checking cluster log for badness... 2023-08-25T03:36:22.046 DEBUG:teuthology.orchestra.run.smithi002:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-08-25T03:36:22.078 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi002.front.sepia.ceph.com 2023-08-25T03:36:22.079 DEBUG:teuthology.orchestra.run.smithi002:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-08-25T03:36:22.305 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi002.front.sepia.ceph.com 2023-08-25T03:36:22.305 DEBUG:teuthology.orchestra.run.smithi002:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-08-25T03:36:22.439 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi002.front.sepia.ceph.com 2023-08-25T03:36:22.440 DEBUG:teuthology.orchestra.run.smithi002:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-08-25T03:36:22.520 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi002.front.sepia.ceph.com 2023-08-25T03:36:22.520 DEBUG:teuthology.orchestra.run.smithi002:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-08-25T03:36:22.606 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi031.front.sepia.ceph.com 2023-08-25T03:36:22.607 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-08-25T03:36:22.787 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi031.front.sepia.ceph.com 2023-08-25T03:36:22.788 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-08-25T03:36:22.880 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi031.front.sepia.ceph.com 2023-08-25T03:36:22.880 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-08-25T03:36:22.947 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi031.front.sepia.ceph.com 2023-08-25T03:36:22.948 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-08-25T03:36:23.189 INFO:tasks.ceph:Archiving mon data... 2023-08-25T03:36:23.190 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379011/data/mon.a.tgz 2023-08-25T03:36:23.191 DEBUG:teuthology.orchestra.run.smithi002:> mktemp 2023-08-25T03:36:23.209 INFO:teuthology.orchestra.run.smithi002.stdout:/tmp/tmp.wjlHkMKSFS 2023-08-25T03:36:23.210 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.wjlHkMKSFS 2023-08-25T03:36:23.343 DEBUG:teuthology.orchestra.run.smithi002:> sudo chmod 0666 /tmp/tmp.wjlHkMKSFS 2023-08-25T03:36:23.434 DEBUG:teuthology.orchestra.remote:smithi002:/tmp/tmp.wjlHkMKSFS is 297KB 2023-08-25T03:36:23.595 DEBUG:teuthology.orchestra.run.smithi002:> rm -fr /tmp/tmp.wjlHkMKSFS 2023-08-25T03:36:23.614 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379011/data/mon.b.tgz 2023-08-25T03:36:23.615 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2023-08-25T03:36:23.636 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.7CQEeFA1Bi 2023-08-25T03:36:23.636 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.7CQEeFA1Bi 2023-08-25T03:36:23.802 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.7CQEeFA1Bi 2023-08-25T03:36:23.890 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.7CQEeFA1Bi is 395KB 2023-08-25T03:36:23.987 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.7CQEeFA1Bi 2023-08-25T03:36:24.005 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379011/data/mon.c.tgz 2023-08-25T03:36:24.005 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2023-08-25T03:36:24.061 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.H4xlvS34oD 2023-08-25T03:36:24.062 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.H4xlvS34oD 2023-08-25T03:36:24.203 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.H4xlvS34oD 2023-08-25T03:36:24.290 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.H4xlvS34oD is 395KB 2023-08-25T03:36:24.387 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.H4xlvS34oD 2023-08-25T03:36:24.405 INFO:tasks.ceph:Cleaning ceph cluster... 2023-08-25T03:36:24.406 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-08-25T03:36:24.408 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-08-25T03:36:24.473 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-08-25T03:36:24.473 DEBUG:teuthology.orchestra.run.smithi002:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-08-25T03:36:24.481 DEBUG:teuthology.orchestra.run.smithi031:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-08-25T03:36:24.522 INFO:teuthology.orchestra.run.smithi002.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-08-25T03:36:24.547 INFO:teuthology.orchestra.run.smithi031.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-08-25T03:36:24.553 INFO:tasks.ceph:Archiving crash dumps... 2023-08-25T03:36:24.558 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379011/remote/smithi002/crash 2023-08-25T03:36:24.559 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-08-25T03:36:24.589 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379011/remote/smithi031/crash 2023-08-25T03:36:24.591 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-08-25T03:36:24.619 INFO:tasks.ceph:Shutting down logrotate 2023-08-25T03:36:24.620 DEBUG:teuthology.orchestra.run.smithi002:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T03:36:24.635 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-08-25T03:36:24.708 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-08-25T03:36:24.737 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-08-25T03:36:24.765 INFO:tasks.ceph:Compressing logs... 2023-08-25T03:36:24.765 DEBUG:teuthology.orchestra.run.smithi002:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-08-25T03:36:24.780 DEBUG:teuthology.orchestra.run.smithi031:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-08-25T03:36:24.807 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121197.log 2023-08-25T03:36:24.807 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118537.log 2023-08-25T03:36:24.807 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 /var/log/ceph/ceph-client.admin.121197.log: --verbose -- /var/log/ceph/ceph-client.admin.114230.log 2023-08-25T03:36:24.808 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.118537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114967.log 2023-08-25T03:36:24.808 INFO:teuthology.orchestra.run.smithi002.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.121197.log.gz 2023-08-25T03:36:24.816 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.114230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110995.log 2023-08-25T03:36:24.817 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.114967.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.114230.log.gz 2023-08-25T03:36:24.817 INFO:teuthology.orchestra.run.smithi002.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.118537.log.gz 2023-08-25T03:36:24.817 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110146.log 2023-08-25T03:36:24.817 INFO:teuthology.orchestra.run.smithi002.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.114967.log.gz 2023-08-25T03:36:24.817 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.110995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118031.log 2023-08-25T03:36:24.818 INFO:teuthology.orchestra.run.smithi002.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.110995.log.gz 2023-08-25T03:36:24.818 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.110146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112218.log 2023-08-25T03:36:24.818 INFO:teuthology.orchestra.run.smithi002.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110146.log.gz 2023-08-25T03:36:24.818 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.118031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121704.log 2023-08-25T03:36:24.818 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.112218.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.118031.log.gz 2023-08-25T03:36:24.819 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118215.log 2023-08-25T03:36:24.819 INFO:teuthology.orchestra.run.smithi002.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112218.log.gz 2023-08-25T03:36:24.819 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.121704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119935.log 2023-08-25T03:36:24.819 INFO:teuthology.orchestra.run.smithi002.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.121704.log.gz 2023-08-25T03:36:24.819 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.118215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114599.log 2023-08-25T03:36:24.820 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.119935.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.118215.log.gz 2023-08-25T03:36:24.820 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120829.log 2023-08-25T03:36:24.820 INFO:teuthology.orchestra.run.smithi002.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119935.log.gz 2023-08-25T03:36:24.820 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.114599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109701.log 2023-08-25T03:36:24.821 INFO:teuthology.orchestra.run.smithi002.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.114599.log.gz 2023-08-25T03:36:24.821 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.120829.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-08-25T03:36:24.822 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.109701.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.120829.log.gz 2023-08-25T03:36:24.822 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103456.log 2023-08-25T03:36:24.822 INFO:teuthology.orchestra.run.smithi002.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.109701.log.gz 2023-08-25T03:36:24.822 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109061.log 2023-08-25T03:36:24.822 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.103456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114138.log 2023-08-25T03:36:24.823 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.109061.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.103456.log.gz 2023-08-25T03:36:24.823 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114369.log 2023-08-25T03:36:24.823 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109061.log.gz 2023-08-25T03:36:24.823 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.118844.log 2023-08-25T03:36:24.823 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.114138.log: /var/log/ceph/ceph-client.admin.114369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120165.log 2023-08-25T03:36:24.824 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114138.log.gz 2023-08-25T03:36:24.824 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.0.118844.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.114369.log.gz 2023-08-25T03:36:24.824 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121771.log 2023-08-25T03:36:24.824 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.118844.log.gz 2023-08-25T03:36:24.824 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.120165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121473.log 2023-08-25T03:36:24.825 INFO:teuthology.orchestra.run.smithi002.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.120165.log.gz 2023-08-25T03:36:24.825 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.121771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117499.log 2023-08-25T03:36:24.825 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.121473.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.121771.log.gz 2023-08-25T03:36:24.825 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-08-25T03:36:24.825 INFO:teuthology.orchestra.run.smithi002.stderr: 87.8%/var/log/ceph/ceph-client.admin.117499.log: -- replaced with /var/log/ceph/ceph-client.admin.121473.log.gz 2023-08-25T03:36:24.826 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117546.log 2023-08-25T03:36:24.826 INFO:teuthology.orchestra.run.smithi002.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.117499.log.gz 2023-08-25T03:36:24.826 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115370.log 2023-08-25T03:36:24.826 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.117546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111710.log 2023-08-25T03:36:24.826 INFO:teuthology.orchestra.run.smithi002.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.117546.log.gz 2023-08-25T03:36:24.827 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.115370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116293.log 2023-08-25T03:36:24.827 INFO:teuthology.orchestra.run.smithi002.stderr: 84.5% -- replaced with /var/log/ceph/ceph.log.gz 2023-08-25T03:36:24.827 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109282.log 2023-08-25T03:36:24.827 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.111710.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.115370.log.gz 2023-08-25T03:36:24.827 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.116293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120967.log 2023-08-25T03:36:24.827 INFO:teuthology.orchestra.run.smithi002.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111710.log.gz 2023-08-25T03:36:24.828 INFO:teuthology.orchestra.run.smithi002.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.116293.log.gz 2023-08-25T03:36:24.828 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114046.log 2023-08-25T03:36:24.828 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.109282.log: /var/log/ceph/ceph-client.admin.120967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117407.log 2023-08-25T03:36:24.828 INFO:teuthology.orchestra.run.smithi002.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.109282.log.gz 2023-08-25T03:36:24.828 INFO:teuthology.orchestra.run.smithi002.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.120967.log.gz 2023-08-25T03:36:24.829 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.114046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114921.log 2023-08-25T03:36:24.829 INFO:teuthology.orchestra.run.smithi002.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.114046.log.gz 2023-08-25T03:36:24.841 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-08-25T03:36:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.110101.log 2023-08-25T03:36:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103275.log 2023-08-25T03:36:24.842 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.1.110101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.110708.log 2023-08-25T03:36:24.842 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.110101.log.gz 2023-08-25T03:36:24.843 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.103275.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-08-25T03:36:24.843 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.3.110708.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.103275.log.gz 2023-08-25T03:36:24.843 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0%gzip -5 -- replaced with /var/log/ceph/ceph-client.3.110708.log.gz--verbose 2023-08-25T03:36:24.843 INFO:teuthology.orchestra.run.smithi031.stderr:-- /var/log/ceph/ceph-client.admin.103183.log 2023-08-25T03:36:24.843 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103463.log 2023-08-25T03:36:24.844 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.103183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103228.log 2023-08-25T03:36:24.844 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.103463.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.103183.log.gz 2023-08-25T03:36:24.844 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-08-25T03:36:24.844 INFO:teuthology.orchestra.run.smithi031.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.103463.log.gz 2023-08-25T03:36:24.844 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.103228.log: 91.5% -- replaced with /var/log/ceph/ceph.log.gz 2023-08-25T03:36:24.845 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103320.log 2023-08-25T03:36:24.845 INFO:teuthology.orchestra.run.smithi031.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103228.log.gz 2023-08-25T03:36:24.845 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.110093.log 2023-08-25T03:36:24.845 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.103320.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-08-25T03:36:24.845 INFO:teuthology.orchestra.run.smithi031.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.103320.log.gz 2023-08-25T03:36:24.845 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.1.110093.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-08-25T03:36:24.846 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.110093.log.gz 2023-08-25T03:36:24.846 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103410.log 2023-08-25T03:36:24.846 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-08-25T03:36:24.846 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.103410.log: /var/log/ceph/ceph-osd.4.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.103410.log.gz 2023-08-25T03:36:24.846 INFO:teuthology.orchestra.run.smithi031.stderr: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-08-25T03:36:24.848 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.111480.log 2023-08-25T03:36:24.848 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-08-25T03:36:24.849 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.1.111480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103136.log 2023-08-25T03:36:24.849 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.111480.log.gz 2023-08-25T03:36:24.849 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-08-25T03:36:24.849 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.103136.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-08-25T03:36:24.849 INFO:teuthology.orchestra.run.smithi031.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.103136.log.gz 2023-08-25T03:36:24.853 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103365.log 2023-08-25T03:36:24.853 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.7.log: /var/log/ceph/ceph-client.admin.103365.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.103365.log.gz 2023-08-25T03:36:24.853 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-08-25T03:36:24.858 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mon.b.log: 92.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-08-25T03:36:24.858 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.111484.log 2023-08-25T03:36:24.859 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.1.111484.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.111484.log.gz 2023-08-25T03:36:24.859 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-08-25T03:36:24.859 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mds.b.log: 91.9% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-08-25T03:36:24.859 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.110712.log 2023-08-25T03:36:24.859 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.3.110712.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.110712.log.gz 2023-08-25T03:36:24.861 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.117407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121013.log 2023-08-25T03:36:24.861 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.114921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114875.log 2023-08-25T03:36:24.861 INFO:teuthology.orchestra.run.smithi002.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117407.log.gz 2023-08-25T03:36:24.861 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.121013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114092.log 2023-08-25T03:36:24.862 INFO:teuthology.orchestra.run.smithi002.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.114921.log.gz 2023-08-25T03:36:24.862 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.114875.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.121013.log.gz 2023-08-25T03:36:24.862 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119456.log 2023-08-25T03:36:24.862 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.114092.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114875.log.gz 2023-08-25T03:36:24.862 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118123.log 2023-08-25T03:36:24.863 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.119456.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114092.log.gz 2023-08-25T03:36:24.863 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116983.log 2023-08-25T03:36:24.863 INFO:teuthology.orchestra.run.smithi002.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.119456.log.gz 2023-08-25T03:36:24.863 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.118123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116477.log 2023-08-25T03:36:24.863 INFO:teuthology.orchestra.run.smithi002.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118123.log.gz 2023-08-25T03:36:24.864 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.116983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117139.log 2023-08-25T03:36:24.864 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.116477.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.116983.log.gz 2023-08-25T03:36:24.864 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116661.log 2023-08-25T03:36:24.864 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.117139.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.116477.log.gz 2023-08-25T03:36:24.864 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108998.log 2023-08-25T03:36:24.865 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117139.log.gz 2023-08-25T03:36:24.865 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.116661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110988.log 2023-08-25T03:36:24.865 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.108998.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.116661.log.gz 2023-08-25T03:36:24.865 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116753.log 2023-08-25T03:36:24.865 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.110988.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108998.log.gz 2023-08-25T03:36:24.866 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-08-25T03:36:24.866 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.110988.log.gz 2023-08-25T03:36:24.866 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.116753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119276.log 2023-08-25T03:36:24.866 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-osd.0.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.116753.log.gz 2023-08-25T03:36:24.866 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109513.log 2023-08-25T03:36:24.867 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.119276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114829.log 2023-08-25T03:36:24.867 INFO:teuthology.orchestra.run.smithi002.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119276.log.gz 2023-08-25T03:36:24.867 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.109513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118169.log 2023-08-25T03:36:24.867 INFO:teuthology.orchestra.run.smithi002.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.109513.log.gz 2023-08-25T03:36:24.867 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.114829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103013.log 2023-08-25T03:36:24.868 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.118169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117935.log 2023-08-25T03:36:24.868 INFO:teuthology.orchestra.run.smithi002.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.114829.log.gz 2023-08-25T03:36:24.868 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.103013.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-08-25T03:36:24.868 INFO:teuthology.orchestra.run.smithi002.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118169.log.gz 2023-08-25T03:36:24.868 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.117935.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.103013.log.gz 2023-08-25T03:36:24.869 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114461.log 2023-08-25T03:36:24.869 INFO:teuthology.orchestra.run.smithi002.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.117935.log.gz 2023-08-25T03:36:24.869 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121335.log 2023-08-25T03:36:24.869 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121381.log 2023-08-25T03:36:24.869 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.114461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108902.log /var/log/ceph/ceph-client.admin.121335.log: 2023-08-25T03:36:24.869 INFO:teuthology.orchestra.run.smithi002.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.114461.log.gz 2023-08-25T03:36:24.870 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.121381.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.121335.log.gz 2023-08-25T03:36:24.870 INFO:teuthology.orchestra.run.smithi002.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.121381.log.gz 2023-08-25T03:36:24.870 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.108902.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108902.log.gz 2023-08-25T03:36:24.870 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121289.log 2023-08-25T03:36:24.870 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121243.log 2023-08-25T03:36:24.871 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.121289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111226.log 2023-08-25T03:36:24.871 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.121289.log.gz 2023-08-25T03:36:24.871 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.121243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119843.log 2023-08-25T03:36:24.871 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.111226.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.121243.log.gz 2023-08-25T03:36:24.871 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.119843.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111226.log.gz 2023-08-25T03:36:24.872 INFO:teuthology.orchestra.run.smithi002.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.119843.log.gz 2023-08-25T03:36:24.872 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121611.log 2023-08-25T03:36:24.872 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112032.log 2023-08-25T03:36:24.872 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.121611.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-08-25T03:36:24.873 INFO:teuthology.orchestra.run.smithi002.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.121611.log.gz 2023-08-25T03:36:24.873 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.112032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.113653.log 2023-08-25T03:36:24.873 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112078.log 2023-08-25T03:36:24.873 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-osd.3.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112032.log.gz 2023-08-25T03:36:24.873 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.2.113653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116021.log 2023-08-25T03:36:24.873 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.113653.log.gz 2023-08-25T03:36:24.874 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.112078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119686.log 2023-08-25T03:36:24.874 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.116021.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112078.log.gz 2023-08-25T03:36:24.874 INFO:teuthology.orchestra.run.smithi002.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.116021.log.gz 2023-08-25T03:36:24.874 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.119686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111940.log 2023-08-25T03:36:24.874 INFO:teuthology.orchestra.run.smithi002.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.119686.log.gz 2023-08-25T03:36:24.875 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110625.log 2023-08-25T03:36:24.875 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.111940.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111940.log.gz 2023-08-25T03:36:24.875 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.110625.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.110625.log.gz 2023-08-25T03:36:24.875 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118491.log 2023-08-25T03:36:24.875 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120119.log 2023-08-25T03:36:24.876 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.118491.log: /var/log/ceph/ceph-client.admin.120119.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.118491.log.gz 2023-08-25T03:36:24.876 INFO:teuthology.orchestra.run.smithi002.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.120119.log.gz 2023-08-25T03:36:24.876 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111336.log 2023-08-25T03:36:24.876 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109776.log 2023-08-25T03:36:24.876 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.111336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117322.log 2023-08-25T03:36:24.877 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.109776.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111336.log.gz 2023-08-25T03:36:24.877 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109776.log.gz 2023-08-25T03:36:24.877 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.117322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116205.log 2023-08-25T03:36:24.877 INFO:teuthology.orchestra.run.smithi002.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.117322.log.gz 2023-08-25T03:36:24.877 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115780.log 2023-08-25T03:36:24.878 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.116205.log: /var/log/ceph/ceph-client.admin.115780.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116205.log.gz 2023-08-25T03:36:24.878 INFO:teuthology.orchestra.run.smithi002.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115780.log.gz 2023-08-25T03:36:24.878 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116891.log 2023-08-25T03:36:24.878 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111338.log 2023-08-25T03:36:24.878 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.116891.log: /var/log/ceph/ceph-client.admin.111338.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116891.log.gz 2023-08-25T03:36:24.879 INFO:teuthology.orchestra.run.smithi002.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111338.log.gz 2023-08-25T03:36:24.879 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111357.log 2023-08-25T03:36:24.879 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120875.log 2023-08-25T03:36:24.879 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.111357.log: /var/log/ceph/ceph-client.admin.120875.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.111357.log.gz 2023-08-25T03:36:24.879 INFO:teuthology.orchestra.run.smithi002.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.120875.log.gz 2023-08-25T03:36:24.880 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111618.log 2023-08-25T03:36:24.880 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115013.log 2023-08-25T03:36:24.880 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.111618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109962.log 2023-08-25T03:36:24.880 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.115013.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.111618.log.gz 2023-08-25T03:36:24.880 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.109962.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115013.log.gz 2023-08-25T03:36:24.881 INFO:teuthology.orchestra.run.smithi002.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109962.log.gz 2023-08-25T03:36:24.881 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121059.log 2023-08-25T03:36:24.881 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110989.log 2023-08-25T03:36:24.882 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.121059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114277.log 2023-08-25T03:36:24.882 INFO:teuthology.orchestra.run.smithi002.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.121059.log.gz 2023-08-25T03:36:24.882 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.110989.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110989.log.gz 2023-08-25T03:36:24.882 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.114277.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114277.log.gz 2023-08-25T03:36:24.882 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120303.log 2023-08-25T03:36:24.883 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121861.log 2023-08-25T03:36:24.883 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.120303.log: /var/log/ceph/ceph-client.admin.121861.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.120303.log.gz 2023-08-25T03:36:24.883 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118399.log 2023-08-25T03:36:24.883 INFO:teuthology.orchestra.run.smithi002.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.121861.log.gz 2023-08-25T03:36:24.883 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-08-25T03:36:24.883 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.118399.log: /var/log/ceph/ceph-mds.c.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118399.log.gz 2023-08-25T03:36:24.884 INFO:teuthology.orchestra.run.smithi002.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-08-25T03:36:24.884 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119230.log 2023-08-25T03:36:24.884 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110629.log 2023-08-25T03:36:24.884 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.119230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109328.log 2023-08-25T03:36:24.884 INFO:teuthology.orchestra.run.smithi002.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.119230.log.gz 2023-08-25T03:36:24.885 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.110629.log: /var/log/ceph/ceph-client.admin.109328.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.110629.log.gz 2023-08-25T03:36:24.885 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110617.log 2023-08-25T03:36:24.885 INFO:teuthology.orchestra.run.smithi002.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.109328.log.gz 2023-08-25T03:36:24.885 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110422.log 2023-08-25T03:36:24.885 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120441.log 2023-08-25T03:36:24.886 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.110617.log: /var/log/ceph/ceph-client.admin.110422.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.110617.log.gz 2023-08-25T03:36:24.886 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110996.log 2023-08-25T03:36:24.886 INFO:teuthology.orchestra.run.smithi002.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110422.log.gz 2023-08-25T03:36:24.886 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.120441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113879.log 2023-08-25T03:36:24.886 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.110996.log: 87.5%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.120441.log.gz--verbose 2023-08-25T03:36:24.887 INFO:teuthology.orchestra.run.smithi002.stderr:-- /var/log/ceph/ceph-client.admin.109107.log 2023-08-25T03:36:24.887 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.113879.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110996.log.gz 2023-08-25T03:36:24.887 INFO:teuthology.orchestra.run.smithi002.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.113879.log.gz 2023-08-25T03:36:24.887 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.109107.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.109107.log.gz 2023-08-25T03:36:24.887 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111664.log 2023-08-25T03:36:24.888 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119889.log 2023-08-25T03:36:24.888 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.111664.log: /var/log/ceph/ceph-client.admin.119889.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111664.log.gz 2023-08-25T03:36:24.888 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119889.log.gz 2023-08-25T03:36:24.888 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120027.log 2023-08-25T03:36:24.888 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120257.log 2023-08-25T03:36:24.889 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.120027.log: /var/log/ceph/ceph-client.admin.120257.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.120027.log.gz 2023-08-25T03:36:24.889 INFO:teuthology.orchestra.run.smithi002.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.120257.log.gz 2023-08-25T03:36:24.889 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120921.log 2023-08-25T03:36:24.889 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114184.log 2023-08-25T03:36:24.889 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.120921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121815.log 2023-08-25T03:36:24.890 INFO:teuthology.orchestra.run.smithi002.stderr: 87.3%/var/log/ceph/ceph-client.admin.114184.log: -- replaced with /var/log/ceph/ceph-client.admin.120921.log.gz 2023-08-25T03:36:24.890 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.121815.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.114184.log.gz 2023-08-25T03:36:24.890 INFO:teuthology.orchestra.run.smithi002.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.121815.log.gz 2023-08-25T03:36:24.890 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114507.log 2023-08-25T03:36:24.890 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.113000.log 2023-08-25T03:36:24.890 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.114507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117751.log 2023-08-25T03:36:24.891 INFO:teuthology.orchestra.run.smithi002.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.114507.log.gz 2023-08-25T03:36:24.891 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.0.113000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110627.log 2023-08-25T03:36:24.891 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.113000.log.gz 2023-08-25T03:36:24.891 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.117751.log: /var/log/ceph/ceph-client.admin.110627.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.117751.log.gz 2023-08-25T03:36:24.891 INFO:teuthology.orchestra.run.smithi002.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110627.log.gz 2023-08-25T03:36:24.892 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117232.log 2023-08-25T03:36:24.892 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116339.log 2023-08-25T03:36:24.892 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.117232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112310.log 2023-08-25T03:36:24.892 INFO:teuthology.orchestra.run.smithi002.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117232.log.gz 2023-08-25T03:36:24.892 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.116339.log: 88.7%/var/log/ceph/ceph-client.admin.112310.log: -- replaced with /var/log/ceph/ceph-client.admin.116339.log.gz 2023-08-25T03:36:24.893 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112310.log.gz 2023-08-25T03:36:24.893 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109605.log 2023-08-25T03:36:24.893 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119502.log 2023-08-25T03:36:24.893 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.109605.log: /var/log/ceph/ceph-client.admin.119502.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109605.log.gz 2023-08-25T03:36:24.894 INFO:teuthology.orchestra.run.smithi002.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.119502.log.gz 2023-08-25T03:36:24.894 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110008.log 2023-08-25T03:36:24.894 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111365.log 2023-08-25T03:36:24.894 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.110008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119640.log 2023-08-25T03:36:24.894 INFO:teuthology.orchestra.run.smithi002.stderr: 89.4%/var/log/ceph/ceph-client.admin.111365.log: -- replaced with /var/log/ceph/ceph-client.admin.110008.log.gz 2023-08-25T03:36:24.894 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.119640.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111365.log.gz 2023-08-25T03:36:24.895 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119640.log.gz 2023-08-25T03:36:24.895 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116523.log 2023-08-25T03:36:24.895 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-08-25T03:36:24.895 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.116523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111848.log 2023-08-25T03:36:24.895 INFO:teuthology.orchestra.run.smithi002.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.116523.log.gz 2023-08-25T03:36:24.896 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.111848.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111848.log.gz 2023-08-25T03:36:24.896 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115872.log 2023-08-25T03:36:24.896 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116159.log 2023-08-25T03:36:24.896 INFO:teuthology.orchestra.run.smithi002.stderr: 88.9%/var/log/ceph/ceph-client.admin.115872.log: -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-08-25T03:36:24.896 INFO:teuthology.orchestra.run.smithi002.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.115872.log.gz 2023-08-25T03:36:24.897 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.116159.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116159.log.gz 2023-08-25T03:36:24.897 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119548.log 2023-08-25T03:36:24.897 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117659.log 2023-08-25T03:36:24.897 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.119548.log: /var/log/ceph/ceph-client.admin.117659.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119548.log.gz 2023-08-25T03:36:24.897 INFO:teuthology.orchestra.run.smithi002.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.117659.log.gz 2023-08-25T03:36:24.898 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110284.log 2023-08-25T03:36:24.898 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114645.log 2023-08-25T03:36:24.899 INFO:teuthology.orchestra.run.smithi031.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-08-25T03:36:24.921 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.110284.log: /var/log/ceph/ceph-client.admin.114645.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.110284.log.gz 2023-08-25T03:36:24.922 INFO:teuthology.orchestra.run.smithi002.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.114645.log.gz 2023-08-25T03:36:24.922 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110622.log 2023-08-25T03:36:24.922 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110628.log 2023-08-25T03:36:24.923 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.110622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113979.log 2023-08-25T03:36:24.923 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.110628.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110622.log.gz 2023-08-25T03:36:24.923 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.113979.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110628.log.gz 2023-08-25T03:36:24.923 INFO:teuthology.orchestra.run.smithi002.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.113979.log.gzgzip 2023-08-25T03:36:24.923 INFO:teuthology.orchestra.run.smithi002.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.117705.log 2023-08-25T03:36:24.924 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116569.log 2023-08-25T03:36:24.924 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.117705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120349.log 2023-08-25T03:36:24.924 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.116569.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.117705.log.gz 2023-08-25T03:36:24.924 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.120349.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.116569.log.gz 2023-08-25T03:36:24.924 INFO:teuthology.orchestra.run.smithi002.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.120349.log.gz 2023-08-25T03:36:24.925 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114691.log 2023-08-25T03:36:24.925 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121519.log 2023-08-25T03:36:24.925 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.114691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109868.log 2023-08-25T03:36:24.925 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.121519.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114691.log.gz 2023-08-25T03:36:24.925 INFO:teuthology.orchestra.run.smithi002.stderr: 87.8%/var/log/ceph/ceph-client.admin.109868.log: -- replaced with /var/log/ceph/ceph-client.admin.121519.log.gz 2023-08-25T03:36:24.926 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114323.log 2023-08-25T03:36:24.926 INFO:teuthology.orchestra.run.smithi002.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.109868.log.gz 2023-08-25T03:36:24.926 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.114323.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.114323.log.gz 2023-08-25T03:36:24.926 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110986.log 2023-08-25T03:36:24.926 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121151.log 2023-08-25T03:36:24.927 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.110986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115324.log 2023-08-25T03:36:24.927 INFO:teuthology.orchestra.run.smithi002.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.110986.log.gz/var/log/ceph/ceph-client.admin.121151.log: 2023-08-25T03:36:24.927 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.115324.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.121151.log.gz 2023-08-25T03:36:24.927 INFO:teuthology.orchestra.run.smithi002.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115324.log.gz 2023-08-25T03:36:24.927 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111331.log 2023-08-25T03:36:24.928 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117843.log 2023-08-25T03:36:24.928 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.111331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115151.log 2023-08-25T03:36:24.928 INFO:teuthology.orchestra.run.smithi002.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111331.log.gz 2023-08-25T03:36:24.928 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.117843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109916.log 2023-08-25T03:36:24.928 INFO:teuthology.orchestra.run.smithi002.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.117843.log.gz 2023-08-25T03:36:24.929 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.115151.log: /var/log/ceph/ceph-client.admin.109916.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115151.log.gz 2023-08-25T03:36:24.929 INFO:teuthology.orchestra.run.smithi002.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109916.log.gz 2023-08-25T03:36:24.929 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119138.log 2023-08-25T03:36:24.929 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.113657.log 2023-08-25T03:36:24.929 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.119138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112264.log 2023-08-25T03:36:24.930 INFO:teuthology.orchestra.run.smithi002.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.119138.log.gz 2023-08-25T03:36:24.930 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.2.113657.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.113657.log.gz 2023-08-25T03:36:24.930 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.112264.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112264.log.gz 2023-08-25T03:36:24.930 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113925.log 2023-08-25T03:36:24.930 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110983.log 2023-08-25T03:36:24.931 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.113925.log: /var/log/ceph/ceph-client.admin.110983.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.113925.log.gz 2023-08-25T03:36:24.931 INFO:teuthology.orchestra.run.smithi002.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.110983.log.gz 2023-08-25T03:36:24.931 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110192.log 2023-08-25T03:36:24.931 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115416.log 2023-08-25T03:36:24.931 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.110192.log: /var/log/ceph/ceph-client.admin.115416.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110192.log.gz 2023-08-25T03:36:24.932 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115416.log.gz 2023-08-25T03:36:24.932 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110990.log 2023-08-25T03:36:24.932 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117453.log 2023-08-25T03:36:24.932 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.110990.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110990.log.gz 2023-08-25T03:36:24.932 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.117453.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.117453.log.gz 2023-08-25T03:36:24.933 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112356.log 2023-08-25T03:36:24.933 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110626.log 2023-08-25T03:36:24.933 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.112356.log: /var/log/ceph/ceph-client.admin.110626.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.110626.log.gz 2023-08-25T03:36:24.933 INFO:teuthology.orchestra.run.smithi002.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.112356.log.gz 2023-08-25T03:36:24.933 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110100.log 2023-08-25T03:36:24.934 INFO:teuthology.orchestra.run.smithi002.stderr: 95.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-08-25T03:36:24.934 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119184.log 2023-08-25T03:36:24.934 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.110100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117029.log 2023-08-25T03:36:24.934 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.119184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117889.log 2023-08-25T03:36:24.934 INFO:teuthology.orchestra.run.smithi002.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.110100.log.gz 2023-08-25T03:36:24.935 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.117029.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.119184.log.gz 2023-08-25T03:36:24.935 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109559.log 2023-08-25T03:36:24.935 INFO:teuthology.orchestra.run.smithi002.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.117029.log.gz 2023-08-25T03:36:24.935 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116067.log 2023-08-25T03:36:24.935 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.117889.log: /var/log/ceph/ceph-client.admin.109559.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117889.log.gz 2023-08-25T03:36:24.936 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.116067.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.109559.log.gz 2023-08-25T03:36:24.936 INFO:teuthology.orchestra.run.smithi002.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116067.log.gz 2023-08-25T03:36:24.936 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-08-25T03:36:24.936 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114415.log 2023-08-25T03:36:24.936 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110054.log 2023-08-25T03:36:24.937 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.114415.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.114415.log.gz 2023-08-25T03:36:24.937 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.110054.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110054.log.gz 2023-08-25T03:36:24.937 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116385.log 2023-08-25T03:36:24.937 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115508.log 2023-08-25T03:36:24.937 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.116385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116845.log 2023-08-25T03:36:24.938 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.115508.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.116385.log.gz 2023-08-25T03:36:24.938 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.116845.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.115508.log.gz 2023-08-25T03:36:24.938 INFO:teuthology.orchestra.run.smithi002.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.116845.log.gz 2023-08-25T03:36:24.938 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115554.log 2023-08-25T03:36:24.938 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109375.log 2023-08-25T03:36:24.938 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.115554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117075.log 2023-08-25T03:36:24.939 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.109375.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115554.log.gz 2023-08-25T03:36:24.939 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.117075.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.109375.log.gz 2023-08-25T03:36:24.939 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117075.log.gz 2023-08-25T03:36:24.939 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115197.log 2023-08-25T03:36:24.939 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120073.log 2023-08-25T03:36:24.940 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.115197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111339.log 2023-08-25T03:36:24.940 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.120073.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.115197.log.gz 2023-08-25T03:36:24.940 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.111339.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.120073.log.gz 2023-08-25T03:36:24.940 INFO:teuthology.orchestra.run.smithi002.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111339.log.gz 2023-08-25T03:36:24.940 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114783.log 2023-08-25T03:36:24.941 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119981.log 2023-08-25T03:36:24.941 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.114783.log: /var/log/ceph/ceph-client.admin.119981.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.114783.log.gz 2023-08-25T03:36:24.941 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.119981.log.gz 2023-08-25T03:36:24.941 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116799.log 2023-08-25T03:36:24.941 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112171.log 2023-08-25T03:36:24.942 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.116799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116431.log 2023-08-25T03:36:24.942 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.112171.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.116799.log.gz 2023-08-25T03:36:24.942 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.116431.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112171.log.gz 2023-08-25T03:36:24.942 INFO:teuthology.orchestra.run.smithi002.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.116431.log.gz 2023-08-25T03:36:24.942 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119368.log 2023-08-25T03:36:24.943 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111572.log 2023-08-25T03:36:24.943 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.119368.log: /var/log/ceph/ceph-client.admin.111572.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119368.log.gz 2023-08-25T03:36:24.943 INFO:teuthology.orchestra.run.smithi002.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111572.log.gz 2023-08-25T03:36:24.943 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115642.log 2023-08-25T03:36:24.943 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110997.log 2023-08-25T03:36:24.944 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.115642.log: /var/log/ceph/ceph-client.admin.110997.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115642.log.gz 2023-08-25T03:36:24.944 INFO:teuthology.orchestra.run.smithi002.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.110997.log.gz 2023-08-25T03:36:24.944 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115975.log 2023-08-25T03:36:24.944 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111802.log 2023-08-25T03:36:24.944 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.115975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109197.log 2023-08-25T03:36:24.945 INFO:teuthology.orchestra.run.smithi002.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115975.log.gz/var/log/ceph/ceph-client.admin.111802.log: 2023-08-25T03:36:24.945 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.109197.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.111802.log.gz 2023-08-25T03:36:24.945 INFO:teuthology.orchestra.run.smithi002.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109197.log.gz 2023-08-25T03:36:24.945 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119594.log 2023-08-25T03:36:24.945 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118307.log 2023-08-25T03:36:24.946 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.119594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120395.log 2023-08-25T03:36:24.946 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.118307.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.119594.log.gz 2023-08-25T03:36:24.946 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.120395.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.118307.log.gz 2023-08-25T03:36:24.946 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115734.log 2023-08-25T03:36:24.946 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120395.log.gz 2023-08-25T03:36:24.947 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110621.log 2023-08-25T03:36:24.947 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118445.log 2023-08-25T03:36:24.947 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.115734.log: /var/log/ceph/ceph-client.admin.110621.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.115734.log.gz 2023-08-25T03:36:24.947 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.118445.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110621.log.gz 2023-08-25T03:36:24.947 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118445.log.gz 2023-08-25T03:36:24.947 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114737.log 2023-08-25T03:36:24.948 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111986.log 2023-08-25T03:36:24.948 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.114737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.112992.log 2023-08-25T03:36:24.948 INFO:teuthology.orchestra.run.smithi002.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.114737.log.gz 2023-08-25T03:36:24.948 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.111986.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111986.log.gz 2023-08-25T03:36:24.948 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.0.112992.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.112992.log.gz 2023-08-25T03:36:24.949 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110376.log 2023-08-25T03:36:24.949 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121565.log 2023-08-25T03:36:24.949 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.110376.log: /var/log/ceph/ceph-client.admin.121565.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110376.log.gz 2023-08-25T03:36:24.949 INFO:teuthology.orchestra.run.smithi002.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.121565.log.gz 2023-08-25T03:36:24.949 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115059.log 2023-08-25T03:36:24.950 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118261.log 2023-08-25T03:36:24.950 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.115059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115688.log 2023-08-25T03:36:24.950 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115059.log.gz 2023-08-25T03:36:24.950 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.118261.log: /var/log/ceph/ceph-client.admin.115688.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.118261.log.gz 2023-08-25T03:36:24.950 INFO:teuthology.orchestra.run.smithi002.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115688.log.gz 2023-08-25T03:36:24.951 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109467.log 2023-08-25T03:36:24.951 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111756.log 2023-08-25T03:36:24.951 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.109467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110238.log 2023-08-25T03:36:24.951 INFO:teuthology.orchestra.run.smithi002.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109467.log.gz 2023-08-25T03:36:24.951 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.111756.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.111756.log.gz 2023-08-25T03:36:24.952 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119797.log 2023-08-25T03:36:24.952 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.110238.log: /var/log/ceph/ceph-client.admin.119797.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110238.log.gz 2023-08-25T03:36:24.952 INFO:teuthology.orchestra.run.smithi002.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119797.log.gz 2023-08-25T03:36:24.952 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121657.log 2023-08-25T03:36:24.952 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111894.log 2023-08-25T03:36:24.953 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.121657.log: /var/log/ceph/ceph-client.admin.111894.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.121657.log.gz 2023-08-25T03:36:24.953 INFO:teuthology.orchestra.run.smithi002.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111894.log.gz 2023-08-25T03:36:24.953 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.118840.log 2023-08-25T03:36:24.953 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121105.log 2023-08-25T03:36:24.953 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.0.118840.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.118840.log.gz 2023-08-25T03:36:24.953 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.121105.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.121105.log.gz 2023-08-25T03:36:24.954 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115826.log 2023-08-25T03:36:24.954 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117983.log 2023-08-25T03:36:24.954 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.115826.log: /var/log/ceph/ceph-client.admin.117983.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.115826.log.gz 2023-08-25T03:36:24.954 INFO:teuthology.orchestra.run.smithi002.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117983.log.gz 2023-08-25T03:36:24.954 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112124.log 2023-08-25T03:36:24.955 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115105.log 2023-08-25T03:36:24.955 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.112124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114553.log 2023-08-25T03:36:24.955 INFO:teuthology.orchestra.run.smithi002.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112124.log.gz 2023-08-25T03:36:24.955 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.115105.log: /var/log/ceph/ceph-client.admin.114553.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.115105.log.gz 2023-08-25T03:36:24.955 INFO:teuthology.orchestra.run.smithi002.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.114553.log.gz 2023-08-25T03:36:24.956 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121427.log 2023-08-25T03:36:24.956 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118077.log 2023-08-25T03:36:24.956 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.121427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116113.log 2023-08-25T03:36:24.956 INFO:teuthology.orchestra.run.smithi002.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.121427.log.gz 2023-08-25T03:36:24.956 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.118077.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.118077.log.gz 2023-08-25T03:36:24.957 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.116113.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.116113.log.gz 2023-08-25T03:36:24.957 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117797.log 2023-08-25T03:36:24.957 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110330.log 2023-08-25T03:36:24.957 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.117797.log: /var/log/ceph/ceph-client.admin.110330.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.117797.log.gz 2023-08-25T03:36:24.957 INFO:teuthology.orchestra.run.smithi002.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.110330.log.gz 2023-08-25T03:36:24.958 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115462.log 2023-08-25T03:36:24.958 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119322.log 2023-08-25T03:36:24.958 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.115462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120211.log 2023-08-25T03:36:24.958 INFO:teuthology.orchestra.run.smithi002.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.115462.log.gz 2023-08-25T03:36:24.958 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.119322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117186.log 2023-08-25T03:36:24.959 INFO:teuthology.orchestra.run.smithi002.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119322.log.gz 2023-08-25T03:36:24.959 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.120211.log: /var/log/ceph/ceph-client.admin.117186.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120211.log.gz 2023-08-25T03:36:24.959 INFO:teuthology.orchestra.run.smithi002.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117186.log.gz 2023-08-25T03:36:24.959 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116937.log 2023-08-25T03:36:24.959 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112510.log 2023-08-25T03:36:24.959 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.116937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109822.log 2023-08-25T03:36:24.960 INFO:teuthology.orchestra.run.smithi002.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.116937.log.gz 2023-08-25T03:36:24.960 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.112510.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.112510.log.gz 2023-08-25T03:36:24.960 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.109822.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109822.log.gz 2023-08-25T03:36:24.960 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-08-25T03:36:24.960 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118607.log 2023-08-25T03:36:24.961 INFO:teuthology.orchestra.run.smithi002.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-08-25T03:36:24.961 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109653.log 2023-08-25T03:36:24.961 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.118607.log: /var/log/ceph/ceph-client.admin.109653.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.118607.log.gz 2023-08-25T03:36:24.961 INFO:teuthology.orchestra.run.smithi002.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109653.log.gz 2023-08-25T03:36:24.961 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117592.log 2023-08-25T03:36:24.962 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111227.log 2023-08-25T03:36:24.962 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.117592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118353.log 2023-08-25T03:36:24.962 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.111227.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117592.log.gz 2023-08-25T03:36:24.962 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.118353.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111227.log.gz 2023-08-25T03:36:24.962 INFO:teuthology.orchestra.run.smithi002.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.118353.log.gz 2023-08-25T03:36:24.963 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120716.log 2023-08-25T03:36:24.963 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120783.log 2023-08-25T03:36:24.963 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.120716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113222.log 2023-08-25T03:36:24.963 INFO:teuthology.orchestra.run.smithi002.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.120716.log.gz 2023-08-25T03:36:24.963 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.120783.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.120783.log.gz 2023-08-25T03:36:24.964 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109421.log 2023-08-25T03:36:24.964 INFO:teuthology.orchestra.run.smithi002.stderr:gzip /var/log/ceph/ceph-client.admin.113222.log: -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-08-25T03:36:24.964 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.109421.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.113222.log.gz 2023-08-25T03:36:24.964 INFO:teuthology.orchestra.run.smithi002.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109421.log.gz 2023-08-25T03:36:24.965 INFO:teuthology.orchestra.run.smithi031.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-08-25T03:36:24.966 INFO:teuthology.orchestra.run.smithi031.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-08-25T03:36:24.966 INFO:teuthology.orchestra.run.smithi031.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-08-25T03:36:24.966 INFO:teuthology.orchestra.run.smithi031.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-08-25T03:36:24.967 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103321.log 2023-08-25T03:36:24.968 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.96722.log 2023-08-25T03:36:24.968 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.103321.log: /var/log/ceph/ceph.tmp-client.admin.96722.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.103321.log.gz 2023-08-25T03:36:24.968 INFO:teuthology.orchestra.run.smithi002.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.96722.log.gz 2023-08-25T03:36:24.968 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120487.log 2023-08-25T03:36:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116707.log 2023-08-25T03:36:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.120487.log: /var/log/ceph/ceph-client.admin.116707.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.120487.log.gz 2023-08-25T03:36:24.969 INFO:teuthology.orchestra.run.smithi002.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.116707.log.gz 2023-08-25T03:36:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116615.log 2023-08-25T03:36:24.969 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118950.log 2023-08-25T03:36:24.970 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.116615.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116615.log.gz 2023-08-25T03:36:24.970 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.118950.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.118950.log.gz 2023-08-25T03:36:24.970 INFO:teuthology.orchestra.run.smithi002.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119751.log 2023-08-25T03:36:24.970 INFO:teuthology.orchestra.run.smithi002.stderr:/var/log/ceph/ceph-client.admin.119751.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119751.log.gz 2023-08-25T03:36:24.971 INFO:teuthology.orchestra.run.smithi002.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-08-25T03:36:24.975 INFO:teuthology.orchestra.run.smithi031.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-08-25T03:36:24.976 INFO:teuthology.orchestra.run.smithi002.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-08-25T03:36:24.986 INFO:teuthology.orchestra.run.smithi002.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-08-25T03:36:24.994 INFO:teuthology.orchestra.run.smithi031.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-08-25T03:36:24.995 INFO:teuthology.orchestra.run.smithi031.stderr: 2023-08-25T03:36:24.995 INFO:teuthology.orchestra.run.smithi031.stderr:real 0m0.175s 2023-08-25T03:36:24.996 INFO:teuthology.orchestra.run.smithi031.stderr:user 0m0.743s 2023-08-25T03:36:24.996 INFO:teuthology.orchestra.run.smithi031.stderr:sys 0m0.047s 2023-08-25T03:36:25.005 INFO:teuthology.orchestra.run.smithi002.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-08-25T03:36:25.105 INFO:teuthology.orchestra.run.smithi002.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-08-25T03:36:25.106 INFO:teuthology.orchestra.run.smithi002.stderr: 2023-08-25T03:36:25.107 INFO:teuthology.orchestra.run.smithi002.stderr:real 0m0.312s 2023-08-25T03:36:25.107 INFO:teuthology.orchestra.run.smithi002.stderr:user 0m0.928s 2023-08-25T03:36:25.107 INFO:teuthology.orchestra.run.smithi002.stderr:sys 0m0.167s 2023-08-25T03:36:25.108 INFO:tasks.ceph:Archiving logs... 2023-08-25T03:36:25.108 DEBUG:teuthology.misc:Transferring archived files from smithi002:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379011/remote/smithi002/log 2023-08-25T03:36:25.112 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /var/log/ceph -- . 2023-08-25T03:36:25.533 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379011/remote/smithi031/log 2023-08-25T03:36:25.534 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/log/ceph -- . 2023-08-25T03:36:25.650 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-08-25T03:36:25.675 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-08-25T03:36:25.676 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-08-25T03:36:25.679 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-08-25T03:36:25.755 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-08-25T03:36:25.770 INFO:teuthology.task.clock:Checking final clock skew... 2023-08-25T03:36:25.771 DEBUG:teuthology.orchestra.run.smithi002:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-25T03:36:25.776 DEBUG:teuthology.orchestra.run.smithi031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-08-25T03:36:25.788 INFO:teuthology.orchestra.run.smithi002.stderr:bash: ntpq: command not found 2023-08-25T03:36:25.794 INFO:teuthology.orchestra.run.smithi002.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-25T03:36:25.794 INFO:teuthology.orchestra.run.smithi002.stdout:=============================================================================== 2023-08-25T03:36:25.795 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 28 +1772us[+1772us] +/- 75ms 2023-08-25T03:36:25.795 INFO:teuthology.orchestra.run.smithi002.stdout:^* hv02.front.sepia.ceph.com 3 6 377 30 -2138us[-2132us] +/- 73ms 2023-08-25T03:36:25.796 INFO:teuthology.orchestra.run.smithi002.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 +615us[ +615us] +/- 90ms 2023-08-25T03:36:25.796 INFO:teuthology.orchestra.run.smithi002.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T03:36:25.797 INFO:teuthology.orchestra.run.smithi031.stderr:bash: ntpq: command not found 2023-08-25T03:36:25.797 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-08-25T03:36:25.798 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2023-08-25T03:36:25.798 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 29 +1785us[+1800us] +/- 75ms 2023-08-25T03:36:25.799 INFO:teuthology.orchestra.run.smithi031.stdout:^* hv02.front.sepia.ceph.com 3 6 377 29 -2125us[-2109us] +/- 73ms 2023-08-25T03:36:25.799 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 30 +622us[ +638us] +/- 90ms 2023-08-25T03:36:25.800 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-08-25T03:36:25.801 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-08-25T03:36:26.041 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-08-25T03:36:26.042 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-08-25T03:36:26.083 DEBUG:teuthology.orchestra.run.smithi002:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-08-25T03:36:26.187 DEBUG:teuthology.orchestra.run.smithi031:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-08-25T03:36:26.287 DEBUG:teuthology.orchestra.run.smithi002:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-25T03:36:26.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T03:36:26.320 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-08-25T03:36:26.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T03:36:26.355 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-08-25T03:36:26.366 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-08-25T03:36:26.460 INFO:teuthology.task.internal:Duration was 957.471822 seconds 2023-08-25T03:36:26.461 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-08-25T03:36:26.673 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-08-25T03:36:26.673 DEBUG:teuthology.orchestra.run.smithi002:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-08-25T03:36:26.677 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-08-25T03:36:26.726 INFO:teuthology.orchestra.run.smithi002.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-25T03:36:26.730 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-08-25T03:36:27.121 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-08-25T03:36:27.122 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi002.front.sepia.ceph.com 2023-08-25T03:36:27.122 DEBUG:teuthology.orchestra.run.smithi002:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-08-25T03:36:27.147 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi031.front.sepia.ceph.com 2023-08-25T03:36:27.148 DEBUG:teuthology.orchestra.run.smithi031:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-08-25T03:36:27.176 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-08-25T03:36:27.177 DEBUG:teuthology.orchestra.run.smithi002:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-08-25T03:36:27.190 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-08-25T03:36:27.263 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-08-25T03:36:27.264 DEBUG:teuthology.orchestra.run.smithi002:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-08-25T03:36:27.267 DEBUG:teuthology.orchestra.run.smithi031:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-08-25T03:36:27.349 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-08-25T03:36:27.364 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-08-25T03:36:27.365 DEBUG:teuthology.orchestra.run.smithi002:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-08-25T03:36:27.393 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-08-25T03:36:27.423 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-08-25T03:36:27.619 DEBUG:teuthology.orchestra.run.smithi002:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-08-25T03:36:27.622 DEBUG:teuthology.orchestra.run.smithi031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-08-25T03:36:27.647 INFO:teuthology.orchestra.run.smithi002.stdout:kernel.core_pattern = core 2023-08-25T03:36:27.650 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = core 2023-08-25T03:36:27.665 DEBUG:teuthology.orchestra.run.smithi002:> test -e /home/ubuntu/cephtest/archive/coredump 2023-08-25T03:36:27.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T03:36:27.719 DEBUG:teuthology.orchestra.run.smithi031:> test -e /home/ubuntu/cephtest/archive/coredump 2023-08-25T03:36:27.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-08-25T03:36:27.737 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-08-25T03:36:27.753 INFO:teuthology.task.internal:Transferring archived files... 2023-08-25T03:36:27.754 DEBUG:teuthology.misc:Transferring archived files from smithi002:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379011/remote/smithi002 2023-08-25T03:36:27.755 DEBUG:teuthology.orchestra.run.smithi002:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-08-25T03:36:27.796 DEBUG:teuthology.misc:Transferring archived files from smithi031:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379011/remote/smithi031 2023-08-25T03:36:27.797 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-08-25T03:36:27.838 INFO:teuthology.task.internal:Removing archive directory... 2023-08-25T03:36:27.839 DEBUG:teuthology.orchestra.run.smithi002:> rm -rf -- /home/ubuntu/cephtest/archive 2023-08-25T03:36:27.841 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf -- /home/ubuntu/cephtest/archive 2023-08-25T03:36:27.887 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-08-25T03:36:27.903 INFO:teuthology.task.internal:Not uploading archives. 2023-08-25T03:36:27.904 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-08-25T03:36:27.952 INFO:teuthology.task.internal:Tidying up after the test... 2023-08-25T03:36:27.953 DEBUG:teuthology.orchestra.run.smithi002:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-08-25T03:36:27.955 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-08-25T03:36:27.970 INFO:teuthology.orchestra.run.smithi002.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 25 03:36 /home/ubuntu/cephtest 2023-08-25T03:36:27.974 INFO:teuthology.orchestra.run.smithi031.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Aug 25 03:36 /home/ubuntu/cephtest 2023-08-25T03:36:27.975 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-08-25T03:36:27.986 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-08-25T03:36:28.013 INFO:teuthology.nuke:Checking targets against current locks 2023-08-25T03:36:28.042 DEBUG:teuthology.nuke:shortname: smithi002 2023-08-25T03:36:28.043 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-25T03:36:28.068 DEBUG:teuthology.nuke:shortname: smithi031 2023-08-25T03:36:28.068 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-08-25T03:36:28.072 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi002.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379011', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-25 03:15:23.904500', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-08-25T03:36:28.092 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-08-25_01:50:32-fs-wip-rishabh-2023aug3-b5-testing-default-smithi/7379011', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-08-25 03:15:23.906528', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-08-25T03:36:28.131 INFO:teuthology.orchestra.console.smithi002:Power off 2023-08-25T03:36:28.132 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-25T03:36:28.155 DEBUG:teuthology.orchestra.console.smithi002:power off output: Chassis Power Control: Down/Off 2023-08-25T03:36:28.156 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T03:36:28.174 INFO:teuthology.orchestra.console.smithi031:Power off 2023-08-25T03:36:28.175 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-08-25T03:36:28.188 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is on 2023-08-25T03:36:28.199 DEBUG:teuthology.orchestra.console.smithi031:power off output: Chassis Power Control: Down/Off 2023-08-25T03:36:28.200 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T03:36:28.224 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2023-08-25T03:36:32.190 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T03:36:32.225 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T03:36:32.304 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is on 2023-08-25T03:36:32.343 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is on 2023-08-25T03:36:36.309 DEBUG:teuthology.orchestra.console.smithi002:pexpect command: ipmitool -H smithi002.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T03:36:36.344 DEBUG:teuthology.orchestra.console.smithi031:pexpect command: ipmitool -H smithi031.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-08-25T03:36:36.429 DEBUG:teuthology.orchestra.console.smithi002:check power output: Chassis Power is off 2023-08-25T03:36:36.461 DEBUG:teuthology.orchestra.console.smithi031:check power output: Chassis Power is off 2023-08-25T03:36:36.530 INFO:teuthology.orchestra.console.smithi002:Power off completed 2023-08-25T03:36:36.562 INFO:teuthology.orchestra.console.smithi031:Power off completed 2023-08-25T03:36:36.669 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/quota} duration: 957.4718220233917 failure_reason: 'Test failure: test_disable_enable_human_readable_quota_values (tasks.cephfs.test_quota.TestQuota)' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=9cb51bb072ba4b52946cc12ca6f4c5ef status: fail success: false 2023-08-25T03:36:36.670 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-08-25T03:36:36.754 INFO:teuthology.run:FAIL